I go to my fair share of conferences and have noticed that all of the best and most useful things are said in the coffee breaks, in the bar or…basically anywhere outside of the main, organised programme. Alongside this I’ve long lamented the, often ridiculous, range of digital things that people in arts organisations are expected to be experts on and the lack of honest discussion around what success looks like and how people got there.
So with that in mind we (Substrakt) decided to try and remedy this by organising an event that aimed to create a space where these valuable, peer-to-peer discussions could happen, examining the broad range of things that arts folk are expected to make happen on a daily basis. This included procurement, outsourcing, analytics, user testing, content creation and distribution, connecting digital and physical experiences and ended with a look at chat bots, automation and AI.
Rather than writing a blow-by-blow account of the day I’ve tried to highlight the most useful/interesting elements of each discussion below.
- Session 1: Digital as a business function – procurement
- Session 2: Digital as a business function – when to outsource
- Session 3: Analytics, data and agility – analytics and user testing
- Session 4: The digital user experience – Content creation and distribution
- Session 5: The digital user experience – Connecting physical and digital experiences
- Session 6: Future focus – bots, AI and automation in an arts context
- Final thoughts
Throughout the day we tried to return to 3 key questions:
What does success look like?
What’s the value?
What are the obstacles?
Session 1: Digital as a business function – procurement
I often speak to friends and colleagues in the arts sector who struggle with the difficulties of digital procurement, there seems to be very little in the way of shared knowledge on the subject and we, as an agency who are on the other side of the process, frequently see lots of the same missteps being made.
Clarity & Honesty: Broadly people working in arts orgs said they are looking for more honesty and clarity from agencies – an admission or frankness from agencies that perhaps not all of a brief may be achievable for the budget available, or things may have to take longer than anticipated. People also said they wanted it to be explained with much more certainty exactly what the money was being spent on. Communication in general was also identified as an area for improvement on all sides. There was also an admission that arts organisations need to be more honest with themselves about what their priorities are and be able to communicate that more clearly through briefs.
Internal politics: Often big digital projects can highlight, or exacerbate, internal tensions in organisations. From an agency’s point of view it is preferable if they aren’t being used as the tool to solve those tensions and if buy-in from all parts of the organisation is already secured by the time an agency is appointed. A couple of people who have gone through this process a few times said this can often be achieved by doing a lot of the initial thinking before the procurement process begins – involving as many colleagues as you can, as early as is appropriate when writing a brief. People had also seen success in adopting a phased approach to delivery so that not everyone was expecting everything to be there on the initial go-live.
Mix it up, when appropriate: It seemed everyone agreed it was preferable to build long term relationships rather than changing suppliers on a project-by-project basis. However there is also the danger that this can breed complacency – on both sides. It may be that you have a ‘core’ ongoing relationship and you use those people to advise/input on other ‘project-specific’ relationships you may look to develop. This ensures you are still getting quality, ongoing support and input from a group of people who know you well and can advise you from an informed point of view but you are also benefiting from a regular injection of new ideas and fresh perspectives.
Session 2: Digital as a business function – when to outsource
Approach: There seemed to broadly be 3 different types of approach; 1) Outsource all specialised skills (in relation to digital this might include development, content production, strategy, etc) and call on specialist external freelancers/agencies as and when they’re needed, 2) Outsource ‘production’ (i.e. “the doing”) but retain some in-house expertise to act as a central point to manage the outsourcing and provide a degree of in-house capacity, 3) Build specialist, in-house capacity, use very few – or no – outsourcing. It was felt that option 3 was really only feasible for larger organisations or those with significant financial resources to call on. There also seemed to be a feeling that digital is evolving, developing and fragmenting at such a rapid speed that it was difficult to build in-house capacity that matched or exceeded the expertise that was available by working with external suppliers.
Video is an outlier: An area where a number of organisations had seen success with bringing capacity in-house was around content production, specifically video. The main driver for bringing this totally in-house was usually budget. It was also felt that bringing this specific skill in-house gave you easily understandable and usable options to experiment and try things (without being restrained by cost).
Capacity: It was widely acknowledged that inevitably the more complex (and therefore interesting/fun) projects are often those that are outsourced, inevitably this is because in-house staff don’t have the capacity to take these projects on. Discussions raised the prospect of outsourcing some of the day-to-day (‘boring’) things to free up room to tackle (or upskill so you can tackle) these more complex projects. The benefits of this are obvious – staff develop, expertise is grown in-house, etc although this approach may take longer initially it is perhaps a more strategic way to address this issue.
Rationale: Almost unanimously the deciding factor on whether or not to outsource seemed to be one of cost – if it made sense to keep something in-house then people would and could, otherwise they’d look externally. However there was also an acknowledgment and discussion of the other implications around the benefits of in-house/outsourcing; namely that outsourcing exposes you to expertise, contacts, knowledge and ideas that you may not otherwise be able to access whereas building in-house teams allows you to build the capacity to experiment and try things out without having to be as constrained by cost concerns, it also enables you to focus and develop your teams’ skills very specifically on meeting your organisation’s needs.
Session 3: Analytics, data and agility – analytics and user testing
This session was lead by One Further’s Chris Unitt who is always worth listening to on this subject.
Google Analytics is awful: A widespread admission that GA was felt to be baffling, overwhelming and unnecessarily complex. Chris recommended using dashboards and custom reports so you don’t have to delve into the murky depths too often.
Use a range of tools: Consider a mix of tools; usabilityhub.com, optimalworkshop.com, in-house user testing, heatmapping (hotjar, crazy egg) are all worth considering to build a well-rounded range of insights. Having a range of outputs (visual, text and numerical) may also help you make a stronger case internally to people who perhaps don’t respond well to ‘just stats’.
Context is key: Always be aware of the context in which the insights/numbers exist e.g. if you have an extraordinary event one week, any meaningful comparison with that week (or using that week in any benchmarking) is going to be impossible and unhelpful.
Analytics should be organisation-wide: Currently it’s often the case that analytics, data and insights are seen as ‘marketing things’. There’s a lot of value that other parts of an organisation can enjoy by better understanding their ‘digital audience’ but some people felt that historically the people who run these departments aren’t always open to being persuaded by data-based arguments.
Don’t try to measure everything: Although you can measure everything that doesn’t mean you should (as Jeff Goldblum once said…), work out what you want or need to know and identify the most useful way to report on that.
Be prepared to act on your findings: A couple of attendees had really useful examples of where they had changed fundamental parts of the user experience (in one example this involved entirely getting rid of the homepage) in response to insights they’d drawn from their analytics setup. Although inevitably this will require a working culture in which the value of data, and data-based decision-making is recognised.
Session 4: The digital user experience – Content creation and distribution
Content or audience first?: Arts organisations create a lot of content – how much of this is created strategically or for a specific purpose and how much is created just to meet an internal expectation? To interrogate this Ammba’s Robbie Beak examined the two broad approaches you can take to content production: audience-first (i.e. you identify what your audience wants, and you make it for them) or content-first (i.e. you work out who might like your content and you go and find them). Inevitably the ‘right’ answer falls somewhere between the two but it was interesting to see people confronting these two ways of thinking about content creation in a black and white way, on the whole the discussions seemed to agree that when the content was a means to an end (e.g. a trailer for a show, articles about a director, interviews with performers etc) it made sense to produce things that your audience wanted to see. However if you are an organisation for whom the content is the point then you shouldn’t be afraid of following a vision and then seeking an audience – although with the specific examples and case studies that were discussed it was widely experienced that this second approach would usually result in a smaller audience (although this audience may be more engaged).
Channels: Everyone agreed it was probably best to pursue an approach of ‘fishing where the fish are’ rather than trying to reinvent the wheel. However it was stressed that specific channels have specific expectations and demands when it comes to content, what works well on one will not necessarily work well on all. Several examples were also given where organisations had seen success by limiting the number of channels they were active on. Arts organisation resources being what they are (limited and stretched) there has to be some honesty that noone (or at least, very few people) can realistically be active across the full range of channels and remain effective.
Session 5: The digital user experience – Connecting physical and digital experiences
What’s the value?: Broadly ‘useful’ enhancements to the physical experience through digital were welcomed. However this was not really most people’s experience; “gimmicky”, “faffy”, “pointless” and “annoying” were just some of the words used to describe efforts in this direction so far. The value to the user has to be clear and immediate, people are impatient – if a digital aspect has been developed without the user being at the heart of everything then, inevitably, it isn’t going to be successful – the Cooper Hewitt ‘pens’ (with which you can interact with touchtables and ‘tag’ objects during your visit, you can then access your list – after your visit – with additional info about all of the objects you tagged) were identified as a good example of this, easy to use, of value to the user and unintrusive.
Contextual enhancement: Following on from the ‘useful’ point raised above, discussions seemed to settle on location/time-based ‘enhancements’ as the most obvious ways to add useful value to a physical experience. There was less discussion about interpreting a physical experience into a digital one for a digital-only audience, but this is definitely an area I’d be keen to explore in the future. I think there is an interesting discussion to be had around things like live broadcast and the experience of the audience on the receiving end of the broadcast – but these are thoughts for another day!
Social norms: When the issue of ‘phones in theatres’ was discussed almost everyone was against it, although there was recognition that this may simply be because the idea of experiencing a live performance with the aid of, or through, a device was still new (and therefore “a bit yucky”). In 10, 20 or 30 years it’s likely the digital will be so embedded in people’s day-to-day lives and lived experiences that to not be able to use a device during a performance may be the oddity. It’ll be interesting to see how attitudes towards this develop in the coming years.
Technology: A ‘Google Glass-like’ experience was proposed as the most seamless way to implement digital enhancements to a physical experience – although noone was saying Google Glass was a particular success, that type of ‘straight to vision’ augmented reality would be a way to meaningfully enhance museum visits, introductions to new artforms etc. Although obviously there are accessibility considerations with tech like that.
Differentiation: There was a discussion around how much arts organisations should try to differentiate themselves when it came to UX. Of course every arts organisation will have some unique aspect but should that translate into user experiences and interfaces that require additional effort/learning-how-to-use on the part of the user just for the sake of being different? Probably not. There are established expectations, norms and best practise when it comes to user experience and arts organisations need to be aware of those and that their (arts organisation’s) digital products sit within a landscape – users don’t think “I’m using an arts website/app/whatever so I’m happy to spend 5 minutes working out why it doesn’t resemble anything I’ve ever seen or used before”, they’re not happy to do so, and they won’t.
Session 6: Future focus – bots, AI and automation in an arts context
We wanted to end the day with a bit of a look at something we think may begin to impact the arts sector in the not-so-distant future. Namely the growing prevalence of chat bots (in various forms). This discussion was led by one of our senior developers, Sam Knight. Sam kicked things off by looking at the rapid adoption of touchscreens (to the point where it’s now an expected part of most screen interactions) and asked whether or not chat bots would quickly reach the same point and, if so, how organisations should be responding to this challenge. Sam has written an introductory post about the subject you can read here.
Cynicism: The overall feeling was one of cynicism, would this really become something that customers would come to expect (e.g. asking Siri to order you some tickets to the opera)? Certainly recent research (“80% of UK businesses want chat bots by 2020“) indicates this as an area where other sectors are expecting to make rapid advancements so it is not unreasonable to think that the arts sector will experience a knock-on effect as a result.
Complexity: An issue we’re well aware of is the basic complexity of the ticketing-purchasing process in some scenarios; the plethora or performances, seats, prices, concessions, etc mean there are a lot of combinations you need to get a system to understand (e.g. if the user asks for tickets are they implying they want best available unless they specify otherwise), at the moment there are lots of libraries that can help you build chat bots but they are aiming to solve relatively straightforward queries (Sam’s post goes into this in more detail).
Usage: The most obvious use that was discussed was one of augmenting customer service. However there was also some interesting discussion around the delivery of an artistic experience e.g. chatbot as narrator.
I’m really pleased with how the day went, and the feedback we’ve had has indicated that the attendees felt the same. There were some fantastic conversations about a, frankly ridiculous, range of topics. We will be planning more of these, future events will be more focused (on fewer things per event) but I wanted to kick things off by looking at the range of things (from procurement through to analytics through to content through to AI) that digital bods in the arts have to consider.
Anyways, if you’d like to come to future Digital Works events then drop us a line (firstname.lastname@example.org) or watch this space for news on the next one.
Thanks to everyone who came and got so wholeheartedly involved in the discussions, it was great to be able to hear from such a diverse range of organisations and backgrounds. Special thanks to Lucy and Reuben and all the guys at Hackney Empire for being such brilliant hosts.
Let us know what you’d like to talk about at the next digital works via this short survey.
Until next time!