Last year, I took the year off from writing about conferences because I needed to retool how I thought about them and, honestly, I was just burnt out at the number I was attending and I didn’t want to write that many posts.
But I’m back.
I’m also reinstating my conference scorecard. If you don’t remember it, I look at the components of the event itself and give each component and the overall conference a letter grade. I used to call it the conference report card, but that made me sound like I’m a third grade teacher, and that the event planners and participants are kids I’m grading. That is just so wrong.
I admire and am amazed at the conference organizers and event planners. I’m in awe of them. What they have to do is just staggering. Think about it. They have to pull off a conference that involves thousands of attendees and make them all happy and feel like their attendance was productive and thus, worth the days they spent. Plus they have to babysit their VIPS, be they analysts, journalists, or executives of companies who are partners or large customers etc. What a task. Monumental… and the effort is generally under-appreciated.
The value of these technology vendor conferences to the company putting on the show is immeasurable. They are nodal points, inflection points, not just in the history of the company, but in the feelings that a customer, prospect or observer will have over a year. Conferences determine how someone feels about a company from one year to the next.
Think of it this way. If I ask you in a year about Adobe’s Digital Marketing Summit and what you remember, it won’t be that much – one or two cogent things maybe – but you will remember that it was good/great/horrible/whatever conference and that you learned “stuff” that will color your view of what you think about Adobe. It’s all about the impression and the content, not just the content. Otherwise, why would Adobe on Day 2 have Nate Silver, Steve Young, Michael Keaton and Wayne Brady? To impart great focused wisdom about Adobe? Adobe might say, they represented “reinvention” as actors and athletes and Nate Silver is data. But there are lots of people who could credibly and interestingly discuss reinvention. But they are also admired celebrities and the idea of a “cool vibe” doesn’t escape the event masters. They want to keep the attendees engaged, and, like it or not, (I kinda like it), they are more likely to do that with famous people than not-famous ones.
Before I tell you what I am rating, there is an express caveat that the ratings are pretty subjective though I do talk to other attendees to get a feel for their thinking on specific aspects of the event. So its not just me alone speaking. However, to be clear, my “methodology” would fail any of the Adobe Analytics maestros legitimacy tests. Just sayin’. Ultimately, this boils down to a set of opinions, nothing more, nothing less.
I rate on nine categories, each heavily weighted. For example, without revealing the weights themselves, its clear that Keynote (Content) is far more important than Food (VIP) and is weighted accordingly. There is a 20-point scoring system that translates into a letter grade and then the weighted total score does the same. The underlying idea is that a conference is more than just a set of announcements and meetings, it is a consumable experience that impacts the customer’s decisions to engage the company and impacts the broader experience (read: feeling) that the customer has with the company. The factors are not the norm of things that analysts cover, but then who ever thought I was normal?
Look, most conferences are minimally well done and occasionally works of art. I appreciate how much goes into it. These are realistically subjective guideposts to how some of the things that go on at a conference worked according to attendees and me that I randomly and not randomly spoke to. So take them for what they are worth. Unlike the Watchlist, this isn’t me the SOB. This is me who appreciates the work of the event planners and managers. But sometimes things work and sometimes they don’t. I’m not a quant but I’m trying to give the experiences I have at a conference a deeper thought and a recognizable way to identify the results.
So here are the categories and what they each cover:
Keynotes (Content) - This is the messaging, the focus, the details, the presentation and in this particular case, the presenters who work for the conference host, all taken into account in the assessment. That means how visionary, how practical, how detailed, what they said to an audience, given its mindset, etc.
Keynotes (Presentation) -This is the rest of the main stage speakers e.g. the celebrities, the lighting and the screens on the stage, the video content shown, the music, the quality of the demos given, and if demos are given,, and the feel of the whole main stage effort, among other things. The reason that the non-keynote speakers (those who don’t work for the host, but are on the stage like at this conference Steve Young or Michael Keaton for example,) are included here is that they are only ancillary to a message not the message purveyor. So “reinvention” was a message that Adobe wanted to get across, Steve Young was asked about reinvention. They are not the prime deliverers.
Tracks/General Sessions - Did the content of the tracks and general sessions cover what the attendees needed and what the host needed to say? Did the titles meet the expectations of the attendees? For example, at CRM Evolution last year, we had a couple of glitches where the titles of the track didn’t really follow the actual content of the presentation. What was the level of the presentations? Did the content get presented in a way that left something for the attendees to chew on?
Analyst/Press Relations - How were the analysts and press treated? Did they have a working environment that allowed them to get out the coverage they needed on the conference? Did they have the materials they needed? Were the AR/PR teams responsive to requests? What was the general environment? Friendly? Standoffish? Neutral? Were the “asks” of the analysts or press met (e.g. one on one meetings) as best as possible (Note: its not always possible to accommodate everyone for everything they ask)?
Food (VIP) - How was the food and drink given to the VIPs which is often different or at least in a different location than the general session?
Food (General Session) - How was the food that was provided for thousands of people? This might not seem to be a matter of concern, but the bulk of the people there paid to go with meals of some level included. The quality and availability of those meals and snacks etc.
Exhibition Hall - The partner pavilions are critical parts of any event. Prospects, customers, analysts, and journalists, get a feel for who is willing to throw their lot in with the hosts of the event. Partner ecosystems are a critical part of a tech company in particular’s offering. So who is represented? How wide are the pathways to walk to make it easy for an attendee to talk to the partner. What about the organization of the hall? The breadth and depth of the partners represented? The quality of the booths, etc.
Crowd Energy - How into the conference is the crowd? This doesn’t mean how loud they are, but how involved are they in the event. There is a palpable energy that is highly noticeable over the course of an event. It has an impact on the residual feelings about the experience. Electricity works.
Ambiance - This is the conference center environment, the room set up, the hallways, the swag that all the attendees are offered, and the concert that may or may not be part of the event, the lighting and the Wi-Fi. All those things that add to a feeling in an environment.