« Former secret meeting place now rubble | Main | Hiring Listeners »
July 11, 2006ROI Backlash?
Posted by David McCann
Responses (11)
Still at MPI's World Education Congress in Dallas. First: I made an error yesterday when I reported that MPI announced it will hold its 2008 WEC in Houston. Actually, MPI will host its 2008 Professional Education Conference, a January event, in Houston. So it won't be as hot (temperature-wise) as I had thought.
Anyway, I promised yesterday to write something today about MPI's focus on ROI. In short, my opinion is that this focus is not widely popular among members.
Many meeting planners believe they should pay attention to measuring the impact of meetings because that is what they've heard they should do. And it makes sense. If there's anything an organization wants from a meeting, it's a return on its investment. And to know if you're getting a return, you need an effective measurement of it.
The problem is, there does not appear, to me, to be any models for measuring ROI that are understandable to the vast majority of the meeting planner audience. That's not to demean planners; but they're planners, not mathemeticians/economists, as the available metrics demand. I believe planners are picking up on this; many who have sat through ROI sessions come away bewildered, and some are coming around to the idea that ROI can't very well be proved, after all. And that's not all; not only are these models not understandable to most meeting planners, it's a valid question as to whether they are workable models at all. I've been covering this industry for 20 years, and there is no consensus on the value of proving ROI, which method is the best, and whether it can be proven at all. The lack of some consensus throws all methods into doubt, I believe.
MPI's focus on ROI says to planners: This is important. You have to know this. If you don't, you're not doing your job. I believe this sets planners up to fail, based on the arguments above.
I know this won't be a popular opinion among MPI's movers and shakers. It does reflect the thoughts of numerous planners, both within and outside of MPI, with whom I have spoken recently.
Respond to article (11)TrackBack
TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8341c5cc553ef00d8342a355353ef
Listed below are links to weblogs that reference ROI Backlash?:
» What's the real ROI? from Face2Face Meetingsnet
David McCann wrote an interesting post on MISoapbox about MPI's recent focus on return on investment, and whether or not that focus is setting planners up to fail (he thinks it isread his post for arguments as to why he thinks this might be). I ... [Read More]
Tracked on Jul 13, 2006 11:26:43 AM
Comments
ROI is a funny thing to measure, Dave - it's not always a financial return and it should be (IMO) always a return on what all the stakeholders gain by holding a meeting. I think we're not "there" yet - that is, planners are still struggling to get their organizations to set meeting goals and objectives before they can do anything on measurements. Suppliers measure ROI for their owners and management companies on busn. booked and dollars. It's not so easy for planners and for meetings.
I want us to find a way to figure out what people got from a meeting - whatever the meeting: a staff meeting, convention, seminar, conference, etc. It would seem everyone would want to know about the success factors. How can we capitalize on them? Build on them? Correct something if we don't know?
Posted by: Joan Eisenstodt | Jul 11, 2006 5:19:05 PM
Dave and Joan,
Perhaps I'm being idealistic here, but why make this an either/or proposition? I agree with the idea that it is important for the industry, somehow and at some time, to find a way to quantitatively prove ROI, as that will show meeting sponsors and those who doubt the value of meetings that there is value in them. Also, no matter how you slice it, those at the top of an organization want to see numbers or they won't keep spending money, period.
But I also agree with Joan's point, that we need qualitative measures of ROI.
Maybe part of the industry's problem is that we're trying to find one easy answer when, in fact, there are lotsa different ways to slice this.
Posted by: Rayna | Jul 12, 2006 10:27:31 AM
What I'm saying is that the ROI models that have been out there to date -- and which have evolved over a many-years period of time -- have not taken hold with the meeting-planner audience, because they're too complicated and there is no evidence that they are valid at all. There is a lot of frustration over this. It may just be that efforts would be better placed in different directions.
Posted by: David McCann | Jul 13, 2006 10:55:57 AM
I think the approach that Jerry McGee from Ambassadors has taken in a recent series of white papers is almost spot-on: Qualitative Return on Objective (ROO) is the best measurement that planners can give their bosses, who from there can weigh that info against other internal and external (market) factors to determine how much the meeting contributed a quantitative ROI.
You can read the white papers on MiMegasite.com, under the "Industry Research" tab.
Posted by: rob carey | Jul 13, 2006 12:36:00 PM
Dave
I agree with you. I believe ROI has been too much of a focus at MPI for too long. While I have taught/presented (college course and at industry events) ROI and I do feel it can be explained in a practical manner so that it actually can be implemented and used to show return, I feel that the MPI "versions" do tend to be complicated and hard to implement. Very few of us teach it from a planner perspective and give "step-by-step" practical examples of how to do it. And, BTW, it really isn't ROI, it's ROO!
Thanks for bringing this up - it's been in/on my mind for a long time!
Posted by: Vicky Betzig, CMP | Jul 13, 2006 1:01:26 PM
Dave is fundamentally correct.
Return on Investment is simply that - the amount of money earned as a result of the money invested in an event. Any other definition is spurious and useless. As a CEO, if Planners cannot justify spend on the events that they are running with a direct connection to revenues, then there will be some firing done.
When an event can be directly linked to sales leads, and through a CRM system linked to actual sales, then ROI can be calculated downstream, or at least trends can be reported which are gold to marketeers and sales managers.
This requires a systemmatic connection of the following, through technology:
Event ==> Sales Lead
Sales Lead ==> Closed Business
Most Meeting Planners are incapable of having any influence over the above through either lack of management or understanding how to put such a system in place, so we're down to measuring other meeting results:
Product or Service Awareness
Customer Satisfaction
Am I missing something? I think the two above are about as far as most Planners can go, but even then most do not have the tools to measure them.
Bottom line?
Meetings and events should produce the same kind of metrics that the Advertising indstry must. That they don't is an indictment of the industry, and certainly the biggest association should be providing leadership (MPI).
Regards,
Ray
Posted by: Ray Thackeray | Jul 13, 2006 1:47:10 PM
It seems to me that many in the industry make the quantitive proof of ROI and the qualitative proof an either/or proposition and I think both can provide value.
I agree with the idea that it is important for the industry, somehow and at some time, to find a way to quantitatively prove ROI, as that will show meeting sponsors and those who doubt the value of meetings that there is value in them. Also, no matter how you slice it, those at the top of an organization want to see numbers or they won't keep spending money, period.
But I also agree with Joan Eisenstodt's point, that we need qualitative measures of ROI.
Maybe part of the industry's problem is that we're trying to find one easy answer when, in fact, there are lotsa different ways to slice this.
Posted by: Rayna | Jul 13, 2006 3:13:54 PM
I have discussed with the leadership of MPI that, although their intentions are correct, their guidance on reporting results from meeting and event activity is misdirected. ROI is not the answer.
The easiest way to demonstrate this is through the analogy of how advertising proves its worth. In the advertising world, they too have an abligation to show/report on the results of their expenditures. They do not report on the ROI of an advertising campaign because it is scientifically impossible to prove that a certain amount of advertising will result in certain amount of sales. Instead, they report on qualitative and quantitative measures of the advertising: Impressions (number of people who saw the ad), CPM (cost to advertise to 1,000 people), Frequency (number of times consumers saw the ad), Reach (% of the market who saw the ad). This information professionally presented serves as justification to advertise in the future because an executive in their company has made a judgement based decision that if we get our target market seeing THIS ad, THIS many times then we will likely achieve our sales goals. Note how important it is to report results but they are not saying these ads generated these particular sales even after the fact.
If we as meeting industry leaders try to leapfrog our qualitative and quantitative reporting against objectives and focus solely on the crown jewel of business measurement, ROI, we will fail and we will maintain a position of political weakness relative to comparable roles within companies and their expenditures.
The answer is to begin reporting against meeting industry objectives, for example the following might apply to a key customer event: Touchpoints (number of times existing or potential customers interacted with salespeople or key executives), Relationships Further Developed (number of times key executives spent meaningful time with current customers at an event), or even Value of Relationships (Volume of sales of Customers in the Relationships Further Developed category).
I am not preaching that this is either easy or even completely matured however I do feel insistent that we will lead the lambs to slaughter if we promote ROI as the answer versus industry-based ROO metrics.
Posted by: Jerry G. McGee | Jul 13, 2006 8:24:33 PM
I think that MPI's intentions on the ROI topic make sense. We each need to look hard at the training and find practical ways to use it to benefit our organizations and our careers.
Training managers within corporations have often struggled to measure ROI on an individual training workshop. Why?...because training is a process, not a one time event. Session evaluations are an indicator, but the benefits of a training session could last for years or be a blip on the screen. Most of the benefit is heavily dependent on the ongoing reinforcement, coaching and inspection of the training.
Planners that are able to deliver beyond logistics have the most job security and should be compensated greater. Corporations are constantly looking to outsource work that is not within their core business. A planner that is able to effectively communicate the value that they bring to their organization and help the organization feel good about their meeting investment has moved from being a tactical employee to one that delivers strategic value. It's much easier to replace or outsource the activities of a tactical planner.
Posted by: Dave Lutz | Jul 14, 2006 2:43:32 PM
The inherent nature of meetings and events are extremely qualitative in nature. ROI is a quantitative measurement used to help reduce risk in business decision-making. Using trailing indicators or other methods of financial analysis to validate the investment in a meeting excludes (by the nature of the analysis) many of the reasons corporations have meetings.
Like Mr. McCann, I would contend that there are too many extraneous factors involved in revenue contribution (the financial measurement standard). These extraneous factors could include advertising, product/service quality, competitive product/service, other marketing initiatives, etc. Measuring whether the objectives of the organization have been met by the meeting is a much more reasonable approach.
More importantly, it’s about optimizing and increasing the value events bring to the organization. Making the meeting more effective is the real key and objectives-based measurement is one part of that process.
We've also found that executives call many event ROI numbers into question, because they don't find them credible. There are too many variables and assumptions to apply a classic ROI formula but the real issue is that ROI measures don't tell me what to do to improve the effectiveness of my event next year.
Posted by: Lynn Stadler | Jul 14, 2006 3:21:25 PM
As a practitioner of the Phillips ROI Methodology I have read this discussion with great interest.
I am glad that everyone seems to agree that ROI is the Holy Grail, if only we can find it.
Is the Phillips ROI Methodology really valid, can we deliver a credible ROI number? It may surprise you to learn that several thousand ROI impact studies are made every year in more than fifty countries, using the Phillips model. Sixteen textbooks and dozens of casebooks have been written and all together published in 25 languages. 23000 have attended one or two-day workshops, and more than 3000 have attended 5-day certification workshops, including 75% of Fortune 500 companies. So why have we not heard about all this? Because it has all happened in different industries, measuring ROI for training programs. All we are doing now, with the support of MPI, is to transpose this vast body of knowledge and experience to meetings and events.
It is true that there is no quick fix solution to measuring ROI, if there was, we would all have been doing it a long time ago. It requires certain research skill that many meeting planners are unfamiliar with. So what do we do? Give it up for a bad job or rise to the challenge?
The Phillips ROI Methodology is all about measuring non-quantifiable results against objectives. I wish we had done a better job of making this clear and much confusion could have been avoided. The methodology requires you to set detailed objectives and measure results at five levels where only the ultimate fifth level involves conversion into monetary values in order to calculate ROI. For most meetings you will stop at one of the lower levels, still providing management with a good measure of success.
When measuring ROI, you have to set objective and measure results at all the lower levels, this is the ‘story’ which gives credibility to the final ROI number, it is called the ‘chain of impact’:
Level 1 - Reaction, Satisfaction and Planned Actions; Was it a good meeting, was the venue good, were the speakers good, did you have enough time for discussion and networking, can you use what you learned in your job?
Level 2 - Learning; What did participants learn and retain? You learn information, skills or attitudes, either from listening to speakers, discussing with peers or from experiences.
Level 3 - Application; Did participants use what they learned in their jobs, did they do anything different because of the meeting? Learning is of little value if you don’t apply it.
Level 4 - Impact; this is the result of what they did differently, something which provided value to the stakeholders.
Level 5 - ROI; Impact measures are converted into monetary values and compared with costs, this is how we calculate the ROI.
If you want a brief introduction to the Phillips ROI Methodology, read my article on http://www.eventroi.com (the link is at the bottom of the first page)
Posted by: Elling Hamso | Jul 16, 2006 7:47:22 PM
Subscribe to the RSS Feed
