Atlanta Analytics

Ever wonder why “engagement” and “enragement” are spelled so similarly? Measuring engagement is tricky business, made even trickier by the use of esoteric indices and web-centric (as opposed to economics-centric) metrics to describe the concept.

So, here are some thoughts on how to measure, evangelize, and create delta in engagement.

First things first

We have two key issues in the area of measuring and expressing engagement to our businesses:

1. Most of the makeup of “engagement” consists of metrics that have not been translated to economic value. This requires extra explanation as the measures are passed around the organization, and may alienate certain very important audiences.
2. Largely, the types of “engagement” indices and calculations I’ve heard of are typically non-predictive in nature. They do not fold into or constitute a model that can be used to calculate the return of incremental investment or a shift in budget allocation* across media or resources. Typically, these engagement models are used to show delta in an index when delta in revenue or ROI is hard to create or isn’t enough for a pushy organization.

* Shifting your budget allocation based on distribution of historical ROI values is a highly unsophisticated long term strategy (fine in the short term) because it fails to consider the impact you can have on ROI by delivering better messaging and a better experience (read more).

It’s the economy, stupid!

The first thing to deal with is a nugget of wisdom made popular by James Carville. Fortunately, it’s a concept that the measure (argg! hate that word) community is transitioning toward as we speak: making measures economic in nature. Tweets, Likes, Pageviews, subscriptions, shares, etc. are the language of web metrics, and the good news is that they are all worth something. We just have to figure out what.

So let’s get started. The first thing to lay groundwork on is that there are two great, pre-existing models we can use to estimate the worth of something.

The first one is obvious: real return, which describes the actual dollars we get in a transaction. In a paid search campaign, we can see the money that the campaign drove in real terms. In truth, we only see a certain percentage of the money, because real return models don’t intrinsically consider ripple effects: that a single click could lead to a positive experience, a sale, lifetime value that can’t all be captured in our data systems, the potential for word to spread to their family and friends, etc., etc.

Real return is both a blessing and a curse: it gives us cold, hard data, but it also impregnates our businesses with the idea that everything is suddenly measurable in cold, hard terms. That is completely inaccurate.

The second model is probably the one that will be more powerful for us in translating engagement into value, and doing so in a predictive way: willingness to pay (WTP). WTP describes the investment a business is comfortable making to get a desired outcome, often an outcome that is non-financial in nature, or difficult to translate to financial terms. Investments in radio advertising, for example, are based largely on willingness to pay: “We can deliver your message to over 300,000 listeners in our lunch time block.” The advertiser does some back of the napkin math to figure out what the financial impact of 300,000 listeners may be (the business owner obviously wants some sort of a return), and they decide on whether the ad is worth the cost.

Truthfully, WTP models can and should be derived from real return data, but getting too obsessed about turning WTP into real return will bring your decision making to a halt. You must fight the urge to get stuck in this trap; keep moving.

Getting your Financial Figures

Now, it’s time to figure out how to put dollar signs on your tweets, likes, etc. One of the best places to start (if your business is large and does traditional advertising) is by finding proxies in the traditional advertising side. If a single post on Coke’s facebook wall reaches an audience of 3mm viewers (measurable), and is liked/commented by 10% of viewers (measurable), and that is seen by their friends (measurable), you can estimate the total reach. How much does it cost you to get that same reach with a billboard?

Now add a little premium to that figure because a) facebook is an interactive medium and builds affinity and b) you may have learned something about your creative or brand from the comments you received, which has value far above and beyond what non-interactive, traditional media can provide (you have to pay extra for the focus groups, in fact).

If you aren’t a big place with huge advertising budgets, you can just investigate what it would cost you if you did. Make a few phone calls to the newspaper, TV station, radio, or an agency, and see what you will get for what price in terms of reach for an audience that fits your target market.

You’ll have to go through your whole organization and look at each action, trying to understand the value of each effort (you will probably be using a mix of real return and WTP/proxies in each scenario):

• What am I willing to pay for a paid search click, email open, or display impression? (ROI + extra lifetime value + word of mouth potential)?
• What am I willing to pay for an additional developer? (cost or incremental revenue of releasing a feature or product earlier)?
• What am I willing to pay for a usability person or designer? (potential upside of a range of basis-point deltas in conversion rate [a fancy way of saying build a matrix] for applicable traffic sources).

ROI is NEVER the answer to any of these questions because ROI fails to consider anything but a moment in time. On the WTP side, don’t forget to factor in other costs like salary, resources, efficiency loss, etc. in addition to the hard cost of the media or whatever it is you’re looking at.

No, this is not going to be something you do in a week. But it’s also not something you have to do for your whole company before you can start making predictions. Take off a few bite-sized pieces and see where you get with it.

Making it predictive

Now that you have your models for valuation and a method of getting some numbers to work with, it’s time to make sure that your model is a predictive one, not a backwards-looking one.

Creating engagement data for the past is easy. Creating it for the future is much harder. The good news about a WTP model is that it is pretty forwards-looking, though: you are focused on what an incremental set of eyes, an incremental sale, etc. is worth to you. If you are stuck in the ROI trenches, you will never be able to understand what a sale is worth, and you will never be able to state what you are wiling to pay to create a new customer or rekindle the flame with an old one. As long as you get things back to WTP, you will have a predictive model on your hands (it would take me 20x the length of this post to actually start discussing a viable model, so if you’d like a post that dives into real models, please leave a comment!)

Delivering the message

Now here’s the critical part: when you deliver the message, deliver a prediction, not a predictive model. Use your model; don’t ask others to use it. Here are a few examples:

• “Mr. CEO, our work has turned up an opportunity to bring in an incremental 5% market share with a $1.75mm investment, all in.”
• “Mr. CMO, we have found a way to reallocate 10% of our radio budget to social media and improve that budget’s reach by 50%.”
• “Mr. COO, we have found that there is a significant tradeoff between process and productivity in the IT department that we estimate has cost us $7.5mm in revenue in the last 2 quarters. Some changes in our workflow can close that gap in the next 6 months.”

Also, deliver the right message to the right people. CEOs care about stock price and company valuation, which means cash flow, market share, and innovative things to talk to the world about. COOs care about efficiency and good ideas that will impact the business directly. CMOs care about using their budget effectively and being able to show the value of their existence in terms of changes since the last CMO was fired 3 months ago.

Your mileage with your executives may vary, but keep in mind that different executives come with different goals and perspectives. Try to guess what motivates them and reposition your idea in a way that suits their personal or professional goals.

What happened to engagement?

Nobody gives a crap about engagement. Engagement is just a stepping stone on the way to “worth,” and it’s a non-predictive stone. Once you’ve moved on, you’ll never look back.

“All of your data, under one roof.”

So good in theory, but in reality, the marketing promise has written checks the hiney is unable to cash.

Data centralization is something that people have been working on for 50+ years. Our digital data warehouses are only one of the more current attempts at the myriad ways businesses have tried to get it all in the same place so we can see the whole picture. And you know how much of a struggle this effort has been for us in digital? It’s always been that way. It’s actually always been worse.

Unfortunately, the promise that data centralization makes will never be realized. The promise is that centralization will get us away from the clutter of decentralized data and give us one place to build from. I don’t think we will ever get there.

For one thing, the data fights centralization. In all fairness, it is very, very hard to centralize data and link information from our various systems (CRM, point of sale, web analytics, web marketing/advertising channels, call center, etc.) And once the data is linked, it’s incredibly hard to collapse trees of data where metrics don’t match up (web uniques, customerIDs, credit card #’s, email addresses, invoice #’s, TV and radio performance data, coupon redemption, finance metrics, etc.).

This is HARD. Being a CIO would suck.

The reason I don’t think we will get there is because today, linking data and creating models almost always means sacrificing fidelity. Take cross-channel attribution, for example. The problem is that revenue is duplicated across the reporting delivered at the channel level: email reports $10,000 and search reports $10,000, but we only sold $15,000. Many tag management solutions today “fix” that problem by choosing which of the tags to fire, excluding the other tags. But is this a fix at all? We just lost fidelity into the revenue that each channel participates in. If two different agencies manage this media, they just lost some ability to understand how their media influences revenue; they lost some ability to optimize and boost results, because now they are only working with part of the story.

These tools fail to take outside influences into consideration like the creative, changes in budget, visitor history, etc. They teach marketers to assume that media channels intrinsically are “closer” or “further” from the purchase, or more/less related, and that this will be consistent over time. In other words, they suggest that because email was an early touch and search was a late touch last month, it will continue to be that way in the future and that’s how you should optimize the campaigns. Hogwash.

Yes, it is true that guns don’t kill people and tools don’t tell stories; people do. But when we create technologies that “fix” problems, we are stating that what comes out of the tool represents the truth, the whole story, or something like that.

The promise of BI and data centralization is that when information is married and central, we get a better picture of what’s happening and can make better decisions. It’s this fallacy that has led your company’s management to make decisions that make your head spin. It’s that false promise that leads businesses to invest millions in a solution that blurs the focus and liposuctions critical details that will influence decision making.

Am I saying that progress hasn’t been made? No. Immense progress has been made, and data centralization is a hugely important thing. But it isn’t the answer. It’s one of two barrels we need to employ to understand what is happening. The other is the specialist view. The highly trained and sometimes myopic perspective that crafts per-channel, per-market, per-issue solutions. Specialists on their own are highly dangerous. Generalists on their own are, too. It’s only when both fire together and you can take superb, but realistic solutions to market that you win.

This is just a, “where do you spend your time,” idea. If you are investing an inordinate amount of your attention into centralizing, de-duping, common-keying, etc., at the cost of per-channel, per-market, per-idea optimization, I believe this isn’t the best use of your time, as a ninja analyst.

It’s your choice how to tackle optimization. You can start at the central: Omniture Genesis, multi-channel attribution, etc., but you MUST go to the individual branches. You can’t make decisions from the trunk. Or you can start at the branches to find opportunities, but you MUST look at the trunk to make sure that growing one branch won’t tip the tree over.

“Simplification” through centralization is an illusion. What we do is hard, and it’s going to stay that way. It’s just a decision on how we’re going to spend our time.

OK, last post of the week, I promise.

While I was waiting to cross the street the other day, I noticed something that made me ask the question, “Who in the world thought that was a good idea?”

About 50% of the cars making a right at this intersection were hitting the curb, ruining their pretty wheels, and looking generally silly. For the angle of the turn, the curb was just horribly designed. It extended into the intersection, and for cars to make the turn if a car was coming the other way, they practically had to run over the curb to prevent the front end of their car from sweeping out into oncoming traffic.

I see situations like this all over the place, both in the physical world and on the web. Most of the time, these types of problems seem so obvious, but not to the people who can actually fix the problem or who could have expected / avoided it in the first place. Why is that?

In our industry, we tend to get consumed with the tools we have in our arsenal. We have a flow of data coming in, and like in the Matrix, we tend to watch that flow and look for kinks in the mesh. But there is a lot of data that doesn’t come in table or chart form. A lot of the data that our offering creates exists in a more qualitative sense. It’s there if we’re paying attention to the road ahead, rather than driving by watching the speedometer.

I think a lot of organizations miss out on some pretty high-potential enhancements to their offering, their marketing, to everything they do because they are looking at the dials rather than the road. This particular curb may not cause a “blip,” because maybe there are 9,999 other curbs with no issue. The averages smooth the curve. But it doesn’t mean that there isn’t a significant problem out there, and maybe the charts and tables aren’t the easiest way to see it.

Some of social media’s greatest untapped potential is this qualitative view. People are out there talking. While we are trying to produce an ROI on twitter, we’re ignoring the “curb checks” that our customers are screaming about. We’re stuck on the dials, rather than the road.

Companies spend hundreds of thousands (and often millions) of dollars trying to make the spreadsheet and chart-creating tools better and better, or to get more of these tools. These tools, sadly, will never, ever substitute for looking at the road. No speedometer, odometer, GPS, or anything will ever be good enough to prevent you from missing something big happening on the other side of your windshield.

Of all the tools in your arsenal, I believe common sense may be the best one. How can you use it — and get your company to use it — more often? Take a look through the windshield today. I bet you’ll see some pretty interesting stuff.

You haven’t been trained on that. You don’t have enough direction. People won’t cooperate long enough to make things happen at your company. There isn’t a process for this. Tell me how I prioritize this against the 100 other things I’m doing. That’s not how things are done around here.

Maybe you just aren’t meant to make a difference. Maybe you’re just being paid to measure stuff and produce reports. Maybe.

But maybe you’ve been hired because the company needs someone to stir the pot. They need someone to shine a light that finally unseats the loudmouths who make all of the decisions based on gut feel and steamroll their colleagues. Maybe you have talents that can help your company wake the hell up, unfreeze the crippling processes that have been put into place, rethink how they market and react. Maybe that, instead.

In the real world, there are obstacles. There are stubborn people. There is a lack of direction and leadership. Get over it. You aren’t writing term papers with word limits and assigned reading any more. The grade you get is subjective. That’s life.

As analysts, we are the last people on earth who need “direction” from some internal T-Rex with an agenda. We let the customer do the guiding, not the boss. Learn from what you observe. Craft a plan to put what you learn to action. Make your business react. Don’t let them call you a measurer. Make them call you a leader.