Feb
8
The folks over at SiteTuners have been working on a great piece of technology for simulating eye tracking. But be warned: it’s in beta and does come with a little bit of baggage so far. But Tim Ash is one smart dude, and he’s out there gathering feedback on forums and blogs to make the tool a winner when it’s out for good.
People like tools like this because they are intended to solve arguments quickly. We can use data from users or computer simulations to tell the bonehead CEO that his ideas of putting jumping pink kittens and 24 calls to actions are bad ones. And this is great, so long as the data is inherently valuable.
The trouble with AttentionWizard so far is that it evaluates web sites like you would evaluate photography or art, using design concepts like contrast, weight, etc., with an obvious dose of “top-leftedness,” meaning that the tool anticipates where headers and text would start and shows greater focus there. And it is also seemingly weighted heavily toward identifying outlier colors.
But that’s not exactly how people evaluate web sites. The classic example is banner ads and how conditioned people are to avoid them, regardless of how bright and obnoxious they are. In AttentionWizard’s demo video, you can see this error in their demo of 1-800-flowers.com, where the house ad gets a lot more attention than it would in the real world.
But to be fair, this demo does reach good conclusions, and you can see how much more straightforward the end result is. But the point is it reaches good conclusions with the wrong data.
Unfortunately, we live in a phase of web analytics where most of the tool data is taken at face value. So, people see a heat map and believe that’s reality. Or they get a report about keyword performance and assume that keyword A is better than keyword B, just because it has a higher ROI (but without seeing if the landing pages make sense, if the prices are competitive, and on and on).
Tim makes great points on his site about what this tool is and isn’t, but right now that’s like John Stewart making points about how his political opinions are just his own: it’s irresponsible because it ignores how powerful credible sources are, and to both Tim and John’s credit, they are credible sources because of their history (and charm, of course!).
Below, I’ve shown a few demos of where AttentionWizard deviates from human behavior so radically that it can actually drive the wrong decisions. And I’m not trying to show pictures of half-naked women or anything that the tool shouldn’t or couldn’t technically be capable of, because I do think that’s an unfair test.
Test 1: Pattern recognition
Here’s one that humans will get right away. Just try to take your eye off of the star. AttentionWizard didn’t get it at all.
In the world of the web, this is important for low-contrast sites that might use shapes and graphical weight to point things out. We probably won’t see an example this simple online, but it is possible. I’m envisioning package design – you see 11 identical-shaped shampoo bottles on Amazon.com and one crazy-shaped one. Where does the eye go?
Test 2: Position-sensitivity
So here’s one where I’ve placed calls to action at 1150 pixels. This one tests to see whether the tool will hold up to the boardroom when someone says, “See, people will see this button at the bottom of the page!” Unfortunately, it seems that the tool isn’t yet sensitive to where things are and what screen sizes are most popular, which is definitely something that needs work before AttentionWizard is ready for its close-up.
I think this is an easy win for the tool – something that shouldn’t be too tough to change.
So Tim is to be applauded on creating a great technology. But I would strongly urge people to not circulate the data it renders quite yet, not because the tool is bad, but because data is notoriously misinterpreted in the average company – taken for face value.
People can caveat their tools all day long, just like Charles Barkley could say he was not a role model, but we have to be responsible to the real world and produce things that don’t send mixed messages with the expectation that people are smart enough to filter the data. They aren’t. Not yet.
Leave a Comment~ FIN ~
Oct
21
I’m a metaphor guy. Here’s hoping you like them, too. I think they do a great job of explaining things without sounding whiny and desperate, and that’s great because the world of web analytics is getting very whiny and desperate for the recognition and authority we think we deserve.
So, do we?
Web Analytics-ology
What happens when something hurts, you have a cold, or you’re a generally weak person according to Tom Cruise? You go to your general practitioner or internist to see what’s up.
When you get there, you’ll start talking about what you’re feeling, the doctor will hit your knee and look in your ear, and he’ll figure out what’s going wrong. If you have something seriously wrong, he’ll send you to a specialist, but most of the time, he can recommend a solution to your problem that will put your body back on course. If the internist sees something wrong with your ear and fixes it, the ear doctor down the hall doesn’t throw a hissy fit and slash the MD’s tires. If the internist gives you a splint for a sprained wrist, no orthopedic surgeon goes to their boss whining about how the internist is trying to do their job. They trust that the internist knows enough to treat the patient well. And they do, most of the time.
So let’s think about what a day in the life of a large web operation looks like. What happens when the web analytics person tries to make recommendations on how to change the color and location of the call to action on a landing page? Usability and designer folks go ape shit. What happens when they find that the paid search campaign is blowing budget on a non-converting keyword, recommending that we eliminate it from the campaign? Nuclear war with marketing. How about when they realize that there’s a major browser incompatibility and they recommend a few changes to the CSS and HTML? You’re about to get shivved by a programmer.
Where does this territorial nature come from? Well, for the first time, let’s stop blaming all of these specialists and look inward. Do we really know usability? Do we really know design? HTML? The answer is probably sort-of, but we haven’t had the years of training that doctors go through, so the real answer is that we really have more opinions than knowledge, in all likelihood. And that drives these specialists nuts. The real truth is that the average analyst creates more work for these people, rather than reducing work that doesn’t require the full gamut of the specialists’ skill sets, like the internist does.
I’ve said before that web analytics people aren’t specialists, and it’s true. But I don’t think as a whole, we’re good enough yet to call ourselves generalists. We need training. We need to get out of the specialist mentality of training harder and harder at what we’re already good at, and we need to get into the mentality where we train our weaknesses, researching usability, design principles, HTML, PHP, javascript, etc. etc. We also need to study business fundamentals, marketing principles, branding, corporate communication, SEO, and more. Knowing a little about all of these things will make analysts much more valuable than knowing statistics, forecasting models, and all of that mess ever will. There are no shortage of specialists in the world that can handle that work, and we’ll bring them in when they’re needed. But there’s an enormous shortage of generalists who can direct traffic and questions to the right places, and clean the easy messes up themselves.
7 Comments~ FIN ~
Oct
6
Has cash flow been demoted? Is audience the new king?
Filed Under Web Analytics in Business | 1 Comment
I was talking back and forth with someone I’ve begun working with about Google’s future: are they going to take over the world? How can they give analytics away for free? What does the future look like?
I was writing a reply when I realized this might be an interesting thing to explore here. From a business standpoint, what’s going on here? Are we seeing a shift in how businesses, even publicly-traded ones, treat cash flow? Google seems to make a lot of questionable profitability decisions. But should we really care?
I think we’ve seen Google consistently give away products that support their key revenue sources. They know that paid search works wonders, so providing a tool like analytics on a search stipend is a smart move – it’s a tool that proves the value of search with increased depth, and allows customers to compare search ROI to other channels that compete for budget. Also, Google is right to think that people have a right to understand what’s happening on their web sites for free. It will get thousands of SMBs to consider and test marketing [that Google knows will be like crack for them] spends to increase their traffic and sales. Without a free option to measure, it’s likely that significantly fewer business would take that investment risk. Plus, it builds on the friendship model: people love your brand, so your brand will thrive. And, of course, there is an upsell opportunity for Urchin.
Culturally, Google was founded on free tools without any real, concrete plans to monetize something the guys thought would make the world a better place. Shortly thereafter, text ads were put in use and the rest is history, from a business standpoint. 90% of their revenue comes from paid search, but this hasn’t changed their roots that providing the internet the best tools for free will keep people around. And one of Google’s key differentiators is that, from the moment they began, their pace of product creation and innovation has not slowed or been stymied by the business’s needs to find profit models to fit every line of work. Asking Google to monetize their other products would be like asking the Friends writers to monetize every line in the script. They know they don’t have to do it because their product has so much gravity, it will support other, more natural revenue sources that don’t risk ruining the fabric of that attraction in the first place.
If anything, pace has increased as Google [labs] has put out / bought / refined tools that permeate our day-to-day more and more. This is why gmail became so dominant over Yahoo mail. While Yahoo was busy figuring out how to sell display ads and make weather and news partnerships to layer on their very good mail service, Google stripped it down to the basics and made an online client that gets out of your way. Once they did that, they made it better and better at a pace Yahoo found impossible to match. And all of these gmail users are now using Google, and in many cases, about to subscribe to Google’s enterprise mail and calendaring systems. Potentially a big attack on Microsoft and an eventual move into enterprise-level data management, sourced storage, SaaS hosting, etc. solutions?
I guess the point is that Google realizes that they don’t have to over-think the business side of things so long as they maintain / grow their audience with that gravity. While it’s true that at their scale, basis points mean millions of dollars, I think they’ve realized that the process that goes into the management of basis points affects the quality and pace of their output, representing an enormous opportunity cost in terms of audience.
While I don’t know if this represents a phase shift in pushing Wall Street’s juice-squeezing mentality to the curb, I do think that an extreme, corporate value-creation mentality would be the beginning of the end for Google. Trying to talk to a creative programmer about profitability is like trying to talk to a banker about creativity. They both think that if they build it, people will come. And while they’re both right, the age we’re in is siding with Mark Zuckerberg, Steve Chen, Jeff Bezos, Jack Dorsey and other people who are creating incredible magnetism on the internet, perhaps with some disregard to how they’re going to pay the rent. While you might argue that Bezos is a hell of a business person, you might also argue that his focus on business hasn’t interfered with his focus on what made Amazon important in the first place.
I’d never say that we should stop worrying about monetizing business lines, but I do think that Google might be on to something that many businesses can learn from. It might just be okay for successful businesses to do great things for their customers and prospects to increase loyalty, reach, and brand love without any direct attempt to derive a return. And I’m not talking about giving away pens, golf towels, shirts, USB drives or notepads. Why shouldn’t BMW visit your city and teach owners and non-owners alike how to drive race-ready cars for a weekend, for free? Because they’re giving up profits. But I would gladly give up today’s profits for tomorrow’s, if today’s were sufficient. BMW, are you listening? This is a really good idea.
1 Comment~ FIN ~
Sep
21
Last Wednesday, we found out how to make an onion and chocolate sandwich and were left to wonder how it could possibly taste good. With Adobe’s acquisition of Omniture, you can tell that nobody quite gets this one: ADBE stock dipped just slightly, and OMTR hopped up to just under $22 a share, up from around $17 (quite a bump, but still adding up to less market cap than the purchase price, and neither of these responses is anything exciting). So why did Adobe buy Omniture? Let’s take a look at the case.
Financial Reasons
One answer is obvious: now’s as good a time as any to buy Omniture. They’ve scooped up tons of great companies and technology over the past few years (through acquisitions), have a tremendous client roster, and somehow have managed to be squarely in the red almost every year since going public. Adobe must feel like they can manage the business better than the mormons, and they’re buying at a good price.
If you’ve ever been to Omniture Summit or Omniture HQ in Orem, UT, you’ve no doubt realized that Omniture likes things that look cool. They spend a lot on creative work (or have a lot of internal people to do it) because every piece of collateral you touch or presentation you see is filled to the gills with 3D images and fly-throughs that explain their products in more dimensions than you use them. Omniture’s offices are pretty cool, too, but nothing crazy. They’re just pretty cool. You’ll also notice that their employees are paid very, very, VERY well compared to standard Orem, Utah salaries, and that would be okay if their customer service wasn’t such an issue. There are a lot of places for Adobe to step in and manage budgets, without a doubt.
Clients
Adobe has tons of clients. Omniture has tons of clients. They’re probably just about all the same clients…
Business Model
Adobe writes software. Omniture writes software. They both have talented programmers. Check.
Adobe distributes software in boxes and via downloads as client-side design tools, and bills by the product version. Omniture delivers software as a service and lacks the server infrastructure to make the tool as responsive as it should be – and they bill by the byte. This expansion of infrastructure will equal major dollars out of Adobe’s pocket if they want the tool to feel as snappy as Google Analytics any time soon. Ding.
Adobe provides service over the phone or Internet if you have trouble with their software (which is usually your fault). It’s pretty simple. Omniture provides service as a major component of their offering: the tool is incredibly complex, prone to installation error (not their fault), prone to misinterpretation (again, not their fault), and needs a lot of hand-holding. This service model has tiers: account managers for most accounts and engagement managers for big names and big-bucks (or big complainer) customers. Ask any client of Omniture’s what their biggest complaint is. Definitely customer service: the tool is great, but support response time is just awful. Since Adobe has a completely different customer service model, bring out the wallet again! Ding!
The future of computing is definitely software as a service. We have Google Docs, online email, calendars and task management, online photo editors, online design and layout software, and we’re expecting online versions of MS Office in the very near future. So of course, Adobe will want to leverage Omniture’s experience developing online applications, and will definitely want to leverage analytics to improve the user experience over time. No question, although it could be a while. Check.
The Internet, Analytics, etc.
Online marketing is a pie everyone wants a piece of, and integrating SearchCenter into SiteCatalyst was a big step in the right direction for Omniture. While tools like SearchIgnite and Marin still probably capture more interest because of their power in an agency environment, SearchCenter does pretty well and represents a huge investment by Omniture. An investment that has zero benefit to Adobe. Ding.
From a web analytics perspective, it’s a little perplexing until you consider Flash. Yeah, you have Dreamweaver, but that’s so obvious and such an easy cut & paste job that it’s not even worth mentioning (so forget that I did). Getting Flash tracking to be built in without the designer having to do anything is a very big deal, but it won’t be easy to create those hooks. Same thing with Flex/Air, etc. The internet is definitely going the way of the RIA, so an analytics tool that builds itself in is a huge boon.
The trick will be in how to do this. With so much manual naming of page names, site sections, eVars, props, and on and on, these tools definitely aren’t going to do the work for you. There will still have to be a significant amount of strategy developed around analytics, but perhaps with the bonus of not having to worry about tagging. From a time savings and data accuracy perspective, this could be a huge plus.
And comscore?
Now this is definitely worth getting excited about TODAY. Want to know how your page views for yesterday stack up against the industry leaders? Want to know how your conversions stack up? What your traffic source breakdown is vs others? Want to be alerted when your competitor ups their paid search budget? Done. And right next to your data. Very cool.
Hopefully we’ll hear something from these guys on their plans. I’m definitely eager to understand the deal beyond fishing around in the bargain bin.
I’m still confused about…
I still just don’t get it. There are too many places where there is no perceived benefit at all. While there are definitely opportunities to stitch the online components together (creation software like Dreamweaver, Flash, etc.), there are too many places where each company will be speaking a different language. SearchCenter, as I mentioned, will benefit Adobe zero. Photoshop, InDesign, Lightroom, and the majority of Premiere and After Effects will have nothing to do with Omniture. The customer service and sales models are completely different. And Adobe doesn’t stand to benefit from investment in server farms and increased bandwidth for years – until they develop their products as a service.
In other words, the venn diagram that describes synthesis here is just a thin sliver of overlap as far as I can tell, unless the benefits are much farther out than I’m seeing.
2 Comments~ FIN ~
Sep
2
Ever have one of those startling moments of clarity when you suddenly realize that everything – literally EVERYTHING – we’re trying isn’t working? Ever had one of those moments last three years? Oh, me neither. I was just asking.
I think we need to realize that now is the time to take a step back and look at what we’re doing on the internet. What better time to worry a little less about the risk of putting a company on cruise control than when everything is out of our hands anyhow? Now, I’m not in any way advocating taking our eyes off the road; I’m just saying let’s just take our attention off of trying to work the pedals and control everything one piece at a time to reflect on what the hell is happening to us.
We’ve just gone through a long period of excess where every decision was right and newcomers to the stage were getting standing ovations. And we know now that it was quickly followed by a stage where all of our mistakes came back to haunt us. A quick drive through town shows us that: a lot of businesses with the lights out.
There are things I think we need to start thinking / worrying / crying about today. And I mean TODAY. Here are two of them, with more to come later.
1. Do the people I trust really know what they’re talking about? Are they holding that against me?
Your agencies. Your employees. Your neighbor who “fixed” your lawnmower. Ever feel like every time you find expertise or help, everything goes to hell? There is a reason for it. There’s even a phrase for it. You’re being bullshitted.
There are three types of internet professionals:
- People who genuinely know what they’re talking about and are arrogant, lazy jackasses because of it (”It will take 3 months to develop that software [while I play Freecell].“)
- People who are full of it (”We guarantee top position in Google.“)
- People who genuinely care (”We are on it. We will do everything we can to succeed.“)
In essence, you have competence alone (Usain Bolt), work ethic alone (Bernie Madoff), or a combination of the two (Bill Gates, Steve Jobs, Jeff Bezos, Tiger Woods, Michael Jordan, Jesus). How do you picture these different people in your mind? Can you see that some of your employees / agencies / neighbors strike a similar chord with you? What category do they fall into? We all hate both Usain and Bernie, and while we might grimace at a healthy ego in people like Steve Jobs and Tiger, they’ve earned it and we respect them for it. And they’ll continue to earn it rather than resting on their accomplishments.
I used to work at an agency that doubled in size and earned its true stripes in the industry. We celebrated the managing director of the office for facilitating this achievement, but everyone couldn’t stand this person because this person was faking their way through it and taking credit for some of my very talented colleagues’ hard work. Sure, we doubled in size under this person’s watch. But what would we have done if the role were filled by Warren Buffet? But this person was content with their success and felt that knowing what you’re doing isn’t necessary if you’re succeeding.
But success does not equal success. A high school running back can set a school record, but never go to the NFL. We have to all remember that we have a long way to go and a lot to learn. We can rest after our 10th interview in Fortune Magazine.
Fire all of your Usains and Bernies. Today. And get feedback so you know who these people are. Bernie Madoff scammed some of the smartest and most successful people in America. Don’t be so egotistical as to think you aren’t being fooled, too. When you hire the next one (sometimes they’re hard to identify in interviews and you find out later), fire them once it becomes apparent. Period. You’re actually doing them a favor – this is the only way they will learn that change is necessary. Their egos flare every time they get away with their parlor tricks, so firing is the only way to teach.
2. Do I have an actual vision for my web site? Do I know the difference between goals, objectives, strategies, and tactics?
Generally, I reject this sort of corporate speak and feel like it puts us in a death spiral of executive blabber, but this part is important. It’s how you plan and achieve. If you don’t have a plan for your business, everything is an accident, both success and failure.
Goals, e.g., “Attract the most qualified users to the site and convert them with maximal efficiency.”
- Measurable: NO. Seriously. Get over it.
- Scope: Large
Objectives, e.g., “Increase our web marketing ROI by 25% / Increase conversion rate by 10%.”
- Measurable: YES. Woohoo!
- Scope: Large
Strategies, e.g., “Understand the value of various traffic sources and optimize both spend and landing pages to increase conversion.”
- Measurable: NO!
- Scope: Small/Medium! Strategies are NOT at the company/site level. They are discrete to certain goals and objectives and qualitatively describe the methods employed to achieve.
Tactics, e.g., “Identify top 5 SEO landing pages and align copy with popular keywords being used. Remove extraneous form fields. Test ’sale’ creative in PPC ads. Conduct A/B test of the ‘Buy Now’ button color.”
- Measurable: YES! Think of measurement as checkboxes with results: I did this and here’s what happened.
- Scope: Small
By understanding what each one of these things are, we can write down what we want to achieve, how we want to achieve it, how we know whether we’ve achieved it, and how we know what helped / hurt us in achieving it. I can’t think of any better way to understand what the hell is going on.
Please, share some of your experiences!
4 Comments~ FIN ~
Aug
28
Google Analytics Advanced Segment and Custom Report Sharing Going Live…soon
Filed Under Web Analytics News | Leave a Comment
So, the fabled sharing of custom reports and advanced segments is live for select logins in Google Analytics. Guessing this means that we’re going to see this feature released live very soon!
Google Analytics’ Advanced Segments are a powerful way of defining just the cross-section of information that you’d like to see a report for. That might be looking at your Google PPC search campaign, or it might be something as complex as looking at repeat visitors who came in through email referrers and have more than 6 page views. Pretty cool.
As you can see in the screenshots, the new options for “Hide from profile”, “copy”, and “share” are up and running. Also, the same options exist for Custom reports.
Custom Reports, if you’re curious, are a great way to get the exact metrics you seek, when a comparable report isn’t already available in Google Analytics or when you’re too lazy to find it. These are a killer way to see whole numbers for goal completion or get geographic and campaign data mashed up. In fact, they’re a great shortcut for buried reports that already do exist, but a word of caution: the data in custom reports may be sampled and therefore not match exactly to the same built-in report.
Looking forward to seeing these in everyone’s profiles! Not sure how long this has been going, but it’s the first time we’ve noticed it. Kudos to @whitneyhannan for the find.
Leave a Comment~ FIN ~
Aug
18
The New Search Engine Optimization – SEO 2.0
Filed Under Web Analytics and Search Engines | 3 Comments
Specialists. They are wonderful, smart, and talented people who can push boundaries and solve complicated problems. They are also the people who invented lead plumbing, asbestos insulation, and solved our toxic waste disposal issue by sinking leaky barrels down to the ocean floor.
When specialists solve a problem, they often create other, unforeseen problems because they tend to look through their own self-imposed keyholes. Over time, these specialists broaden their horizons after realizing that the materials they use and methods they employ matter just as much as the predicted outcome. Turns out copper won’t make poison of your drinking water.
So you’re probably asking me to get to the point – what does this have to do with SEO?
I’ve been fortunate to have worked with many professional SEOs over the years who understand that the point of SEO is not to gather a trophy collection of high rankings in Google, but to have those trophies produce traffic, interaction, and ultimately revenue (or some other comparable value). To this end, they understand that ranking number 1 for “women’s fashion” and sending users to a page that looks like this is a failure:
Women’s Fashion – Fashion for Women who like Fashion
Women’s fashion is a fashionable, womanly way of being a woman with style (commonly referred to as Fashion, or Women’s fashion). Check out this link to women’s fashion (it comes right back to this page! surprise! FASHION). Maybe you’re looking for fashion BY women, or fashiony women.
Whatever you want in the world of women’s fashion, we have fashion for the woman in your life (unless you are a woman, and want fashion). Let’s say hooray for fashion, fashion, fashion! Go women! Fashion!
Let’s do some math:
Fashion + women = women’s fashion.
by,
Women McFashionton
Of course, that’s a bit of an extreme example, but we’ve all seen it.
Unfortunately, I’ve also had the displeasure of working with some SEOs who have a total disregard for usability (although truthfully, I’ve seen this problem much more frequently the other way around: usability engineers who pretend that search isn’t a top-5 use case, when it’s usually #1). They literally say, “Our job is to get rankings, and it’s someone else’s job to make the conversions happen.” That attitude is the lead pipe with mercury strychnine lining of the internet. Since SEO involves the modification of the very pages that will need to produce conversion post-click, this is an incredibly irresponsible, naive, and downright ridiculous way of going about search engine optimization, but I’d say that 3 out of 4 SEO engagements end up more or less this way.
This notion of unaccountability is the age-old plague of specialists who have not yet matured (or learned) enough to realize the value of synthesizing efforts into a single fabric, rather than loosely stitching together a hideous quilt of specialist’s favorite patterns.
How to pick a good SEO
So with all of that specialist ragging behind me, let’s break it down into how you can prevent this disaster on your site (or your company’s).
1. Get a reporting sample. You’re looking for a report that talks about what’s happening with rankings, and WHY. If your SEO doesn’t have an opinion on why movement is happening, what are the chances they’ll come up with a tactic to change it? You’re also looking for post-click actions in the report, and again WHY. Your SEO should be trying to drive good traffic to your site, and either diagnosing or doing their best to help your usability / analytics resources diagnose any issues with on-site performance. Let’s not forget, you’re investing in SEO for a return, not a trophy case.
2. Talk to the SEOs, both the ones who do the selling and the ones who will do the actual work. Larger agencies may put less-experienced staff on the busy work of keyword buildouts and title + meta creation, but guess what? Those are arguably the most important tasks. Make sure that their process includes a review and some sort of feedback from analytics and usability folks (or a savvy businessperson) to validate that these phrases will work on these pages. Refine keyword lists, even if you have to give up search frequency, to best fit the purpose of the target page and ensure a positive experience for visitors if engines choose that page as the best for that key phrase (and of course, there’s no guarantee of that). Also, try to get a general feel for the value they place on the return of the program. If it “feels” like all they care about is rankings, they aren’t going to be a true partner.
3. Demand 6-month performance reviews. This is something that both you and your SEO should love. One of the hardest parts of search engine optimization is deciding how to keep it going. By having in-depth reviews of engine and site-side performance every 6 months, the strategy and tactics can be refined to continue boosting performance over time as the algorithm and competitive aggression changes.
Follow these three rules and I’m sure you’ll be shocked by how many candidates are weeded out. Remember that the professions of SEO and analytics / usability are VERY different, so you shouldn’t expect to get one person that wears both hats: that would just be a waste of a specialist’s time. What you’re looking for is a specialist who is aware of and cares about the collateral effects of their work. This person/agency will apply their specialized knowledge responsibly so you don’t end up with mutant fish at the bottom of your sea.
3 Comments~ FIN ~
Aug
11
Here’s an exercise that will make you a better analyst and a better businessperson. Even if you’re not the “web analytics guy,” do this anyhow and translate it to how you think about your web site.
Go to the mall and prop yourself down on a bench in front of of a store (how about the Apple store, because it has glass windows and you can usually tell what someone bought. And if you get the hankering to buy something, as a shareholder, I’d appreciate it). What do you see? People walk in, mill about inside, and then walk out, some of whom are carrying boxes and bags full of their shiny new toys.
If you were to measure the “engagement” of the Apple store, what would it look like? You’d have metrics for entries, metrics for exits, and metrics for conversion rates. If you had particularly good eyesight, you might even be able to tell how many products people looked at, what those products were, how long they played with them, whether they talked to a salesperson, and infer whether that “touchy-feely” experience influenced (or was indicative of) purchase potential. If someone was passing out bright orange flyers at the entrance of the mall, you could also see how successful that was at driving traffic to the store (they’d be carrying them). And if you were to come up with a set of metrics to outline, “Apple Store Performance,” it would probably look like this:
Store Use:
- Entries
- Entries with flyers
- Time in store
Interactions:
- iPod views, duration
- Laptop views, duration
- Desktop views, duration
- % Talked to salesperson, duration
Conversion:
- iPod sales
- Laptop sales
- Desktop sales
- Exits
So you get it – we have a bunch of simple metrics that we can start to tie together. Were people with orange flyers more likely to buy? Were people who talked to a salesperson longer than 4 minutes more likely to buy? Were people that used the iPod for longer than 1 minute more likely to buy? You get it. What we’re trying to figure out is what’s happening in the middle, because we know that the middle determines whether people walk out with armfulls of shareholder value.
We also know that handing out those flyers is getting more people in the top of the funnel, which, if we can preserve conversion rates, means more buyers coming out of the bottom of the funnel. We may even choose to have salespeople treat orange flyer holders differently.
So let’s start translating:
- Entries = Visits
- Flyers = Paid Search, Natural Search, Display, Email campaigns
- Views, durations = Page Views, Product Views, “engagement” (whatever that means)
- Sales = Conversions
So my question is this: once we do this translation, are we still looking at what happens in the middle? Or more importantly, are we still doing something about what we’re seeing in the middle? The answer is usually no.
In a retail store, the employees polish the iPods, train employees how to better talk to customers, and physically re-arrange the store in an effort to make the middle more effective. Online, we watch, objectively, as if we are a third party being. We measure. We say that the orange pamphlets aren’t working because they’re not walking out with bags, but we didn’t notice that the salesperson assigned to talk to the orange pamphlet holders has huge pit stans and smells like Taco Bell. We’re seeing that he’s talking to people for over 3.5 minutes, the magic number that increases conversion rate, but we’re not seeing that for the first minute, he’s talking about how hung over he is. We’re boiling qualitative things into quantitative metrics in an effort to understand what works, and we’re missing the whole point: we’re getting this information to change it, not just to know it.
In the store, this guy’s hygiene is obvious. Online, we might not realize the qualitative aspects of a landing page make a keyword with a high click-through-rate a repulsive experience when the same page does great for another keyword. What I’m talking about is especially true for your marketing efforts – paid search, SEO, display, email, social, etc. If you’re not thinking of the middle (judging a media by its endpoints, cost and return), you’re not learning anything. Your lowest performing media could be your best performing media if you changed a few things. But to know what to change, you have to be looking at the middle.
When Avinash talks about the 90/10 rule, I believe this is where he’s going with it. We’ve created huge, sophisticaed tools that can tell us wonderful things in the language of numbers, but we need people looking at why, telling the stories, understanding what can be done in the middle to get more bags out the door.
3 Comments~ FIN ~
Jul
22
Are you an analytics beagle or a performance wolf?
Filed Under Practicing Web Analytics | Leave a Comment
So, I can’t even begin to describe how lame I feel this analogy is, but it’s effective, I think.
Beagles are great hunters. They have an incredible nose, they’re very smart, they have great endurance and they’re faster than you’d ever imagine.
Wolves are also great hunters. Fast, silent, great instincts, and accurate.
So what’s the difference between the two? While one is pointing and barking at the issue at hand, hoping that someone else will finish the job, the wolf is dining.
What I see these days is a lot of beagles. Incredibly smart, diligent, and creative people who bark and point at problems, letting the company know, “Over here guys! Come take a look!” The beagles will hang out comfortably by the fire until they’re called to duty, asked by the master to find what the company is hunting for. They’re given a scent and they’re off on the trail. The company pours some puppy chow into their bank account every few weeks and pats them on the head as they progress in skill. If they do a great job, they might get a nicer mat to sleep on or a shiny new collar. But someone else is called in to handle the problem, and those people end up getting ahead.
The wolf, however, actively hunts. They don’t wait for a master to ask them to eat, they feel the hunger and get off their wolf asses and kill something. They drag the kill back to the den and feed their young, training them how to kill for themselves, be independent, defend themselves. They also choose and plan their approach, starting with nearby meals that will be easier to bring down. When they need to bring down a large beast, they collaborate seamlessly, knowing there will be plenty of time to argue over who eats first after the beast is tackled.
So, if you’re an analytics practitioner, what do you do? If you only know how to follow a scent, learn how to kill. Learn design. Learn usability. Learn HTML, PHP, SQL, etc. Learn the financial backbone of the business – the core drivers of success. Stop talking about page views and start talking about profit. Actively seek problems that you know in your gut and bring them down with the data you know how to retrieve better than anyone. Stop settling for being told what to look for, where to go. Start getting hungry, and take all of the credit that’s due to you, sharing the credit that’s due others. Teach others how to kill. Be a wolf.
If you’re a company struggling with crap analysts, make some good hires, pay for training, and reward performance. Not report-producing performance, PROFIT-enhancing performance. Dis-reward (I know that’s not a word, thank you very much) the soft performance measures of old. The kid staying until 9:00 pm to produce a report that shows you 5 data points and 0 recommendations is on the chopping block. The kid staying until 4:00 and taking a 2-hour lunch who gives you 1 data point to support 10 recommendations is going to the corner office, and she desperately needs a better manager who can challenge her and set the bar.
Are you a beagle or a wolf today? Don’t be a beagle tomorrow.
Leave a Comment~ FIN ~
Jul
16
3.5 things that keep you from finding good web analytics people
Filed Under Web Analytics in Business | Leave a Comment
Probably the most frequently asked question I get is, “Evan, how can I tell a good analyst from a bad one? What should I be looking for in a resume?”
Of course, the astute analyst will immediately recognize that those are two questions. Well done!
Usually, my answer is a simple one: “You probably already have everything you need; you just need to start asking more / better questions of the people you have.” The truth is that most businesses have already done a pretty decent job of hiring, believe it or not. People who know me well will probably be surprised to hear me say that, but it’s true.
So what’s the problem? Well, cue the first the three “things”:
Thing 1: Good web analytics people are carefully disguised as . . . your own employees!
What? Yes, it’s true. Good web analytics practitioners are all over the place. They’re designers. They’re usability people. They’re your product managers, and even sometimes your IT people. What these people don’t have, though, is your trust, the right tools and training, and an environment where they can collaborate and learn.
It’s very important that a large organization acknowledges the value of a healthy cat fight, but demands that each fighter does their homework. Most everyone has an opinion on how things should be done, and a lot of those opinions are valid, but nobody is going to concede or collaborate if there isn’t some degree of data involved, and that’s where analytics comes in. See, these people are yearning to be part-time analysts, and some of them full-time, if it wasn’t seen as a reporting job.
So what do you do? First, when people say, “I’d like to learn more about . . .,” make sure your gut reaction is not asking them why they want to learn something new, or even worse, why they don’t know this stuff already. They’re asking you how to be a better employee, so don’t be a jerk – help them help you, Jerry Maguire style.
Thing 2: People with web analytics “experience” are most often not your most promising web analytics candidates.
This is the part where you start wondering if there is any good news. It’s coming, I promise. But first, let’s keep chipping away at reality.
The sad truth is this: very, very few businesses have figured out how to take advantage of web analytics in their organization. They have competitive intelligence people, business intelligence people, product strategists, upper management, designers, and more, all telling the business, “Here’s what we need to do differently.” Add to that the fact that most of these people are using different tools to measure and justify what they’re dreaming up, so they all get different numbers for the same thing, which makes the executives not trust any of it, or side with some numbers and not with others, based on which vendor paid for the more expensive steak dinner.
Yes, we know it’s silly, but it’s true. No amount of us analysts saying, “That’s just silly logic,” is going to change this behavior, so it’s incumbent on us to figure out another, more successful way of convincing management that even though the numbers aren’t the “right”, they’re all “correct.”
The result of this organizational difficulty is that most people who have been hired as analysts are almost exclusively a source of reporting for a business. While the analysts may add some text and reasoning to the report that they’re delivering, that only makes it a report with text, not an analytics deliverable. These analysts, although they may indeed be talented, have been so stymied by their previous organization that they are too pacifist to argue their valid points and force your company to accommodate their valuable input.
You need fighters, and most current analysts have been too badly beaten: they’re scared dogs.
Thing 3: Your interview process prevents you from hiring good people.
Looking under the category of “hard to change,” we’ll see this one at the top of the list. Here’s the deal: the people who are going to be interviewing for the best web analytics person out there are going to be threatened as hell when they actually find one. Let’s take Avinash Kaushik, for example. He might be the nicest guy on the planet, but people know that if they hire him, things are going to change and that scares the hell out of them.
Things like peoples’ job performance, accuracy of insights, flaws in designs, poor calls to action, poor product planning and / or research will be revealed. Gaps in knowledge will be exposed where people should be experts. Flawed management techniques and approaches toward resolving conflict will be called to the mattresses. Those who have decided to go with anecdotal choice A over anecdotal choice B will be asked why they didn’t consider data-driven choice C.
It’s important that the people doing the interviewing and hiring are able to get their own fears and egos out of the way. I think that most people can easily identify the people who are intimidated or defensive, so just make sure that you involve some good people who are more interested in company success than personal protection, especially if they are junior to the true decision makers – they will quickly identify people they can learn from and want them nearby.
Thing 3.5: Your idea of an appropriate salary is way out of whack.
What fraction of a percent does your head web analytics guy have to move the conversion rate to pay for his whole salary in two weeks? If you were to really Moses this one and part the seas for a talented person to point out flaws and implement recommendations, what would a 0.5% increase in conversion rate mean to your business – or even a 0.3% or o.1%? If you’re a major (or even a middle-tier) online retailer, would this not equal hundreds of thousands of dollars of additional revenue per month?
So just think about it – if you want to put someone behind the steering wheel of your company’s data-driven decision making, how much should you be paying that person? If you can get past what “analyst” means in the web space and start thinking of what “analyst” means in the financial space, for example, consider the potential impact that someone can have when they are highly effective at pointing out barriers to conversion AND architecting specific, implementable solutions. I can tell you, it’s enormous.
Leave a Comment~ FIN ~