Well, my grumpy old men streak continues: not only do I not love the definition of web analytics, I am also finding myself fussing about what we consider our web analytics tools; the things we use as web analysts that help us do the job. Tools imply two things: mechanical advantage and outcomes (hammers increase your power; the outcome is two things stuck together by a nail), and the tools we choose to wield determine what level of mechanical advantage and what outcomes we should expect.

The trouble is that I think the majority of us look inside our toolboxes and we are blind to the best tools available for the job at hand. And, you guessed it, both our mechanical advantage AND our outcomes suffer, making us lose focus on the job at hand.

That job is cash flow.

Period.

So let’s take a look at the various schools of thought when it comes to web analytics tools, and peer into the toolbox through their eyes.

School 1 (Bachelor’s Degree): Tools are Data-Gathering Thingies

Yes, all web data-gathering thingies are web analytics tools. BUT not all web analytics tools are data-gathering thingies.

There is a lot of attention paid to tools that are relentlessly created and improved to get you more and more insight into what is happening out there. These tools render thousands of reports at this point, they can tell you what people cut and paste from your site, what users do in flash, what their income level is, and whether they have an innie or an outie. These tools are like flashlights. When you crawl into the attic, they allow you to find more and more things that were already there to begin with.

Mechanical advantage is how much the tool can show you (and how easily you can knit a data sweater), and the outcome is awareness/knowledge/data puke, etc.

If you relegate yourself to this school and think that these are the tools in your Batman utility belt, it’s no wonder nobody asks you for any thought. If you think the tools at your disposal are just dragnets that you sort through and clean up in Excel to satisfy requests for reports, that’s all your output will ever be, and that’s all your value will ever be.

School 2 (Master’s): Tools are Reports, Data, Analysis to bend ears and drive change!

On the second step to enlightenment, you start to see some other tools in your utility belt. These are tools that can stun your internal enemies and help turn naysayers into advocates. Recognizing reports, analysis and insight as tools (rather than work product) means that your actual work product will be INFLUENCE. At this level, the analyst starts to realize that they wield some power and can use these tools to turn heads. As the tools have more polish (more complete and focused analysis and conclusions), they provide more mechanical advantage.

So at this level, the outcome is influence, and with more polish and reputation, you gain mechanical advantage in the ratio of effort in per influence out.

If you’re at this level, you are making a contribution to the company by getting people in other departments thinking. Oftentimes, those people in other departments are thinking about how to make your death look like an accident, once the flaws in their thinking are exposed, but hey, at least they’re thinking! You are gaining value, but other people are still doing the work. When things don’t work, you get to tell the organization that a correction is needed. But you don’t make that correction, and you don’t get the credit when the correction works, because what the correction was wasn’t your idea. Identifying what does not work is not equal to identifying what does work.

School 3 (PhD): Tools are other disciplines (UX, SEM, SEO, IA), ACTION

At the super ninja badass level, the best tools in your arsenal finally come into focus. The best tool is always ACTION. This means you have shone your flashlight on an issue, you have brought the right people in and influenced them, and now you’re about to whip out your mighty hammer of DO THIS. At this level, you are the general practitioner. You have done the rotations in the other disciplines, and you can send the majority of your patients home healthy without redirecting them to a specialist. But you also know when to not cross the line and take on an open-heart surgery.

This is what makes successful web analytics (and business) people: the realization that your real tools are not the the tools used to identify the trend, and they are not the tools used to communicate that the trend is a bad one that needs to be fixed; they are the ones used to actually solve the problem and improve the situation.

At analytics nirvana, your tools produce cold, hard, beautiful Benjamins as an output, and your mechanical advantage is brought about by your ability to synthesize information and ideas: drawing on all of the competing priorities that go into a finished interface or architecture, and recommending (or testing) the ideas that balance all priorities for the betterment of the user and the bottom line. Very, very few people in your organization will be capable of balancing competing priorities (your company’s compensation plan virtually guarantees it), so solving one problem while also solving / minimizing downside to others is an incredibly valuable output that almost nobody in your company can do as well as you can. The only way that you can see the problem from everyone’s perspective, however, is to own their perspective by knowing how to do what they do.

Getting your PhD

I don’t think you can call yourself a web analytics ninja without a decent degree of competency in the following disciplines:

• Usability
• Information Architecture
• SEO
• Various web marketing/advertising tactics (PPC, display, email)
• Social Media (I do not classify this as web marketing or advertising, and I don’t believe you should, either)
• Design (it’s okay if you suck at it, but put some effort into it so you have credz when you start drawing out ideas)
• Copywriting
• Making sites (HTML, .NET, PHP, SQL, Javascript, etc.)

If you’re missing skills, don’t fret. While it can take considerable time for you to learn about all of this stuff, you’re going to love it because you’ll immediately see new action tools popping up all around you. Hopefully, your organization is one where people love to cross boundaries and teach different disciplines about their own specialties (picture a medical school). I don’t see this much, but if you’re around these types of people, consider yourself blessed.

If you don’t work at a company like this, either go to a company that does, or you’ll have to try your best to get the med school mentality: if the cardiologist faints in the middle of a procedure, the remaining people in the room have enough working knowledge to get the patient out of the situation alive. If you can’t say the same for your organization, that may be a BIG problem.

You might also like:

Are you a web analytics beagle or a performance wolf? (Lame title on purpose)

Google announced a secure version of their search, located at https://www.google.com. This is great if you’re worried about the black helicopters, but for analytics, this is very not great at the moment.

As good buddy and coworker Brian Ussery pointed out yesterday, when you come from https, for security reasons, your referral data is NOT retained. This means that if people search and find your site on Google SSL, you’ll never know that they came from search — they’ll appear to be direct traffic to your site.

The exception to this rule is if they click on a paid search ad that has your campaign parameters (or auto-tagged GCLID parameter) present. They will then be recorded as paid search.

I’d recommend that in the next month or so, you take a look at what percentage of your paid search is coming from Google SSL to get a feel for whether your users are taking advantage of this new offering. If you’re getting a significant percentage of traffic from the https site, you may infer that users are also generating natural search traffic from the same site, while you’re blind to it.

A good way to go about this would be to write a little script that looks at document.referrer. If it is blank (or your own site), but the URI has GCLID or utm_* parameters present, drop a note in your custom variable to identify that visit as coming from Google SSL. After a few weeks, you’ll have a good idea of what percentage of your paid search traffic is coming from the secure site vs the folks coming from regular Google, or ReGoogle.

I am making up words now so I can take credit for them in 5 years.

Go take a look at Brian’s blog post, and the rest of his blog, which is pretty excellent.

The folks over at SiteTuners have been working on a great piece of technology for simulating eye tracking.  But be warned: it’s in beta and does come with a little bit of baggage so far.  But Tim Ash is one smart dude, and he’s out there gathering feedback on forums and blogs to make the tool a winner when it’s out for good.

People like tools like this because they are intended to solve arguments quickly.  We can use data from users or computer simulations to tell the bonehead CEO that his ideas of putting jumping pink kittens and 24 calls to actions are bad ones. And this is great, so long as the data is inherently valuable.

The trouble with AttentionWizard so far is that it evaluates web sites like you would evaluate photography or art, using design concepts like contrast, weight, etc., with an obvious dose of “top-leftedness,” meaning that the tool anticipates where headers and text would start and shows greater focus there. And it is also seemingly weighted heavily toward identifying outlier colors.

But that’s not exactly how people evaluate web sites. The classic example is banner ads and how conditioned people are to avoid them, regardless of how bright and obnoxious they are.  In AttentionWizard’s demo video, you can see this error in their demo of 1-800-flowers.com, where the house ad gets a lot more attention than it would in the real world.

But to be fair, this demo does reach good conclusions, and you can see how much more straightforward the end result is.  But the point is it reaches good conclusions with the wrong data.

Unfortunately, we live in a phase of web analytics where most of the tool data is taken at face value.  So, people see a heat map and believe that’s reality.  Or they get a report about keyword performance and assume that keyword A is better than keyword B, just because it has a higher ROI (but without seeing if the landing pages make sense, if the prices are competitive, and on and on).

Tim makes great points on his site about what this tool is and isn’t, but right now that’s like John Stewart making points about how his political opinions are just his own: it’s irresponsible because it ignores how powerful credible sources are, and to both Tim and John’s credit, they are credible sources because of their history (and charm, of course!).

Below, I’ve shown a few demos of where AttentionWizard deviates from human behavior so radically that it can actually drive the wrong decisions.  And I’m not trying to show pictures of half-naked women or anything that the tool shouldn’t or couldn’t technically be capable of, because I do think that’s an unfair test.

Test 1: Pattern recognition

Here’s one that humans will get right away.  Just try to take your eye off of the star.  AttentionWizard didn’t get it at all.

In the world of the web, this is important for low-contrast sites that might use shapes and graphical weight to point things out.  We probably won’t see an example this simple online, but it is possible. I’m envisioning package design – you see 11 identical-shaped shampoo bottles on Amazon.com and one crazy-shaped one.  Where does the eye go?

Test 2: Position-sensitivity

So here’s one where I’ve placed calls to action at 1150 pixels.  This one tests to see whether the tool will hold up to the boardroom when someone says, “See, people will see this button at the bottom of the page!” Unfortunately, it seems that the tool isn’t yet sensitive to where things are and what screen sizes are most popular, which is definitely something that needs work before AttentionWizard is ready for its close-up.

I think this is an easy win for the tool – something that shouldn’t be too tough to change.

So Tim is to be applauded on creating a great technology.  But I would strongly urge people to not circulate the data it renders quite yet, not because the tool is bad, but because data is notoriously misinterpreted in the average company – taken for face value.

People can caveat their tools all day long, just like Charles Barkley could say he was not a role model, but we have to be responsible to the real world and produce things that don’t send mixed messages with the expectation that people are smart enough to filter the data.  They aren’t.  Not yet.