A recent post in Occem’s Razor claims that we can’t measure users’ engagement using web analytics tools.
Although I’m a big fan of this blog, I got to say that this time I disagree with the conclusions in this post.
Avinash and Theo claims that the reason we can’t measure engagement is that we can’t differentiate between positive and negative engagement.
I think they are right to some degree. It’s true that if you only look at criteria such as number of visits, recency and depth of visit you won’t really understand engagement.
But that’s exactly why we in NuConomy decided to offer a different way to measure engagement.
Engagement is not just how many pages I’ve seen in a site. It is about how engaged I am in the brand and activities of the site.
Engagement is the combination of all of the user interactions in the site. It could go from purchases, media uploads, ratings to comments, share with friends and many other things.
Let’s take an example:
User A, goes into YouTube. He goes through a few channels and user pages. He browse about 30 pages which brings him deep into the site, but he doesn’t watch even a single video.
User B goes into YouTube. He watch the top 5 videos in the home page. He rate 3 of them and make comments on the other two. He than continue to upload two videos of its own. During all this time, he doesn’t browse more than 5 pages in the site.
Now - Which of the users do you think is more engaged with the YouTube brand?
This kind of measurement is exactly what’s in the heart of our product NuConomy Studio. We let you measure all kind of user interactions, and even combine them to get a definite "engagement" rank to each of your users or content pieces.
It’s true that even than we can’t be certain about the how much a user love the brand, but we believe it brings you very close to that.
Shahar - Good stuff here. This has been a well discussed topic in the web analytics space for many months now (at least a year or so). Another well-respected luminary in the WA space, Eric Peterson looks at it in a way that resembles your thinking.
http://blog.webanalyticsdemystified.com/weblog/2007/10/example-uses-of-the-visitor-engagement-metric.html
Also, the vendors are starting to speak about visitor engagement as well: http://www.webtrends.com/Products/WebTrendsScore.aspx
This is great stuff and a step in teh right direction. The question is - how do you value the additional level of “engagement” and how can you report and measure the same from web-wide activities (e.g. Flickr galleries, blog post/comment “discussions”)?
We are wrestling with these issues as well:
http://johnbell.typepad.com/weblog/2008/02/why-comparing-s.html
Shahar - this is what I need to do to effectively analyze the social networking communities I have developed for lifelong learning - I have a number of reserach projects going, all of them seeking to capture the engagement that goes on within an online community among users and artefacts and groups -
Lets say that you can measure engagement as described/defined - “the combination of all of the user interactions in the site”.
My interest is in how the results are interperted and what relevant content/response/action will be delivered in return.
I believe the drive to measure the degree/depth of engagement is critical. I am concerned that marketers will respond inappropriately or worse inadequately. There always seems to be a dis-connect in this area as the technology evolves ahead of the marketer and the culture in which he/she exists.
Perhaps your technology could include a “sister” component/module that helps facilitate/interpert a set of recommendations - that are likely to insure/produce even more engagement! Utilizing your engine + engagement attributes + client/product specific attributes.
WHY would you do this? To produce better results for your clients.
Also, I went back and read your other posts. I notice a core theme which I think will serve you well in this endeavor. You are consumer/prospect centric - you recognize their role and power in the process. New media (in particular,and being interactive) should never under estimate the significance of the consumer. Like you I cringe at the use of old media tatics/strategy being applied to new media.
Shahar,
We too deeply believe in using engagement as a key component in measuring web site success. However, I would like to take it one step further.
The problem with many metrics and reports is that traditionally they have not been actionable. They simply reported on what happened in the past, without any context or real knowledge. However, when KPI’s came along they taught us the goal of associating metrics with key business objectives. By trending KPI’s over time, key stakeholders for the website are able to understand whether or not the website is successful in terms of the key objectives of the web site property.
Engagement in my mind is an extension of the KPI’s. In order to use engagement you have to measure it in terms of the strategic objectives of the website. Take your youtube example - do you want the user to be engaged with the many features of the website (viewing videos, searching for videos, emailing videos), or do you want them to be exposed to as many pages for as long as possible in order to expose them to your brand and to Ad impressions.
At the end of the day, as you pointed out, engagement for those two scenarios is measured differently, but in my mind the most important part is ensuring that you are measuring engagement as it relates to the key objectives of your site.
Dan Piche
Interesting topic, and agree with your opinion.
I am very curious to understand the competitive advantage your tool offers, especially in the context of the combination of (a) web analytics consultants blogging their own opinions of what this as-yet-undefined metric comprises of and (b) web analytics vendors providing an increasing compendium of white papers on the subject.
The industry appears to be leaning towards creating a single engagement metric, whereas I think this is not addressing the root of the problem. Media becomes more participatory, therefore our measures need to address the participation. Visitors to web content interact in different ways, both in terms of their observed behaviour and in terms of their responses to content; this behavious happens on and off site, on and off line and is tremendously important in defining how to develop content that meets the needs of customers. There is no need to declare a single engagement metric, nor is there any challenge in using existing web analytics tools to achieve this. The challenge though is how to make this simpler - I’m hoping your tool offers some insight into this.
Look forward to learning more about what you’re doing!
Cheers, James.
I agree and I think Engagement is set of metrics (total time in seconds, page views per unique, % return uniques) that can be uniquely combined based on the sites business objectives. Each industry will vary and can use a subset of metrics to define engagement. The big missing piece that no one seems to focus on is the content or product owner. In the YouTube example, the video(s) is the brand or the product that determines how well YouTube does. If I have videos on multiple sites, I want to know how my brand is performing across all of them. Everyone focuses on publishers and advertisers, but what about Musicians, product managers, manufacturers…the people that have business ownership of the actual product or service being sold or viewed on multiple sites.
Let’s take an example. I am the VP of Products for Hasbro, a toy manufacturer. My director of marketing spent 30% of my budget on the Internet. How did we do? To answer this question, we need to aggregate the performance of the transformer brand across the top 15-30 sites where they are marketed and sold. Some of these are huge retailers like Walmart, some are specialty etailors like eToys - every industry has a whole subset of different site types. With affiliate relationships and web 2.0, my products and content are everywhere. I want to know the total number of unique consumers that got to a product details page related to transformers and I want to compare that against the 1,256 transformers I sold on the Internet last month - what was my true conversion, does my demand vary by state, does it vary by channel type? If I launch a $10,000 campaign in week 6, how do my metrics change in week 7? Did my conversion or engagement go up? Knowing that a publisher has an average engagement time of 35 seconds with 8 page views per unique, and a 60% demographic of Males between the ages of 18 and 30 only tells me two things: Should I advertise there and do I want my product on their site. Outside of those two questions, my visibility is very limited as to how my product or service performs and how consumers engage with it in my industry.
It’s a unique view that expands engagement to the product or content owner, and one that we are focusing on.
Best,
Josh
Hi All.
Sorry for the late response to your comments.
I got to admit that we are simply overwhelmed with the number of requests and emails we got in the last few days, and it takes us some time to go through them.
John:
You right. This is a problem. Currently we deal with in-site measurement. Using out tool we believe that you can understand the exact contribution of every action inside the site (aka - post, comments, photos, videos, etc).
Still, there is no doubt that people can be also influenced at the same time from their interactions with the same brand in other places like social networks, flickr, etc.
A company that actually tries to measure this influence is Visible Technologies (Nielsen Buzzmetric is another one).
I actually met with the guys of Visible a couple of times recently and we got to some really cool ideas on how we can combine in-site and cross web analytics together.
Doing so can answer questions such as “what is the value of good publicity?”
“Is the sentence there is no such thing as bad press is actually true?”
Michael - Write me up to shahar at nuconomy dot com.
I’ll be happy to hear more about your research.
Susan:
I think you are right. It seems that marketers tend to move much slower than we would want them, especially in many markets outside of the US.
And you are absolutely right on the direction analytics tools should take.
One of the things we worked on is the ability of you to say in the end of the day what is important for you (aka - from what activity you do money :)), and the platform will figure out automatically what kind of engagement you need to aspire to.
Behind the scenes there are some really cool data mining algorithms that calculate the affect different activities has on other activities.
But I think that the most important thing is that publishers and marketers as one must close the loop and use the insights they get from the analytics platform to actually change for better the user experience of their sites.
We decided from day one that all the information we gather must be available for easy consumption using http services. We encourage our customers to integrate the results of the reports into their site i a way that create new web experience for the user.
Without that, analytics will always stay just information for the sake of information.
Dan:
You are absolutely right. As I wrote to Susan. I believe that the number one problem of analytics today is that most people do analytics for the sake of analytics.
There is a big separation today between the analytics platforms and the operational tools.
Knowing who are your top engaging users in a UGC site is fun, but rewarding them on their contribution makes a change.
Knowing what content is the most engaging one is interesting, but pushing it to the home page create a difference.
This is one of the main things we come to solve with our platform. But still, there is much more work to be done there.
By the way - I’ll love to hear more about what you do. What you write in your site seems very interesting…
Joshua:
That’s a big problem you try to solve. It’s also a very big technical problem.
How can you collect the sales and “engagement” information about your product from hundreds of sites?
If you use your own widgets to syndicate the content, it can be done (we actually have a customer that is doing just that). But if you simply allow other sites to sell your products… this is very tough.
I don’t remember all the details, but Ford was knows just in that. They actually built the information system for all the agencies that worked with them in order to be able to control the information.
Hi Shahar,
Thanks for the response. Your sensitivity and focus on the consumer/prospect experience is remarkable. In addition, your desire to make the results easy to interpert and actionable are a win for marketers. We’re the ones who will weild these tools for the good of those we “serve”. When the direct marketing insight is not integrated with technology we have “analytics for the sake of analytics”.
Take a momant to focus on the dynamics of the market that you will be selling to. The dis-connect within the organization, marketing depts.,IT, is real. It’s not just about results, results translation and next steps. Check out the findings/insight from the following article:
http://publications.mediapost.com/index.cfm?fuseaction=Articles.san&s=76322&Nid=39299&p=431509