LiveLogger & World Usability Day Sessions

Well, the Usability Testing Methods webinar that i attended as part of World Usability Day was informative and well worth the time. Frank Spillers, who has been performing Usability tests for 10 years now was the host. Much of the first part of the presentation was to help level the table a bit, bringing those with no experience in the field up to speed on some of the simple goals (ease of use), different types of basic Usability testing (down the hallway, low-fidelity, mixed-fidelity and high-fidelity) and measurement types (qualitative vs. quantitative). And with the best practices stuff out of the way, we delved into Frank's insight and experience in testing.

It has always been my belief that everyone brings something different to the table, so for all of us in the Usability field, fresh or experienced it can't hurt to have an open mind when it comes to different styles. For instance, some of the major players in the industry have found the most effective number of participants to be 5 in order to minimize costs and time. But experts like Frank tend to think that if most of the time is spent in the setup of the study, then what is an extra couple of hours running a few more participants through in order to have some more concrete data to supply the client.

The Forgotten Metrics
Frank also discussed what are referred to as the "Forgotten Metrics", which include ease of learnability and ease of understanding, which can give valuable insight into your study, often times more value than the ever important "time on task" metric. One practice for determining ease of learnability is what is called a loop-back task, which essentially has the participant perform a very similar task to an earlier task to determine if they are able to streamline the experience this time around.

Time on Task is a metric that can also be misinterpreted. There are 2 trains of thought when it comes to this one, and i tend to believe that as long as a user is moving towards their goal successfully, then time is not as much of an issue. And Frank is of the same opinion on this issue. As long as, when you tie in the quantitative and qualitative results, the user has a positive experience on the site, because ultimately that is what they will bring with them. If you have the analytics in place already, or if you are measuring 2 very like sites, then it's worth it, but otherwise, don't knock yourself out over it.

LiveLogger
The LiveLogger tool for Usability testing was released today to mark World Usability Day by Experience Dynamics. This appears to be a really helpful and simple tool that shows the most benefit in letting you focus on your participant, the real reason you're performing your Usability test in the first place. What it does is acts as a dynamic notebook that tracks the time for you and allows you to choose from certain actions so that when you choose that action, it marks the time and lets you make a note regarding the circumstances. As anyone trying to take these types of notes can agree to, it's not always easy to keep this type of information in order and still pay full attention to your participant. Of course the tool is much more dynamic than that, for instance, all of the export features that allow you to categorize each of the user's data, from the errors that they experienced and any points of confusion along the way. But this is where i really saw the value in it.

The presentation itself was really worthwhile, Frank keeps you interested and brings a lot of insight that you can only get from spending this long in the industry. And their new product, LiveLogger, looks well worth a demo download. If you plan on performing a small usability study in the next little while, I would really recommend giving it a shot, it'll make organizing those sessions a lot easier.

Usability and Omniture Analytics in the B2B Field

Qualitative and Quantitative, Yin and Yang… There is not enough talk between the two worlds. In most Usability studies, professionals are tasked with improving the user experience, including the reduction of errors, time on task and subjective impression of a site without consideration for measuring quantifiable metrics. Site improvement recommendations, user feedback and observations are gathered, poured over and delivered to the client for the improvement of the site in question. Though the majority of usability professionals are not currently using any type of quantifiable metrics (as documented by the Usability Professionals Association in a survey from the summer of 2005), it is widely agreed upon that there should be standardized metrics in place for tracking the ROI for user-centered design processes. But with the business-to-business field, this can be a long cycle before any improvement is seen, as it is understood that the purchasing funnel is far more diverse and considerably longer than any business-to-consumer transaction.

Just recently, Omniture released a paper discussing the 5 Most Critical Steps to Track in B2B Online Marketing, which is a powerful piece that simply outlines ways to track some of the most important metrics in the B2B field. Not only does it outline these steps in a simple and tactical way, it inadvertently acknowledges, at least in a couple of areas, the best ways to track the success of usability improvements. In this document, they discuss the fact that the "B2B Buying Cycle encompasses a series of micro-conversions as prospects progress through different phases," and that these micro-conversions can act as "leading indicators of offline sales," allowing you to see improvement online and assume improvement offline, depending on the integration of your offline sales metrics to your analytics. Now, with any Usability Study, you will need a problem to fix, whether it’s known and you have some hypotheses about what to expect and how to approach the issue, or it’s unknown and is flushed out during the process of testing; but if these are being performed on a B2B type site, then the known issue can likely be tagged to any one or combination of these types of micro-conversions, or if the issue arises through testing, then creating a dashboard to measure the improvement historically as well as after the fact should be a simple process. These micro-conversions can be something as simple as video views, product comparison tools or case study downloads, depending on the target you are testing against and the issue you have on the site.

This brings up the point that, if you do not have any type of analytics in place like Omniture Sitecatalyst, or Hitbox, then ROI or online measurement are terms that you should put out of your mind. Keep things relatively qualitative and measure your offline successes and customer feedback. But if you do have an analytics solution, then you have an immediate way to quantify the success of your qualitative usability study.