|
|
|
Page 4: Appendix A — Interview with Bob LinDell
|
|
In order to try and blunt the criticism of test bias, I had a neutral, objective third party observe one of the testing sessions. Below is my interview the observer, Bob LinDell.
Q: What were the most significant differences between user performance on the HTML site vs. the Flash site?
Users on the HTML site were able to find product details much faster and were able to make faster and better decisions more quickly. On the Flash site the users were often frustrated with the fact that they couldn't see any detail at all, not even the names of the products. When users were assigned a task to find a specific item or compare two items, they often went right past the item they needed when it was displayed on the screen. | "Users on the HTML site were able to find product details much faster and were able to make faster and better decisions more quickly." |
For instance, when trying to compare the gold and platinum Garland necklace, the users didn't realize that the product they wanted to see was displayed on the screen, they just had to click on it. The HTML site at least gives the name (GARLAND diamond necklace in platinum) of the product. In the case of the flash site, we had users who knew *exactly* what they wanted, but were unable to find it because the little thumbnails in didn't give enough information to the user. The HTML site allowed the user to quickly scan the page for keywords (gold, Garland) to find the product they wanted.
One of the largest differences in the two sites was the lack of a "Back" button in the Flash site. This may have to do with the expertise of the users (see below) | "One of the largest differences in the two sites was the lack of a "Back" button in the Flash site." |
but navigation through the HTML site was much faster, and users were able to more easily back themselves out of a section they didn't want to be in by using the browser's "Back" button. The Flash interface had no button that allowed the users to step back in their history and reverse-navigate themselves out of dead ends. Despite the best efforts at on-screen navigation, users still wanted to click on the "Back" button.
Q: Having previously used the HTML and Flash versions yourself, what surprised you about how users interacted with both sites?
Some users abandoned the browsing features of both sites and headed directly to the search area. This may be in part because the people used as test subjects were more advanced and comfortable with leaving the categorization and going directly for information. So much time is spent on the categorization and presentation of products, I was surprised to see that users often spent little time in that area.
Q: In your opinion, was there any testing bias (HTML vs. Flash) on the part of the tester or the test itself?
Yes. The group of people that the testers were selected from was not an accurate representation of the web world. The request was put out for testers to a group of people who spend a majority of their day connected to the Internet in some way or another. Some of the testers have designed entire e-commerce sites in the past and have a very good understanding of how sites are structured, how products are normally categorized and common navigational themes used throughout the commerce sites. I think this would definitely skew the results from what I would expect from a group of "normal" internet users. The more advanced users are used to scanning websites for information, they are more bold when it comes to "exploring" on the site and are more confident that they will not "lose their place" in the site or get stuck down a navigational tree and not know how to get back to the top level. Perhaps also stemming from the advanced user base was the tendency to use the search functionality of the site, coming from experiences with sometimes hundreds of other commerce sites, the users knew that the best way to find specific products was not to browse, but to search.
|
|
|
|