home | golf | booze | film | music | stocks | web | misc  archive 
you are here: dack.com > web > flash vs. html: a usability test > page 3
inside web:
/web overview
The excellent intranet cost analyzer
Best practices for designing shopping cart and checkout interfaces
Flash is evil
The web economy bullshit generator
Flash vs. HTML: a usability test

Page 3: Observations and Conclusion
I'll be the first one to admit that this is not a perfect test. Plus, it's only a single test between two web sites. However, in this case, the HTML site graded out better than the Flash site — by a wide margin — in both objective and subjective measures.

Objective Measurement
On the timed questions, those using the HTML site answered the questions faster than those using the Flash site. In the case of the "comparison of fact" question, testers using the Flash site took twice as long (on average) to answer the question. The results may have been even more lopsided had users not abandoned the browsing method and headed for the HTML-based search page to get answers to the questions.

Testers using the HTML site completed the timed task more than 40% faster than those using the Flash site. In a development that should be alarming to the web folks at Tiffany.com, two of the four people using the Flash site had serious problems with the Add to Cart functionality, thinking they had added items to their cart when they hadn't.

Subjective Measurement
Simply timing how long it took people to answer questions and complete tasks may not be a definitive method of comparing one web site against another. What about a site's appearance, its graphics, its "cool factor?" Rating the sites' more ethereal qualities was the idea behind the post-test survey.

If you were pulling for the Flash site in this matchup, the results aren't pretty. The Flash site scored higher than the HTML site on only a single criterion: Readability of Content. The HTML site rated much higher on Overall Ease of Use and Speed, but it also rated higher than the Flash site on Coolness and Reinforcement of the Tiffany Brand, criteria one would expect the Flash site to excel in.

But, But, But What About the Brand?!
An oft-heard argument in favor of Flash sites is that they're better than HTML sites at expressing a company's brand. But in this study people rated the HTML site better than the Flash site at reinforcing the Tiffany brand. This could be the result of the limited tester pool or other problems with the test, but I don't think so. It's because the strength of a company's brand on the Web — even for a so-called "branding site" — is about more than animation, rollovers, and a pulsating soundtrack. It's also about ease of finding information, speed, and overall ease of use, or, The Complete User Experience, something most Flash sites just don't provide as well as their HTML equivalents.

I'm hopeful that this usability test will be the beginning of more research into the overall performance of Flash sites, because information about the topic is essentially nonexistent. In some respects, Flash developers are flying blind, creating entire sites without any information about how, in general, users rate Flash sites compared to an HTML equivalent.

Until more research is available, developers may want to think twice before building the next "wall-to-wall" Flash site. If the results of this study are to be believed, it would indeed be foolish to build an entire site in Flash if the best interests of the client and their customers were kept in mind.

« back to results forward to appendix A »

flash vs. html
» Index
» 1 - Introduction
» 2 - Results
» 3 - Observations
» 4 - Appendix A
» 5 - Appendix B

email: dack@dack.com© 1998-2023 dack.com