from Evaluation of Human-Computer Interaction Whilst Using the World Wide Web
by Nicolas J. Blaza · The Buckinghamshire College, a college of Brunel University · May 1996
4.1 Introduction
"Good HCI depends upon HCI designers and design teams having a wide range of knowledge about both humans and technology, and about how they relate to each other."
"In the first decades of computer software development, senior programmers designed text editors, operating system control languages, programming languages, and applications packages for themselves, and for their peers" (Shneiderman, 1992). Today however, with the trend towards very powerful multi-user workstations, more powerful PCs, improved communications, multitasking and multimedia technology becoming more accepted into our homes and workplaces, it is the consumer who is demanding a more user-friendly computer, and these senior programmers are having to adapt to a new way of thinking. In some cases Shneiderman writes that these intuitions may be "out of date or not appropriate".
This chapter looks at how HCI has progressed and how it is being used as a tool for designers, from the Xerox Star interface in the late 1970s to the designing of a Web page today.
"The egocentric style of the past must yield to humanity and a genuine desire to accommodate the user's skills, wishes and orientation" (Shneiderman, 1992).
In order to do this, designers therefore must seek more interaction with the users during the design phase, during the development process and throughout the system life cycle.
The place of HCI in relation to various stages of the design process, and the best procedures for assisting designers to achieve good usability design, have been studied for many years.
4.2 Usability Lab and Iterative Testing
Shneiderman (1992) writes that there are three pillars of design that can help user-interface designers to turn ideas into successful systems. They are not guaranteed to work, but they facilitate the creation of excellent systems (figure 4.1). This report is orientated around the Usability Lab and Iterative Testing "pillar". Shneiderman stresses throughout his book that each new application design has different needs, but he does suggest some guidelines that would help in the process of design.
Shneiderman recognises that professional societies of usability-laboratory staff are starting to develop in software houses (Shneiderman, 1992). This is shown in section 4.5 with a study on Sun Microsystems.
The procedures for evaluating usability vary greatly depending on the goals of the usability study. The author of this project wants to evaluate whether Web browsers are suitable for a variety of different people: "subjects should be chosen to represent the intended user community" (Shneiderman, 1992).
Figure 4.1 — Three pillars of successful user-interface development rest on a foundation of theories and experimental research. (Controlled Experimental Results; Theories and Models of Human-Computer Interaction.)
An effective technique of testing a system is to invite users to perform a few well-chosen tasks and to "think aloud" about what they are doing. The tester should be supportive to the user, not taking over or giving instructions, but prompting and listening for clues about how the user is dealing with the system. The user should also be invited to make general comments or suggestions, or respond to specific questions. Usability-laboratory testing is effective because real users perform real tasks.
4.3 Motivations for Human Factors in Design
There are five measurable human factors that Ben Shneiderman (1992) has identified which are central when evaluating the usability of a product.
- Time to learn specific functions: How long does it take for typical members of the user community to learn how to use the commands relevant to a set of tasks?
- Speed of task performance: How long does it take to carry out a benchmark set of tasks?
- Rate of errors by users: How many and what kinds of errors are made in carrying out the benchmark set of tasks?
- Human retention of commands over time: How well do users retain their knowledge after an hour, a day, or a week?
- Subjective user satisfaction: How much did the users like using various aspects of the system?
It is these five factors that the author will be basing his usability survey on.
4.4 Usability Principles
The primary objective of an interactive system should be to "allow the user to achieve particular goals in some application domain" (Dix et al., 1993).
The HUSAT Research Centre, Easton (1982) described various issues involved in the process of introducing Information Technology into the workplace, and of those the most important regarding this report are: those involving examples of the user, pilot systems, trials and experiments, progressive implementation of facilities, evaluation of users, and assistance to help the learner from another source (i.e. an organisation).
Gould and Lewis (1983) have devised a methodology from their experience and have proposed four precepts for design and usability. Preece and Keller (1989) have synthesised these concepts and proposed their own set of principles and have developed some key precepts for the process of design for usability. The essentials of these fundamental features are:
User-centred design
Designers must understand who the users will be and what tasks they will want to do, and so direct contact with the user is a must.
Participative design
A panel of expected users should work closely with the design team, and be given rough drafts of the application to enable the users to make useful contributions in the design of it.
Experimental design
Early in the development process the expected users will actually do pilot trials and use simulations, and later prototypes, to do real work. Measures should be put in place to evaluate performance and reactions of the users.
Iterative design
The difficulties revealed in user tests must be remedied by re-design, and must be repeated as often as is necessary until the usability specification is satisfied.
Some of the bigger software houses have human-factor groups or a usability laboratory that are "a source of experience and expertise in testing techniques" (Thomas, 1994; Gould et al., 1991).
4.5 Case Study: Sun Microsystems User Interface Design Centre for W.W.W Site
One such company to have a usability laboratory is the Sun Microsystems User Interface Design Centre (SMUIDC). On their Web page there are details of how they went about designing and testing their Web page before releasing it onto the W.W.W.
This particular Web site goes into a lot of detail explaining how to do good usability surveys. The author thought it would be appropriate, although not actually designing a Web page, to evaluate how the SMUIDC team go about prototyping their Web page.
4.6 Fundamental Design Concepts at SMUIDC
SMUIDC don't believe that a company can succeed on the W.W.W. just by "putting some cool stuff out there".
The usability team, before they even started to design the Web page, discussed what it should contain: "Who needs another Web site? People are suffocating from information overload, so W.W.W designers have to become much more user-oriented and provide value-added information to attract traffic to their server" (Sun Web page, 1996).
SMUIDC decided to provide value-added information such as a monthly magazine cover, and to be highly selective in choosing a small number of cover stories. "Some people don't understand the value of less is more, but if everything is highlighted, then nothing has prominence. The SMUIDC estimate that it costs the world economy about half a million dollars in lost user productivity every time one more design element is added to Sun's home page" (Sun Web page, 1996).
"It is the responsibility of the Web editor to prioritise the information space for the user and to point out a very small number of recommended information objects. The beauty of hypertext is that the user can then browse the information space further and dive deeper into the specific information of interest to that individual user" (Sun Web page, 1996).
Three major findings from the extensive usability studies were:
- People have very little patience for poorly designed WWW sites. As one user put it: "The more well-organised a page is, the more faith I will have in the info." Many other users told the SMUIDC that they would be out of a server, never to return, if they got too many server errors or "under construction" signs.
- Users don't want to scroll: information that is not on the top screen when a page comes up is only read by very interested users. In their tests, users stated that a design that made everything fit on a single page was an indication that the designers had taken care to do a good job, whereas other pages that contained several screens' worth of lists or unstructured material were an indication of sloppy work that made them question the quality of the information contained on those sites.
- Users don't want to read lots of text. Reading speeds are more than 25% slower from computer screens than from paper. Users skip over any text that they deem to be useless (e.g. welcome messages or introductory paragraphs). Instead they scan for highlighted items (e.g. hypertext links).
4.7 Testing Paper Mock-up of Homepage
The first usability studies, according to the SMUIDC team, are done on paper prototyping. The team printed out a mock-up of the "Homepage". The test was conducted by showing the page to users and asking them to first comment on their general impression of the page, and then to point to any element on the page that they thought they could click on, and tell them what they expected would happen.
This simple method provided the team with early feedback indicating the importance of a prominent placement of the month name (Sun Microsystems were planning to update the page every month, and so it is important to let the user see what month's information they are viewing). The team also discovered that the "What's Happening" bar needed to be more prominently placed.
The results from the paper testing are all recorded on video tape to assist in the evaluating at a later stage.
4.8 Traditional Usability Test of Running System
The second method of usability testing, and the most common, is by way of testing a user interface that is running on a computer within a usability lab (Sun Web page, 1996). The user sits down at the computer and starts working. Normally, users are given a series of set tasks (e.g. find information about Netscape).
For this particular type of user interface (i.e. a Web page) SMUIDC also found it important to let the user spend some time exploring the information space freely; they wanted to see what interface elements the users naturally found interesting without any prompting. The team did keep some set tasks since they also wanted to know how usable the design was for people who needed to find specific information.
The SMUIDC also did tests at other Sun offices in Europe and Asia to assess international usability of a Web page. The usability lab has a large one-way mirror that allows team members in the control room to observe the test and discuss its outcome without interrupting the test user.
4.9 Usability Testing of WWW Designs
The team used several rounds of usability testing to improve the user interface for Sun's new WWW pages, and conducted competitive usability tests where users were observed while browsing WWW pages from other companies.
The category grouping used for the main icons on the home page was derived based on a card-sorting study: several users were given a series of index cards with various concepts from the server written on them. The users were then asked to sort cards into piles, placing cards that seemed similar to them into the same pile. Concepts that were placed in the same pile by many users were deemed sufficiently similar that they should place them in the same category in the new design.
Sometimes, rather primitive HTML mock-ups were used, with many dangling links or links that pointed to pages on the old server that used an obsolete user interface. Even so, SMUIDC were able to learn a lot about how people use WWW pages. Usability testing was conducted both in the usability lab in Mountain View, and in Sun offices in several other countries around the world (to assess international usability).
In addition to studying various new design ideas, SMUIDC also conducted a usability study of Sun's old WWW design. The old style contained many usability problems, e.g. inconsistent header bars (icons on the top of Web pages acting like menu icons), but the team wanted to learn what worked well and what worked less well in the old design.
One interesting finding was that the top icon-bar did not look "clickable" (a hypertext link): "the design did not have a clickability affordance but was seen as mere decoration by most users" (Sun Web page, 1996). This specific object was redesigned immediately without waiting for the full redesign to come online. The following figure shows before and after versions of the top row of buttons in the old design:
Figure 4.1a — The Sun Web page before changes.
Figure 4.1b — The Sun Web page after changes (notice the 3D effect buttons).
Changing the button design as illustrated above resulted in 416% increased use over a two-month period (January–March 1995). Considering that the use of the server increased by "only" 45% in the same period, there can be no doubt that usability engineering worked and resulted in a significantly improved button design.
4.10 GUIs on the Web
Developing GUIs for Web pages is slightly different than developing GUIs for the "desktop", writes Erwin Callaway, author of PC Week (November 1995). "Development criteria for the two mediums do not adhere to the same rules. Web page GUI developers must consider that Web users are interactive participants with the interface and can perform a certain level of customisation. A Web GUI needs to be exceptionally clear to compensate for the massive informational space in which it exists. Developers have to work hard to give users a sense of place on the Web site so they don't get lost in a maze of hyperlinks." "Placing a logo or banner on the top of each page is one way to keep users from going astray".
Web surfers use different types of modems and Web browsers, and are connecting to Web sites for a variety of different reasons. "Users don't have to stick around if they don't like the application" (Callaway, 1995).
PC Week instigated a survey highlighting the top ten points to consider when designing Web interfaces. "People hate pages that take too long to load, and features that are annoying can drive potential Web site visitors away from the page."
When a Web page is incomplete there is usually an "under construction" sign on it, and although they are intended to be useful they are also very annoying.
Flashing or blinking hypertext and images are attractive to the human peripheral vision; the problem is, if an item on a Web page is constantly moving, it is very difficult to focus on anything else on the page. This is particularly noticeable on the search pages that are sponsored by advertisers who use bright graphics to draw the attention of the user to their product.
Bold patterns and clashing colours are often used to draw the user's attention to a page, but they can make it very difficult to read text that might sit on top of them. "People's ability to read text on a computer is 25 per cent slower than normal" (Shneiderman, 1992), so with the added hindrance of colour it is made worse. A significant percentage of the population is colour-blind, and adding 256 colours to a screen may cause items to be missed. Callaway suggests using 8-bit colour.
Some pages contain a maze of added links that can cause the user to be lost in a mass of twisted, and often redundant, cyber-paths. Callaway writes that "links to other pages or sites should be limited unless they are part of an organised plan" (1995).
Callaway points out that "there are between 50 and 75 different browsers available to users today, and multiple versions of each". Some designers incorporate features that only work with certain browsers, and Callaway concludes that "8 per cent of people still use text-only tools" (1995).
Forrester's study, "Which Browser?", says that of all overall Web site managers reported, 74 per cent of visitors to their sites use Netscape's browser. The next four were Microsoft, Spry, and Air Mosaic and America Online. Forrester noted that the figure for America Online is probably understated because the on-line service uses a custom binary system.
Although Netscape is the clear leader, those who set up Web sites still face a dilemma. Navigator offers a number of features not supported by other browsers. Pages that take advantage of those capabilities may not display well in other browsers. "Web site developers are struggling with a basic problem," said Forrester. "They must create a site that is accessible by as many people as possible. Yet, to be effective, they must push toward leading features like video, animation, and graphics."
There are three ways of dealing with the problem. The most common answer is to try to be "browser agile", which means using an automatic facility to recognise what browser the visitor is using, letting the user choose between low- or high-graphics versions, or creating pages that degrade automatically and gracefully if the user's browser does not support the latest version. Forrester found that almost half of content providers — 48 per cent — do it this way.
Another 38 per cent of respondents to Forrester's survey said they simply stick to the lowest common denominator, using no features on their sites that the most basic browsers in common use will not support. The remaining 14 per cent — promotional and entertainment sites in particular — simply ignore the less capable browsers and design their pages to look their best through the Netscape Navigator 2.0 browser.
Forrester predicted that Netscape will remain on top, with Microsoft also playing a significant role. The study had mostly harsh words for America Online, which it said offers a poor browser and will sooner or later be forced to dump it and team up with one of the market leaders. "AOL has not been able to form a world-class software development organisation," Forrester said.
4.11 Conclusions
The aim of this chapter has been to highlight the importance of human factors. Usability issues are now considered of paramount importance, and improved usability can be sought through greater attention to users and their involvement in the design process.
There are many productive avenues for assessing user performance and attitudes (Shneiderman, 1992). Written user surveys are an inexpensive and generally acceptable approach with both interviewer and interviewee. If used, a small sample of users should participate in a test survey in order to judge the level of relevant questioning and to ensure all questions are understood.
It is the concept of rapid prototyping and evaluation of users' reactions to applications that is being used by Netscape and most of the software houses today. Iterative design methods are put in place to allow for early testing of prototypes, and revisions are made based on feedback from users. New or improved versions of software are available earlier than before to allow for "beta testing" — testing by users — to find hidden bugs within the system and to report these problems back to the developers, so that small modifications to the application can be made before the official release version.
© 1996 Nicolas J. Blaza. Reconstructed online edition, 2026.
Best viewed in Netscape Navigator 2.0 or higher at 800×600 resolution.
