Apply Usability Methodologies in Intranet Information Architecture in a Real World Context Part III

Printer Friendly Version

This is the third and fourth parts in a series of five articles on the implementation of usability methodologies in the development of an intranet's information architecture. The series is based on a project developed for an educational institution.

Prototype

Our next step was to develop a prototype of the information architecture. We had enough information from the stakeholder analysis and UNA to get a sense for what was needed in the intranet, but we wanted to begin to test our understanding of the user needs we had identified. The prototype was HTML based, but in the format of a site map, rather than an actual web-site. We had considered using a paper prototype because we were aware that when users are presented with a higher fidelity prototype they can get caught up in lower level issues such as the colours, or the fact that the prototype does not work flawlessly. We wanted to prevent users from thinking the prototype was near completion. However, since the site was so big, with many pages (although only two or three levels deep), it would have been cumbersome to make a paper prototype. We also wanted to time the users in the confirmation testing and timing using a paper prototype is meaningless because it takes so long to flip the pages. Thus, we decided on an electronic prototype, but in the form of a site map, not a web site. This way, users would focus more on the information architecture, and not the look and feel or smoothness of the prototype.

Our decision to use an HTML site-map prototype turned out to be the right one for the purposes of our project. It was possible to conduct heuristic evaluations on this model because we couldn’t see where all the links took us until we clicked on them. It was also useful for the confirmation study because the users were aware that the prototype was not reflective of the look and feel of the intranet site. They accepted that it was simply an information architecture. Had we structured the prototype as a Web site, we may have heard comments that the site was boring or incomplete. Time-wise, it would have been easier to make a paper prototype, but in the end it was worth the extra effort because our usability tests took very little time for users to complete.

Heuristic Evaluation

The next step in our process was to conduct a heuristic evaluation. Heuristic evaluation is a method intended to identify potential usability problems with a system as well as aspects that may need improvement. It is informal and subjective and is performed from the point of view of end users. We felt at this point that heuristic evaluation would be an appropriate way to identify problems with the information architecture because it is quick and easy. Rather than jumping into a usability test, we would be able to get a sense for what some of the problems may be before we gave the prototype to users. If there were any show-stopping problems, we could fix them early in our design process. The confirmation test would serve to confirm or disconfirm the problems we identified in this stage.

Two evaluators performed separate heuristic evaluations. They were both “experts” in usability, in that they were familiar with methods in HCI, and with the heuristic evaluation procedure. They were not, however, at all familiar with the organization. This fact was beneficial in the sense that the evaluators would be testing the ease of use of the prototype for users who would be new to the organization, but not for users who had been at the organization for a long time, and who knew how the system currently worked.

The evaluators were given the prototype and explored the HTML site map, trying to make it “crash”. They were only testing the information architecture, but in fact, they identified a few problems with the prototype itself, which was useful to us. The evaluators tried to imagine whether the users would know where to look for the information in the site and asked themselves questions like, “if I were looking for parking information, would I know to look here?”

Each evaluator took a slightly different approach to the evaluation. One evaluator was not comfortable categorizing the usability problems to any of the well-used heuristics (such as Jacob Nielsen’s “Ten Usability Heuristics” http://www.useit.com/papers/heuristic/heuristic_list.html), but rated the problems in terms of severity. The other evaluator used a set of four heuristics, but did not rate severity.

Several problems with the prototype and the architecture were identified, and fixed before the confirmation tests. All of these problems were then made into tasks for the confirmation study. By doing so, we were testing to see whether the problems we identified would be actual problems for users.

Despite the fact that we gleaned valuable information from this methodology, it proved to be the most problematic for us. The biggest problem was that there were few established heuristics that were appropriate for an information architecture, unlike those available for a full-fledged website. One of the evaluators could not assign any heuristics to the problems she identified and the other only found four that were appropriate. Since there were few problems identified, it was not problematic that we couldn’t categorize them. We were able to test all of them in the usability tests. However, in this case we were not able to follow the textbook methods. In future evaluations we will develop our own set of heuristics based on our experience in intranet development and industry best practices.

We also were not experts in the workings of Organization X. Some of the problems we identified with the prototype may have only been problematic to someone who didn’t know the organization. However, we felt that our lack of knowledge was beneficial, as we wanted to design a site, which was easy for all employees to use, whether new or old to the organization.

Finally, the heuristic evaluation did identify some problems, but not all of them. Some of the issues raised by the confirmation test had not been previously identified by the heuristic evaluation. This finding points to the importance of using multiple methods and following-up expert evaluations with user testing.

Next Article: Confirmation Study, Other Issues and Conclusions