UNA - Understanding the End User

A user needs analysis is crucial to the user-centred design process. Identifying issues in the requirements phase can save companies up to 100 times over what it would cost the company to fix the same problems after the system has been delivered. Once completed, a UNA report will be the blue print from which the production team can work, ensuring that the stakeholders' intranet's goals are married to the needs of the end users.

In the first article of this series, we described how our stakeholder analysis gave us an understanding for the organizational structure, and its internal and external communications systems and processes. We learned how people communicate within the organization, as well as how the employee population is segmented into three user groups for the intranet's audiences. Based on this information, we developed an initial draft of the intranet site map, and compiled a list of interview candidates. Our sample included a minimum of five end users from each of the three end user groups: Support Staff, Faculty and Management. In total we interviewed twenty-eight end users.

For this phase of the project, we decided on semi-structured interviews combined with design walkthroughs, contextual interviews and follow up interviews. We felt that this combination of data collection techniques would give us the best information.

After each set of three or four interviews, we revised our site map. At the end of the interview process we had completed a total of five site map iterations.

Interviews and Walkthroughs

The semi-structured interview process began with us sending an information package to the interview subject; this package contained a brief overview of the project, two open ended questions to help the end user contextualize our project, a copy of the site map architecture and a set of instructions on how to read the site map. The interview subject was also asked to make notes in advance of our meeting on any item (or items) they felt were missing, improperly placed, or of general interest on the site map. These notes became talking points for our interview session. This method - also called design walkthrough or paper-and-pencil evaluation - is intended to determine if users are comfortable with the layout and organization of the site.

Combining design walkthroughs with an interview process proved to be a good means of collecting data. The paper-based site map allowed the interview subject to understand the intranet better and relate the information presented on paper to the information he/she uses on a daily basis. Having an outline of questions allowed us to steer the conversation, keeping us on topic.

Occasionally during interviews we make suggestions on potential design enhancements. This worked well in prompting interesting discussions, however, we were aware that this technique has pitfalls, as it could be considered 'leading the witness'. Suggesting design enhancements could also create false expectations for what the eventual functionality of the intranet might be.

Eight interviews were conducted in-person and 20 more over the telephone using this method. These interviews provided us with in-depth information on the end user and details on what pieces of information the intranet could provide them.

Contextual Interviews

The second information gathering process we used was the contextual interview. This process consisted of spending 60 minutes with an interview subject in the context of his or her own work environment. Each participant was asked to proceed through the tasks they would be doing in the course of the normal workday and to focus on tasks directly related to their day-to-day communication. These interviews provided insight into how information flows during daily activity in the work place.

We encountered some problems with following strict contextual interview methodology. For example, it was challenging to keep test subjects focused on their activities, as they were often very interested in talking to us. We didn't want to discourage these conversations entirely as they provided pertinent information. Keeping a flexible approach proved to be the best method to gathering a rich set of data.

Filling in the Gaps

Having completed the semi-structured and contextual interviews, we discovered new questions that needed to be answered. As part of the initial interview procedure, we had asked our interview subjects if we could contact them again to answer any outstanding questions. We returned to five of the interview subjects to get their point of view on an issue, fill in information gaps or shed new light on a topic.

Reviewing the Methodologies

Another issue we faced throughout the data collection process was whether our sample of end users was truly representative. In the stakeholder analysis we found 3 major end user groups. However, within each of these groups there were subcategories. For example, the existence of satellite campuses meant that a support staff end user might have a different need than one at the main campus. Also, teachers, for example, could be categorized as full-time, part-time or contract. We were therefore left with the dilemma of who to interview in order to develop a representative overview of the end users. Given time and budget restrictions, our only option was to ensure that at least one person from each of the sub-groups was interviewed and tested.

Next article: Developing a Prototype and Heuristic Evaluation