Intranet Journal
The online resource for intranet professionals

Designing an Intranet User Survey

12/13/2004

It usually happens anytime from 6 to 8 o'clock; you're cooking dinner or are already in the middle of eating and the telephone will ring. You know there's going to be trouble the moment you hear that silence in between your first Hello and your second, louder HELLO! The person on the other end of the line will finally kick in and display that distinct impression of reading off a script, "Hello Mr. Chin, my name is Pam and we're doing market research on aluminum foil, plastic wraps, and other food storage products. Do you have a few moments to answer some questions?"

What follows is one of two responses on my part: If I'm not too busy, it will be a polite "I'm sorry but I'm busy at the moment." But if I've had a particularly bad day or if something's smoldering on my stove, I may be a little less accommodating and make some loud, shrieking monkey noises to drive them away — after all, it would be rude to just hang up.

This is probably the image that comes to mind when we first hear the word "survey" — the efforts of some large market research firm gathering raw data on our comings and goings to be used for who knows what. But it's high time we wash that bad taste out of our mouths because a survey is an excellent tool to keep in touch with the changing needs of the user community, and to help intranet owners gather requirements for improving and advancing an intranet.

The Importance of Asking Questions

In the article "Your Next Move: Planning Intranet Upgrades" I discussed the importance of not allowing your intranet to become stagnant, to ensure that the system evolves in parallel with users' changing needs. Upgrades — whether bug patches or major system revisions that add new functionality — can represent a large and substantial change in the design and functionality of an intranet. And as such, the user community must play a large role in defining the future direction of their intranet.

But it's a misguided presumption for IT to think that they know what users want without consulting them. Intranets, moreso than many other IT implementations, are user driven. Although IT develops and maintains the entire technological backbone, the meat of an intranet — the content — is managed by those who use it.

The fact of the matter is IT shouldn't guess what their users what, users should be telling IT what they need. Unfortunately, for various reasons — an "us versus them" mentality, an IT staff unreceptive to user needs, a feeling from users that IT isn't listening to them, users not expressing their needs properly — communication between the two groups may fail. But user feedback and response is critical in defining the requirements of all future intranet versions.

My article "Survey Says! Measuring Intranet User Response" explored the differences between passive and active methods of measuring user response to an intranet. While passive feedback methods such as analyzing Web server usage logs can give you a good sense of the wheres and whens — raw, unprocessed statistics about your user community's surfing habits — they don't tell you anything about the whys. Seeing that a particular page was hit 100 times in a single day tells us very little about users' satisfaction with the content they're reading on that page and even less about their satisfaction with the system in general. They may have hit a page without reading it, or perhaps they read it and thought that the information was about as useful as a wet napkin.

Finding out the whys will help you advance your intranet beyond IT's assumptions of what users should want and base upgrades on what they actually do want and need. But in order to do this properly you need to collect user feedback beyond a scattering of casual e-mails and verbal suggestions; you need an easy way to quantify the results of this feedback — and the best way to do this is through an online user survey.

Making the Questions Count

Many surveys and questionnaires I see, IT-related or otherwise, have one common failing: they're just too long. Unless questionnaires have direct relevance to the person reading it, such as medical or financial forms, they will be unlikely to stick around to answer a lengthy set of questions.

Some intranet owners get carried away when developing their user surveys, writing down every minor question that comes to mind. But a single, concise question can be just as effective as three vague questions. And you're much more likely to get users to respond to a survey if it doesn't take too long for them to complete. There's nothing worse than forcing them to sit through and decipher a long list of verbose questions.

Whatever you hope to accomplish with your survey, it needs to be clear and focused, centering around three key intranet areas:

1. Content quality and relevance
2. Site navigation and usability
3. Design and site cosmetics

Sample Intranet User Survey Questions

The types of question to ask will vary depending on the intranet's purpose and your particular goals with the survey. Below are some examples of questions (or in this case, statements) that can be used to cover the three intranet areas I mentioned above.

(Note: In the following examples I used statements rather than questions to give them a more casual feel. They're based on a 1-to-5 scale: 1: Strongly disagree; 2: Somewhat disagree; 3: Neutral/doesn't apply; 4: Somewhat agree; 5: Strongly agree.)

I access the intranet very often throughout the day
I access the intranet remotely at home and/or during business trips.
I usually find what I'm looking for when I access the intranet.
I access the Internet less frequently because the same information is already available on the intranet.
I can find what I'm looking for very easily by:
- navigating the site's menus.
- using the site's search engine.
The intranet's content has direct relevance to my day-to-day activities.
I use the intranet because it contains information not available elsewhere.
I can find current and up-to-date information without going to other sources.
I'm satisfied with the overall quality of the site's design.
I'm satisfied with my overall experience with the intranet.
The intranet's strongest points are:
The intranet's weakest points are:
Some things I would like to see added to the intranet are:
Additional comments:

If you want to get even more detail out of your survey, you can have your users rate the quality of each core intranet section.

Storing and Processing the Results

Getting users to participate in the survey is only half the job; what happens after they hit the form's Submit button? Although your IT staff will most likely be responsible for implementing the online survey, they shouldn't be the ones to have to process the data.

The easiest way to collect survey results is with the use of a mailto script; it requires the least amount of programming. Blackbox mailto scripts can be downloaded from the Internet and tweaked to work with your own survey form. However, the time you or your IT staff save in initial programming will be greatly offset by the amount of manual labor required to process the raw data returned to you. This is because mailto scripts do little more than e-mail the results of individual form submissions to a list of pre-specified e-mail address. It will be difficult to measure the overall results of the survey when you have all the responses spread out among numerous e-mail messages.

A more efficient solution is to have every user's online survey submission automatically stored into a database. A technique I've had success with in the past is to use ASP scripts to process user submissions and store the responses directly into a MS-Access database. Once the survey period is over, the MS-Access database file can simply be sent to the results processors.

The advantage here is that many non-technical personnel are already familiar with the basics of MS-Access (unlike MySQL or Oracle, which requires a bit more SQL knowledge). The database can then be queried to generate reports or graphical representations — pie charts, bar graphs, line graphs — summarizing the results of the survey without much IT involvement. This will provide you with an at-a-glance overview of the results and enable you to find usage and response patterns.

Tips on Running a Successful User Survey

• There are no die-hard rules to running a successful survey but there are a lot of things you can do to increase the likelihood of a high response rate:

• Don't make the survey too long and wordy or users will be less like to respond. Keep the questions simple and to-the-point.

• Depending on the length of the survey, break it up into multiple pages so that no more than five to 10 questions appear on a single page.

• Show a progress or status meter such as "Page X of Y" that informs the user where they are in the survey and how much more they have to do.

• Make the survey anonymous and ensure your users that all submissions will be kept confidential so that users will be more likely to answer honestly.

• An optional department field can be used as an identifier in an anonymous survey. This will allow you to maintain a good level of anonymity while still providing you with a way to graph feedback by user demographics.

• Place a link to the survey in a high-traffic area such as the home page, the "Latest News" section, FAQ pages, and contacts page.

• Make questions multiple choice — using either a scale such as 1-to-5 or worded multiple choice responses. Try to use the same scale throughout the survey unless unavoidable. In other words, don't jump from using a 1-to-5 scale to Yes/No to Relevant/Neutral/Not Relevant multiple choices.

• Provide a free-form text box at the end of the survey to allow users the opportunity to express additional feedback not covered in the survey.

• Writing the survey questions in the first person can give it a more casual and inviting atmosphere.

• Make sure that you run the survey — however long you wish to run it — when the company is at maximum staff, and not during peak summer vacation months or in December and January when most people are off on holidays or recovering from them. This will give you a larger sample size.

• Always show the results of the survey so that users will know that their input is important, that their time was well-spent, that something actually came out of it, and to see what the overall feedback compares to their own views.

• An incentive can be given for the completion of the survey (if the user provides his/her name).

• If you can, try to avoid running a paper-based survey; they require too much manual processing.

Final Thoughts

Running a survey is a great way to allow your user community to express their views and opinions on the current state of their intranet and gives them an opportunity to play a part in the future direction of the system. But it's a two-way street; if you ask users for feedback, they should be able to see the results of this feedback.

They have to know that the time they spent filling out your survey is being viewed by someone on the other end of the Submit button. But if nothing ever comes out of a user survey campaign, don't be surprised if your users hang up on you the next time you need to ask them their opinion.