PHASE 2: USABILITY TESTING

While the work on the next major release progressed from an architectural standpoint, the lengthy UA/UI discussions eventually faded out. It became clear to the committee that it was too early to agree on UI designs. The usability team grew to include two student interns and then began conducting testing. Initial usability testing focused on the currently released versions of cc:Mail for LAN and Mobile across platforms.

User-Centered E-Mail Tasks

Based on the first few rounds of user analysis and task analysis, the usability team created and refined the following "Top 10 list" of the most basic e-mail tasks:

  1. Read. (Differentiate new mail from old; distinguish high priority messages from normal or low; identify the sender, other recipients, date and time sent, etc.)

  2. Write & Send. (Create a message; address it; write it; set priority; optionally save a copy; send it.)

  3. Reply. (Reply to sender, some, or all recipients; keep none, some, or all file attachments.)

  4. Forward. (Recognize a forwarded message; view forwarding information; forward one or more messages with or without forwarding information.)

  5. Delete. (Delete one or more messages; put a message in the Trash or permanently delete it; know if Trash is enabled or how to enable it; set when to empty the Trash; retrieve a deleted message.)

  6. Print. (Print one or more messages and any or all attachments, with or without the header; print a list of messages, e.g., just author, subject, and date.)

  7. Attachments. (Read an attached file, view or launch it; attach one or more files; save attached files to disk.)

  8. Folders. (Create a folder; store one or more messages in a folder; access messages in folders.)

  9. Mailing Lists. (Create a mailing list; address a message to a mailing list.)

  10. Search. (Find a message given the author, subject, date, word in the body of the message, etc.)

This list does not include implicit navigational tasks, such as going back to an open message, getting back to the Inbox, reading next and previous messages, or closing a window. It also does not include all the possible advanced tasks, such as finding copies of messages already sent, saving messages in progress, automating message storage by creating and running rules, etc. This list stood the test of time. The one thing everyone always agreed to is that these are the most commonly performed e-mail tasks and so should be the primary focus of UI design and usability efforts. This was an essential first step that helped the usability team eventually get buy-in on UI changes from the rest of the product team.

Prototype Testing

In addition to testing released versions of cc:Mail, paper, Visual Basic, and Visual C++ prototypes were also tested. These prototypes were primarily designed by development, with contributions from the usability team and the UI designer. Prototype testing was valuable because it provided feedback on non-trivial UI variations without taking as much time as full implementation.

Competitive Testing

The usability team also tested competitive products, such as Microsoft Mail, according to the same user population and the basic tasks. Developers often justified some of the known usability problems in cc:Mail for Windows by placing the blame on the environment, e.g., "That's just the way Microsoft Windows works." Testing competitive products afforded an opportunity to gather usability feedback on alternative design solutions without requiring any additional prototyping or implementation. Competitive testing revealed design ideas of how to do things better as well as what not to do. This testing proved to be a highly valuable source of motivation for change, particularly when a common feature was easier to use in another Windows product.

Beginning of Document | Top of This Page (Phase 2)
Phase 1 | Phase 3 | Phase 4
Results and Examples | Conclusions