course taught in classroom ALL attend 11/26

This first testing experience often strikes fear in the hearts of even the most seasoned web producers. What if after all your hard work, they don't like it? Not to worry! The purpose of this class is to provide you with a safe place to make mistakes and recover. Besides, the user testing most often shows what a great job you have done. That doesn't mean, though, that your product is perfect at this point. What seems logical and clear to you, may not seem so to those who are not as close to or familiar with your web site as you are.

Here's what you want to find out in the user test-run: What "problems" do you need to fix in order to have a web site that truly does for the client what you designed and developed it to do? A problem could be anything from misspelled words and unclear navigation to unanticipated user reactions or even the web site loading so slowly that they user quits and goes surfing at Yahoo. It also could be something as basic as the participants hating the whole concept (not likely, but possible).

I wanted to discuss all testing issues in one lesson, we will be implementing them over the next several weeks. The sequence I have found most effective is to test it with

1. conduct classmates test (CD
   report due - 12/10
   
   
2. conduct client test (DES)
   report due - 12/10
   
   
3. conduct typical users test (PM)
   report due - 12/10
   
   
4. conduct equipment test (PROG)
   report due - 12/10

2. The team will need to develop a "testing instrument" which is a series of questions which can be used to test the site. An example of a questionnaire has been provided. Feel free to modify or create you own original form - sample questionnaire

3. Next send out the questions to the class mailing list. product@floyd.santarosa.edu
Please indicate in the email questionnaire,which 3 groups will test your site and who the responses are to be sent to. The responses must to be sent to one of the CDs on your team. the evaluations should not be sent to the class mailing list.

4. You also need to set a deadline for when the evaluations need to be submitted. I recommend you end all testing 3 days beforeDec. 6thso you have a few days to prepare the report

5. The team member who is conducting the classmate testing needs to decide if everyone who was supposed to fill out the form, has done so. The team assignment page will let you know who is on what team so you can make sure everyone did return the email questionnaire. Five days before the evaluation is due send any missing students a reminder and cc me. If a classmate misses the deadline for submitting an evaluation the CD needs to send me an email reporting this problem Students who submit their evaluation on time with helpful feedback will receive 20 points per project evaluated for a total of 60 possible points.

6. Create a Web page which displays the testing data. This page can be stored within the CDs Web account, but must be linked to - from the Web Project site.

Sample Classmate Testing Reports from former students are listed below:
Harmony Garden Project by Roni Savage
Routes for Youth by Marcy McNeil

Testing with the Client
The designer needs to arrange for the client to test the site. Please note that for all SRJC teams Susan Bagby Matthews is also considered the client.

1. This can be done in a face to face meeting or by email. You may choose to use the same questions you used with your classmates, or modify the questionnaire slightly.

2. In addition to the test questions you want to make sure there are no typos or inaccurate information. I recommend you print out the Web site as it exists and ask the client to make corrections and notes on the printed pages. I often miss things on the screen that stand out in print.

3. Once the client has submitted the testing results the designer prepares a Client Testing Web page which is linked to from the Project site. Sample Client Testing Reports are shown below:Routes for Youth Client Test by Jason Ellis
Health Care for All Californians by Marci McNeil

I also need the designers who are conducting this test to perform a printing test. Please print in color the opening screen and one second level page from your site. Put these print outs in my box or give them to me in class so I can make them into a poster which will be displayed on campus.

Testing with Typical Users

1. Recruit (5-10)typical users who are similar to your intended user. Refer back to the "target market" description. The important thing here is that the test user should, as much as possible, be like your "real" user so you get the most accurate information.

2. Have the test-run participants use the web site and answer questions about the site. Again you can use the same questions used in the classmate testing when surveying the typical users.You may conduct the testing online by sending an email questionnaire and then having the tester send back the answers, or you may meet with them in person and administer a printed questionnaire.

3. If the test is done in person, observe the test-run participants as they use your web site. The important thing here is to try not to interfere. In order to get accurate information, you must not jump in to "help" as soon as you spot an apparent problem. Of course, if participants really get stuck, you do want to work with them so they can continue to test your product.

4. Assemble the typical user testing results and create a Typical User Testing Report web page. Load it on the Web and then link to it from the Project site. An example is shown here: Example of Typical User Test

Testing under various Hardware and Software configurations
The conditions your site should be tested under includes:

1. browsers (Netscape, IE, Modzilla, and Safari (Mac only) ) current version and one version back
2. platforms (Windows XP and Mac OS X)
3. screen sizes (15"- 21")
4. net connection (slow modem, DSL, cable modem, T1)
5. use with adaptive devices found in Hi Tech Center (see below)

I need you to view your web site at the High-Tech Center which contains computers which have adaptive devices for students with disabilities. Call Richard at 527-4890 and set up an appointment for 15-30 minutes sometime within the next 2 weeks. You will be allowed to view your web site using the equipment and determine how the site looks to the disabled

Take note of any problems including:

1. color distortion (is it major or can you live with it?)
2. graphics or text that does not fit on the screen
3. dramatic changes in layout (look for cracking tables)
4. site cannot be navigated at the high tech center
5. length of time the site takes to load on a slow modem

Here are samples of well done equipment tests:
Steve Fontaine - Sonoma County Health Department - Equipment Testing
Dennie Ismalov - Sonoma Literacy Coalitiion - Equipment Test
James Foreman - CCAT Equipment Test

For the next few weeks everyone is busy testing each others sites and presenting the site to the clients. Hopefully the majority of the work is done, unfortunately the details will get you every time.

You may be wondering what needs to be changed as a result of this testing. Here are the class guidelines.

1. Fix cracked tables, broken links, images that do not load,images that do not have alt test, typos, or content mistakes.

2. Do not make major revisions to the site content because the client now has new ideas for what to say. We use the content which was provided by the client before production began.

3. If recommendations are made regarding the site design the team will need to discuss what they want to do. if most folks find the site easy to read and use and one person thinks the font is to small then I would not change font size. If, however several testers say the font is too small and you used CSS I would recommend increasing font size to accomodate the testing results.

Content for these teaching materials were developed by Linda Hemenway. If you have questions or comments on the material I can be contacted through
lhemenway@santarosa.edu
(707) 527-4855

Web site design by Jessica Hirsch - jessicahirsch@email.msn.com

Funding for these notes provided by Project FIVE and Project PICTS supervised by Doug Garrison - Dean of the Petaluma Center.