Reading Seminar 2 - Evaluation

I read chapter 10, 11 and some of chapter 12 and 13 in the first edition of the book.

The chapters main focus was on why evaluation is important and when to use the different evaluation paradigms and techniques. So, some of the reasons to evaluate is to fix problems before you ship the product, to concentrate on proved problems that users discover while testing, and when you release the product you can have confidence in that it will work. Understanding which UX-goals and usability the users require is essential to make a product successful. Evaluation makes it possible for developers to discover these goals and needs. As most of the previous design processes, evaluation is also an iterative process and can be done using different paradigms and techniques. The paradigms reminds a lot of the ones used when establishing the user requirements. A question for the group at the seminar: which evaluation paradigm and technique would be best suited for us?

To answer this we can use the DECIDE framwork. The framwork provides a checklist to help the evaluation process:

1. Determine the overall goals that the evaluation addresses.

              What are the users needs?

              What are we evaluating? Design? Concept? Usability?

2. Explore the specific questions to be answered

              Depending on the goals, set relevant questions questions.

3. Choose the Evaluation Paradigm and Techniques to be answered the questions in step 2

               Choose a paradigm and technique. The paradigm determines which techniques are appropriate.

4. Identify the practical issues that must be addressed.

               Identify potential issues BEFORE starting.

               For example, which users should test the prototype? Are they representing the target group?

5. Decide how to deal with ethical issues 

              (I don't think this is critical at our stage)

6. Evaluate and present the data.

              Try to prevent bias.

               Is the data enough to generalize the findings?

Some more practical issues to address: 

How to observe users in their natural environment without disturbing them

Having appropriate equipment available

Dealing with our short schedules (I feel this is something extra important to discuss)

Selecting techniques that are possible for us to do.

I'm guessing we should start evaluating at early stages of our prototype, but should we start at our lo-fi prototype sketch or sould we make a mockup, such as a wizard of oz prototype first?

Should we focus purely on the conceptual design evaluation at first or do it parallel with heuristic evaluations?