It’s well known that what users say and what users do can be a completely different thing. And they can’t really tell you what they want, but when they see something and use it – they know what they don’t want.

Participants

From the data collected during our survey, we were able to more accurately outline who our target demographic is. We wanted to follow this demographic in order to recruit suitable user testing participants.

Frequent Travellers wanted.

Age: 25-45
Gender: Male or Female
Nationality: Any
Technology Skill: Any

 

Materials

In order to get started with our testing, we had to have the materials prepared. We got most of the materials from Steve Krug’s “Rocket Surgery Made Easy”, especially the instructions for the usability test script.

André created the consent form, looking up to materials Rob provided during the classes, and I found the instructions for the Pre-Test and Post-Test questionnaire in the Usability Testing course I finished on IDF. After 2 sessions, we changed the rate scale from 10 to 7 on our Post-Test following Andrew’s advice that might be better to have an odd number so users can find the middle.

We also came up with the scenario containing specific goals and tasks that will help us to conduct successful aimed sessions every time.

 

Usability Testing Materials
Usability Testing Materials

Process

First, we welcomed and explained to every participant what are the session objectives and their role in the usability testing session. We used open-ended and profiling questions to discover more about participants’ travel habits and their experiences with Aer Lingus mobile and desktop app.

Each participant was given two tasks:

  • find and book the cheapest return flight to Paris,
  • find and book the cheapest multi-city flight to Paris and Amsterdam (in some scenarios we added 25 kg of luggage).

Guerilla usability tests

Apart from conducting the normal tests in the office or at home, I also decided to do some tests in the Starbucks close to my place. Two visits resulted in getting feedback from 9 participants and in less than 2 hours.

One more proof that usability testing can be done anywhere, and the feedback is equally important as it is from the lab/office/apartment.

Analysing the data

After getting feedback from my 17 (8+9) and André’s 2 participants, we decided that the best way to organise them would be through affinity diagrams. We grouped together all important  findings we got in the common themes, and then we voted what to fix.

Affinity Diagram
Affinity Diagram

This helped us to gain a shared understanding of the users’ problems and to summarise the results really fast. Also, thanks to affinity diagramming we had a better insight of the common issues repeated between the participants.

Here you can find all the usability testing videos.

From 8 participants, here’s the data I analysed from Pre-Test & Post-Test Questionnaires:

 

The most important thing when booking a flight
The most important thing when booking a flight is price, which we highlighted in our prototype

 

 

The most important feature used in the flight app
The most important feature used in the flight app is search for the flights, which is our no.1 focus in redesigned flight app
Overall rating after using the app - from 1 to 7
Overall rating after using the app – from 1 to 7

Final Product

You can find the prototype and the Figma file separately by clicking on the links.

 

References

Conducting Usability Testing. (n.d.). Retrieved January 25, 2019, from https://www.interaction-design.org/courses/conducting-usability-testing

Krug, S. (2010). Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems. Berkeley, CA: New Riders.

Nielsen, J. (2000, March 19). Why You Only Need to Test with 5 Users. Retrieved January 20, 2019, from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

Nielsen, J. (2012, June 4). How Many Test Users in a Usability Study? Retrieved January 20, 2019, from https://www.nngroup.com/articles/how-many-test-users/

Pernice, K. (2018, February 18). Affinity Diagramming: Collaboratively Sort UX Findings & Design Ideas. Retrieved January 20, 2019, from https://www.nngroup.com/articles/affinity-diagram/

Preece, J., Sharp, H., & Rogers, Y. (2017). Interaction design: Beyond human-computer interaction. Chichester (West Sussex, United Kingdom): Wiley.

3 - Designing the Prototype
5 - Conclusion