4 - Usability Testing

It's well known that what users say and what users do can be a completely different thing. And they can't really tell you what they want, but when they see something and use it - they know what they don't want.


From the data collected during our survey, we were able to more accurately outline who our target demographic is. We wanted to follow this demographic in order to recruit suitable user testing participants.

Frequent Travellers wanted.

Age: 25-45
Gender: Male or Female
Nationality: Any
Technology Skill: Any



In order to get started with our testing, we had to have the materials prepared. We got most of the materials from Steve Krug's "Rocket Surgery Made Easy", especially the instructions for the usability test script.

André created the consent form, looking up to materials Rob provided during the classes, and I found the instructions for the Pre-Test and Post-Test questionnaire in the Usability Testing course I finished on IDF. After 2 sessions, we changed the rate scale from 10 to 7 on our Post-Test following Andrew's advice that might be better to have an odd number so users can find the middle.

We also came up with the scenario containing specific goals and tasks that will help us to conduct successful aimed sessions every time.


Usability Testing Materials
Usability Testing Materials


First, we welcomed and explained to every participant what are the session objectives and their role in the usability testing session. We used open-ended and profiling questions to discover more about participants' travel habits and their experiences with Aer Lingus mobile and desktop app.

Each participant was given two tasks:

  • find and book the cheapest return flight to Paris,
  • find and book the cheapest multi-city flight to Paris and Amsterdam (in some scenarios we added 25 kg of luggage).

Guerilla usability tests

Apart from conducting the normal tests in the office or at home, I also decided to do some tests in the Starbucks close to my place. Two visits resulted in getting feedback from 9 participants and in less than 2 hours.

One more proof that usability testing can be done anywhere, and the feedback is equally important as it is from the lab/office/apartment.

Analysing the data

After getting feedback from my 17 (8+9) and André's 2 participants, we decided that the best way to organise them would be through affinity diagrams. We grouped together all important  findings we got in the common themes, and then we voted what to fix.

Affinity Diagram
Affinity Diagram

This helped us to gain a shared understanding of the users' problems and to summarise the results really fast. Also, thanks to affinity diagramming we had a better insight of the common issues repeated between the participants.

Here you can find all the usability testing videos.

From 8 participants, here's the data I analysed from Pre-Test & Post-Test Questionnaires:


The most important thing when booking a flight
The most important thing when booking a flight is price, which we highlighted in our prototype



The most important feature used in the flight app
The most important feature used in the flight app is search for the flights, which is our no.1 focus in redesigned flight app
Overall rating after using the app - from 1 to 7
Overall rating after using the app - from 1 to 7

Final Product


You can find the prototype and the Figma file separately by clicking on the links.



Conducting Usability Testing. (n.d.). Retrieved January 25, 2019, from https://www.interaction-design.org/courses/conducting-usability-testing

Krug, S. (2010). Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems. Berkeley, CA: New Riders.

Nielsen, J. (2000, March 19). Why You Only Need to Test with 5 Users. Retrieved January 20, 2019, from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

Nielsen, J. (2012, June 4). How Many Test Users in a Usability Study? Retrieved January 20, 2019, from https://www.nngroup.com/articles/how-many-test-users/

Pernice, K. (2018, February 18). Affinity Diagramming: Collaboratively Sort UX Findings & Design Ideas. Retrieved January 20, 2019, from https://www.nngroup.com/articles/affinity-diagram/

Preece, J., Sharp, H., & Rogers, Y. (2017). Interaction design: Beyond human-computer interaction. Chichester (West Sussex, United Kingdom): Wiley.

3 - Designing the Prototype

Paper Prototypes

As we put our ideas and the feedback we got on the table, we started with sketching and creating our first low-fidelity paper prototypes, which we used to identify the problems in the early stage of the design. They are useful because they're fast and simple to produce, but also to modify.

Paper prototype
Paper prototype

The feedback we got from them was crucial before starting with building the interactive prototype. For this purpose, we used Figma because it's collaborative, and it's easy to connect the screens in the prototypes.


Our first interactive prototype was a slightly better version of the existing app with the easier flow, but just after we did user testing on the first part of the app we got the feedback that we could make the experience more emotional by extending the map and showing the destination prices.

We found inspiration on the AirBnb website, where with all the prices shown on the map the experience still stays positive and users don't find it too cluttered or unusable.


We iterated, and created a new prototype with a bigger map, containing prices and planes when the destination is selected, with improved flow regarding the dates and travel extras.



After getting more feedback from our users, we developed a mid-fidelity digital prototype. We also continued with conducting further usability tests and refining the design before moving it into a high-fidelity prototype.




We wanted to change the existing app, and it was really important that we understand Aer Lingus and their current branding. From our research, we found out that their brand values were:

  • welcoming and warm
  • characteristically and traditionally Irish
  • helpful to their passengers
  • approachable, obliging and communicative


Using these core brand values, and all of the things that inspired us - we created a mood board.

Style Guide

In this stage, and after Hillary's suggestion, we decided to include some Material Design in our prototype and make it look more fresh and familiar to users. Using the style guide, we then developed a high-fidelity mockup of the app.



Even though we had the high fidelity prototype, which users really liked, we were struggling with one and important thing -  they didn't notice the planes moving. During one of the classes, Andrew mentioned the term "change blindness" which is basically people's tendency to ignore the changes when they happen far away from their focus.


The only way we could fix this problem we were encountering was using the animation. The design process resulted in a high-fidelity prototype of the app made using Figma, with the preview element animated in Principle to display that functionality. Some of the key flows of the app are shown in the video below.





Please continue reading, follow on to the next step Usability Testing



Budiu, R. (2018, September 23). Change Blindness in UX: Definition. Retrieved January 15, 2019, from https://www.nngroup.com/articles/change-blindness-definition/

Nigel_french. (2014, January 06). Developing a Mood Board. Retrieved January 15, 2019, from https://www.lynda.com/Design-Color-tutorials/Developing-Mood-Board/141129-2.html

Preece, J., Sharp, H., & Rogers, Y. (2017). Interaction design: Beyond human-computer interaction. Chichester (West Sussex, United Kingdom): Wiley.

Wyatt, P. (2018, May 14). 20 pro tips for creating inspirational mood boards. Retrieved January 15, 2019, from https://www.creativebloq.com/graphic-design/mood-boards-812470


4 Featured

4 - Evaluating The Prototype


Redesign and iterations

The last stage of our process was to test, iterate, redesign and over again in the loop. I personally loved it because in this phase you realise how funny is the way in which human brain works.  Just when you think you fixed everything and done the design in the proper way, there is still something more to do.

That's why it's proved one more time - user testing should be done continuously, regularly and we should listen to our users because sometimes things that make perfect sense to us - don't make any sense to them. It's not enough just to design something and develop it. And the product is never perfect. We should always improve and listen to our users.


User Testing Sessions


We did 11 user testing sessions:

  1. Existing PTSB process - 1 user
  2. Our Redesign - 6 users
  3. Iteration 1 - 2 users
  4. Iteration 2 - 2 users


User Testing Videos
Click on the image to view all the User Testing Videos


After testing our redesign with 6 users, it wasn't so easy to meet up and discuss about the changes to be made because of the lack of the free time all 3 of us had. That's why we decided to go with this approach and screenshot every UT session with included comments to improve.



1. Login process

Below I have outlined and contrasted this:

  • the existing PTSB current login process with a minimum of:
    • 3 screens
    • 7 clicks
    • 3 input fields
    • 20 second process
  • with our proposed flow improvements
    • with "Touch ID" option
      • 1 screen
      • 1 click
      • no input fields
      • 2 seconds process
    • with "Enter PAN" option
      • 2 screens
      • 1 click
      • 1 input field
      • 4 seconds process


Existing process



We tried to develop a product that is usable, easy to learn (that people in general don't feel stupid when they use it), effective to use and that it provides to the users an enjoyable user experience. I think that the first step to get there is to identify the weaknesses of the bad products (like for example in the existing PTSB mobile app - long, painful login process) and try to improve it. With iteration, we should be able to get there.


iterations - login
Changes we did in the prototype after each round of user testing


After first iteration, we decided to change the order of the PAN reset options in an order that users would most likely use it. After the second one, we added the shadow under the Quick Balance button so it's more prominent.


Redesigned, final process



2. Transfer funds to a new person

As I mentioned before, this flow never existed in PTSB mobile banking app so we had complete freedom in executing this task in the best possible way. We kept it simple and with as less steps as possible, asking only the essential information needed for transfer - IBAN, name and the amount.


Existing process



We kept it simple in this one, having in mind that one of the main design principles for the website and app design is simplicity. Nielsen proposes that designers go through all of their design elements and remove them one by one. If it works well, keep it that way.

We also thought about our personas when designing this - and came up with the easiest way to get the stuff done - asking but only things that are necessary.


new payee it


After the 1st iteration we removed the "Card Scan" option because none of our users actually used it. They even thought it wasn't secure enough as an option - so after we compared the value that it brings and the actual cost of it, we decided to remove it.

We actually had a second iteration, after this slide, and it was related to removing the saved payees from the flow when the user selects "+Add New" and gets an input field for IBAN and Name.


Redesigned, final process






Four weeks later our first project has come to a close.

I’m extremely pleased with how our project formed and evolved. We started by conducting research using multiple methods before settling on one problem area to work on. If our project was lacking in one area, I believe it would be not iterating enough before settling on one solution.

There are many aspects to this project I think we excelled at as a team, one of them has to be our strong process flow used from the start. We all participated in each step along the way, playing to our individual strengths and sharing our past experiences to advise of next steps to take.

Rob's great background in psychology came in very useful, specially when creating the personas and scenarios, and conducting user testing. Peter's attention to detail turned to be extremely helpful when doing research, comparing analytics and fixing the prototype after iterations. I did a lot of work on sketching, creating the wireframes and user testing - so I can say that each one of us had their own puzzle piece which we managed to put together and finish this project in the best possible way.




  1. Wroblewski, L. (2018, October 3). Luke Wroblewski on LinkedIn: "Creativity is people who care enough to keep thinking about something until they find the simplest way to do it." -Tim... Retrieved October 3, 2018, from https://www.linkedin.com/feed/update/urn:li:activity:6452919668620476416
  2. Krug, Steve, Elisabeth Bayle, Aren Straiger, and Mark Matcho. Dont Make Me Think, Revisited: A Common Sense Approach to Web Usability. San Francisco, CA: New Riders, Peachpit, Pearson Education, 2014.
  3. Preece, J., Sharp, H., & Rogers, Y. (2017). Interaction design: Beyond human-computer interaction. Chichester (West Sussex, United Kingdom): Wiley.