AMY CHEN

My Role

What I used: Interview, task-based usability testing, content coding

I worked with a vendor for recruitment, designed the study, moderated user testing sessions, analyzed data, wrote a report and held a meeting for findings and recommendations readout.

Impact

1. The team decided not to implement some new experimental design concepts according to the user feedback since they created too much confusion.
2. Before a new feature could be implemented, the team would stay with some parts of the old design for the current features provided in order to avoid misunderstandings.
3. A further study regarding a specific tab in the app would be a part of the next step. From there, a new feature would be officially brought into the future discussion.

Goal

The research goal was to validate new navigation design concepts to see if they met the user needs. The team planned to launch the new user interface in Q1, and thus the research schedule became very tight.

Process

I. Challenge: Tight Schedule

The team tried to validate the design before moving forward to the development stage, and the deadline was in a week. The kick-off meeting needed to be held immediately. The research project manager and the core team helped me squeeze it into my research schedule.

II. Concern: Recruitment

Recruiting 6 participants in 5 days including the weekend was very challenging. Therefore, I created a screener and contacted a vendor immediately once we accepted the research request.

The main criteria for the participants for this study were:

• Checks work email and work calendar on a mobile email app
• Does not work for Citrix or other competitor companies
• Be able to participate in a 30-minute in-person session in Santa Clara

III. Design the Study

I reviewed the wireframes first and worked closely with designers to understand what they wanted to test. Knowing in which part of the design they were lack of confidence and what assumptions they have currently was important for me to create a study guide. By understanding their design thinking and logic, it was easier and faster for me to draft the study protocol.

The main tasks we came up with:

• Compose
• Calendar
• Contact
• Account
• Attachment
• Setting

IV. Execution: 6 Sessions in One Day

The time for preparation or debriefing between sessions was very short. To utilize the time, we prioritized the fast discussion for protocol modification.

In this agile process, most note takers did not have enough time to clean their notes before submitting. A well-designed note-taking form assisted my session observers to take informative notes.

V. Data Analysis

• Read through the Google sheet exported from the note taking form
• Tracked how many participants achieved success at a task at their first attempt, how many of them struggled, and how many participants failed the tasks in the end
• Used content analysis to find common themes in the feedback
• Drafted design recommendation

VI. Final Report

Design recommendations were given based on findings in each slide of the report in PowerPoint so the designers could quickly understand why suggestions were made. The insights I uncovered were shared to adjust current new design before moving on to the development stage.

Reflection

With my internship experience, I was familiar with the team and the product. It helped me start quickly and move fast throughout the process. Except the recruitment, the study was conducted and finished in three days.

I noticed that the recruitment was a significant factor whether a study could be smooth or not. Thus, I worked closely with the vendor to ensure the smooth process for the recruitment.

I also learned from the last experience that the time estimation needed to include enough buffer time, so I would be able to deal with any accidents. I found that was also a reason why I could conduct back-to-back sessions without issues this time.