AMY CHEN

My Role

What I used: Gap analysis, interview, task-based experience testing, content analysis

I worked with a vendor for recruitment, planned and designed the study, conducted the gap analysis, executed user experience testing sessions, analyzed data, wrote a report and held a meeting for findings and recommendations readout.

Impact

The findings helped the team understand the market positioning of the product and the common use cases. The insights guided the redesign effort and the team prioritized features accordingly.

Plan

I planned the research and divided it into two phases:

• Gap Analysis with 10 Competitors on iPhone and Android phone:
  To understand the trends of features, gestures, and layouts of current designs in the market from the competitors
• Experience Testing with 30 Participants:
  To define common use cases and to learn good user experience and desirable features for a good email app

Timeline

Process

Phase 1: Gap Analysis

I reviewed 10 competitors on iPhone and Android phone from 5 aspects:

1. Customization
2. Email
3. Calendar
4. Contact
5. UI Review

Then I created a table with my observation from each aspect. The table became part of the research database. I compiled findings in the final report.

Phase 2: User Experience Testing

I. Recruitment

While working on the gap analysis, I wrote a screener for the vendor to recruit participants for the experience testing in the meantime.

The main participant criterion was non-Citrix employees who use email app on mobile phones daily for work.

II. Study Design

How did I write the study guide:

1. Built around Gap Analysis
2. Reviewed with the core team
3. Pilot testing
4. Finalized Protocol

With the gap analysis, I had a deeper understanding about our competitors and thus when I was writing the study guide for my experience testing, I had a more vivid idea of what I needed to ask and what kind of tasks I wanted to see the participants perform.

In the study, users would use their email apps on their own phones in the first half of the session, and tested another app that they hadn’t used before on the phone we provided in the second half of the session.

III. Execution

I conducted user experience testing sessions with 20 external non-Citrix end users to study:

• When and how the participants used their work email app.
• Why they chose the app.
• What they liked and disliked about the app.
• What they think about Secure Mail/competitor's app.

IV. Data Analysis

I started from cleaning the raw data by:

1. Going back to the recordings
2. Reading through the notes taken by different note takers

to complete and fix the missing data.

Then I used content coding and thematic analysis to analyze the cleaned data and drafted the report.

Final Report: Findings and Recommendations

The final report was a comprehensive competitive analysis that based on the findings from the gap analysis and the user experience testing. The findings were categorized into three different criteria:

1. Learnings from the competitors
2. Findings for user behaviors and use cases
3. Feedback on the experience with Secure Mail

At the end of the report, the design recommendations were provided. The suggestions specified what parts of the user interface needed fixing and why.

Then I held the meeting of the report readout to the local product design team and the engineering team in India.

Lesson

Everything can go wrong. Just keep cool, stay calm, and be flexible.

I encountered many difficulties while running the sessions such as:

1. Technical problems:
   • Server issues: the emails couldn't go through
   • Connection issues: Vysor/Quicktime/GTM disconnected
2. Participant issues:
   • Lateness
   • Not qualified
   • Dragging to overtime

Without enough buffer time, I was panicked when things went wrong. But gradulally I learned how to get the information that I wanted from users in chaotic circumstances. I kept reminding myself to be relaxed, so the users could be relaxed and talked more.

Reflection

What I will do next time: prioritize the final presentation to maximize the impact of research

• Thinking about the audience first:

At school, we always emphasized on a process. But in the industrial context, a result was more important. Thinking about how to present and how the information could be digested helped me display the findings and recommendations effectively.

• Utilizing the power of the raw data:

It was hard to persuade people to make changes. The Quotes and recordings were the solid pieces of evidence that backed up the research.