An online service I use daily recently invited me to alpha test their updated mobile app. It was a fun an interesting experience I thought I’d share to the best of my ability, without breaking the NDA. The process was incentivized by offering Amazon gift cards after completing different milestones.
Once accepted into the study, I was scheduled to attend a Google Hangout interview with several members of the company’s UX team. The purpose of this interview was to lay the groundwork and set expectations for my contributions over the 12 day research phase.
When the hangout was complete, I was invited to a Google+ “Diary” group. The group included a Product Designer—my research buddy and point of contact for the duration of the study—and several other employees; User Research Lead, UX Researcher, several UX Designers, Program Manager, Product Manager intern, and VP Product.
The group was pre-populated with a welcome post from the User Research Lead, reiterating the purpose and expectations. The post also contained a link to a Google document with instructions for downloading the prototype app with Crashlytics, and recording my thoughts with Reflector, which mirrors my mobile device to my desktop and allows me to record.
I was encouraged to record my first session with the app and “think aloud” so after my initial login I fired up Reflector and recorded a 10 minute session, using the app as I normally would, and talking four times as much as I would ever consider. There are some smaller features added in alpha that are missing from the production app I am familiar with—these points were either glossed over or completely missed in the initial recording.
After my first impressions were reviewed, my research buddy asked more targeted questions through the Diary group and encouraged I continue to use the app and post thoughts to the Diary as they occur to me. He emphasized several times that the purpose of this study is to gather feedback as opposed to finding and reporting bugs, and to refrain from doing so unless the issues severely impacted the experience.
A few days later I received an Amazon gift card.
The Product Designer posted an update to the Diary page asking for feedback on a specific feature of the app, which I assumed to be the purpose of this project. He asked to share my general experience with this feature in addition to asking more specific questions i.e. what I like about it, what I don’t like about it, why I use it, etc. Text updates or Reflector session would be accepted for this phase, but I was encouraged to use Reflector.
Following the review of my latest recorded session, the Product Designer asked several targeted questions, presumably on important talking points that I missed during my session. I asked some functionality questions in my recording, and instead of directly answering these questions, he encouraged me to continue exploring a specific area of the app that I had only briefly visited, and asked more questions to help guide my exploration.
A day later I received another Amazon gift card.
The next update was requested after a few days, with the focus being on using a competitor’s app. The target was a specific feature of that app, which was the equivalent of the feature that I was asked to explore in my previous session. Again, recording the session was encouraged. More targeted questions followed after that session was reviewed; what, where, when, why, how.
The diary ended with a Hangout where the Product Designer went over everything I noted in both my text and video updates. Instead of stating my points for confirmation, he simply asked every question that I answered. This felt like a lot of reiteration, but it kept me talking through my experienced and allowed him to interject for clarification where necessary before we severed ties.
At the end of the recap interview I was asked to delete the beta app while connected to the video call. My research buddy thanked me for my time and help, and assured that a final Amazon gift card would follow shortly. The process was more involved than I had initially expected, but it was fulfilling to contribute to a product I care for. The Amazon credit didn’t hurt either.