Designing with Computer Vision
Solving everyday user problems with context-aware mobile solutions
Client: Qualcomm Computer vision team
Role: User research, concept design, design strategy
Team: Anne Konertz, CHad wilkie
Time: 4 months
If a digital assistant could see, what could it do for you? Qualcomm UX was approached by the computer vision engineering team to discover meaningful human-centered applications for computer vision enabled digital assistants.
We started by doing a competitive analysis of the functionalities of different digital assistants and identifying areas for improvement. Next, we framed our research challenge by looking at both mobile phone and home camera usage. After preliminary research, recruited users from two main user groups:
the “super socials”, power users of camera enabled social media applications (e.g. Snapchat, Instagram) between the ages of 16-22
“smart home enthusiasts” people with multiple smart cameras and other digitally enhanced devices in their homes
Through in-depth user interviews and home field visits, we discovered key trends in both groups around patterns of behavior, preferences, unmet needs, and privacy concerns. For example, we discovered that while most users initially bought smart cameras for security reasons, they formed a role in families’ social and emotional relationships as well. One man who traveled frequently used his Nest Cam to say goodnight to his small sons when on business trips.
Facilitating Brainstorming Sessions
After analyzing and synthesizing the raw data from our interviews and field observations, we organized and facilitated a group brainstorm with 20-30 engineers from computer vision, audio, and machine learning teams to draw on their technical expertise, build understanding around the users problems and desires, and generate a large number of ideas.
In the first half of the session, we presented our user research. Next, we split into four teams of five and brainstormed around a poster of a persona we had just introduced. To help prompt and inspire new ideas that were grounded in the research, we created “scenario cards”. These cards combined user scenarios for each persona and a photo of the event or activity. We left time at the end of the session for a physical rapid prototyping session in which the participants built and bodystormed prototypes of their best solutions in use.
From Synthesis to Storytelling
After the brainstorm, we sifted through the ideas and iterated on them with several UX only brainstorms. We came up with a list the best twenty use cases across user groups and refined them into design concepts through storyboarding the experience.
Refining Our Concepts through Video Prototyping
We worked with the engineering team to identify the most promising ideas and then we refined those storyboards further into video experience prototypes. The videos were incredibly helpful in working through areas where QC computer vision team can innovate.
Our research led to:
A series of illustrated human-centered use cases and video prototypes for both mobile phones and smart home cameras.
The discovery of new edge computing technology that Qualcomm could develop to enhance the functionality and user experiences of computer vision enabled digital assistants.