Sharon Kaziunas
Screen Shot 2018-06-05 at 6.27.19 PM.png

Designing with Computer Vision

 
 

Designing with Computer Vision

Solving everyday user problems with context-aware mobile solutions

 

 
computer_vision_header.png

Client: Qualcomm Computer vision team

Role: User research, concept design, design strategy 

Team: Anne Konertz, CHad wilkie

Time: 4 months

 
 

The Challenge

If a digital assistant could see, what could it do for you? Qualcomm UX was approached by the computer vision engineering team to discover meaningful human-centered applications for computer vision enabled digital assistants.

 
 
 

User Research

We started by doing a competitive analysis of the functionalities of different digital assistants and identifying areas for improvement. Next, we framed our research challenge by looking at both mobile phone and home camera usage. After preliminary research, recruited users from two main user groups:

  • the “super socials”, power users of camera enabled social media applications (e.g. Snapchat, Instagram) between the ages of 16-22

  • “smart home enthusiasts” people with multiple smart cameras and other digitally enhanced devices in their homes

Through in-depth user interviews and home field visits, we discovered key trends in both groups around patterns of behavior, preferences, unmet needs, and privacy concerns. For example, we discovered that while most users initially bought smart cameras for security reasons, they formed a role in families’ social and emotional relationships as well. One man who traveled frequently used his Nest Cam to say goodnight to his small sons when on business trips.

 
 
  Images from our fieldwork. 

Images from our fieldwork. 

 
 

 
 
“Sharon did an absolutely amazing job. Presenting qualitative user findings can be tricky. Sharon set them in context to make them relevant for the team and presented them in a very professional and engaging way. It was fantastic!”
— Anne Konertz, UX Lead
 
 

 
 

Facilitating Brainstorming Sessions

After analyzing and synthesizing the raw data from our interviews and field observations, we organized and facilitated a group brainstorm with 20-30 engineers from computer vision, audio, and machine learning teams to draw on their technical expertise, build understanding around the users problems and desires, and generate a large number of ideas.

In the first half of the session, we presented our user research. Next, we split into four teams of five and brainstormed around a poster of a persona we had just introduced. To help prompt and inspire new ideas that were grounded in the research, we created “scenario cards”. These cards combined user scenarios for each persona and a photo of the event or activity. We left time at the end of the session for a physical rapid prototyping session in which the participants built and bodystormed prototypes of their best solutions in use.

 
 
cv_scenario_cards.jpg
 Printing out posters of research personas and scenario cards served as a tangible reminder to focus on user needs throughout the brainstorming session.

Printing out posters of research personas and scenario cards served as a tangible reminder to focus on user needs throughout the brainstorming session.

 
 

From Synthesis to Storytelling

After the brainstorm, we sifted through the ideas and iterated on them with several UX only brainstorms. We came up with a list the best twenty use cases across user groups and refined them into design concepts through storyboarding the experience.

 
 
 A photo of me sorting, ranking, and expanding upon concepts generated by our group brainstorm.

A photo of me sorting, ranking, and expanding upon concepts generated by our group brainstorm.

 Refining our initial findings into experience concepts through sketching and storyboarding.

Refining our initial findings into experience concepts through sketching and storyboarding.

 We iterated on our storyboards and presented them in a deck highlighting how Qualcomm technologies could enhance those experiences.

We iterated on our storyboards and presented them in a deck highlighting how Qualcomm technologies could enhance those experiences.

 
 

Refining Our Concepts through Video Prototyping

We worked with the engineering team to identify the most promising ideas and then we refined those storyboards further into video experience prototypes. The videos were incredibly helpful in working through areas where QC computer vision team can innovate.

 
 
cv_video_prototyping.png
 
 
 The photos above are stills from two separate video prototypes illustrating our best computer vision use cases for mobile phones and home smart cameras. 

The photos above are stills from two separate video prototypes illustrating our best computer vision use cases for mobile phones and home smart cameras. 

 
 

Impact

Our research led to:

  • A series of illustrated human-centered use cases and video prototypes for both mobile phones and smart home cameras.

  • The discovery of new edge computing technology that Qualcomm could develop to enhance the functionality and user experiences of computer vision enabled digital assistants.

Learn more about Qualcomm's computer vision technologies

 
 

Related work