Prototyping a VR app for Merck

a case study by Morgane Santos

virtual reality, prototyping, user experience


Merck is a multi-billion dollar pharmaceutical company that markets to doctors and hospitals. In an effort to improve their sales pitches, Merck looked to virtual reality. Instead of describing how a drug worked, what if doctors could “see” it in action?

I worked with 2 developers for a month to prototype a Google Cardboard app that salespeople could use to show doctors how various drugs worked. I created UI elements, prototyped interactions in VR, and designed the mobile screens for both iPhone and Android.

Video on mobile, non-VR view

Video on mobile, non-VR view

Getting started

We began with a design sprint with several representatives from Merck. We learned how Merck's salespeople pitch new products, what hardware and financial constraints there would be for the VR headsets, and what doctors themselves need to know to make a decision.

Together we decided to make an app for Google Cardboard, so salespeople could use their own smartphones without needing expensive headsets. We also decided to focus on the experience from transitioning between the mobile view into VR, when salespeople would give the phones over to the doctors. This was key in creating a seamless experience.

An early prototype of VR video UI

An early prototype of VR video UI

Researching VR

Because VR is such a new field, there aren’t many well-established patterns. I researched VR best practices and wrote a handbook for both thoughtbot and Merck. I wrote notes for myself as well: if whoever is reading this has an interest in designing for VR, you may find this helpful.

Documenting best practices for the VR team

Documenting best practices for the VR team

To make the VR experience comfortable and easy to navigate, I introduced movement slowly and locked the UI elements to a consistent place.

Since this app would run on smartphones, there would be no hand controls. To get around this, I decided to use delayed raycasting. Users would look at a UI element and a timer would appear. If they looked at the UI element for long enough, it would trigger that element. It was hard to accidentally trigger anything, but it also respected people’s time and intention. User testing helped us find the right speed for these timers.

A later UI prototype

A later UI prototype

Mobile to VR and back again

This app had to work on both iPhone and Android devices. I iterated on a simple flow where salespeople could find the videos they need, download them in advance (or on the spot), and pass the phone off to doctors in a Cardboard headset. The flow would be the same no matter what device it was on, to keep it easy to implement and to feel seamless between devices.

When testing this flow, I focused on speed and ease of use. Salespeople often have no more than 30 seconds with a doctor, and this app couldn’t slow them down at all. We focused on having few categories with few videos, and large images that could be instantly recognizable. To start the experience, all the salesperson would have to do is hit the play button.

iPhone and Android designs

iPhone (left) and Android (right). We worked on designs that would be easy to replicate on both types of devices, rather than staying opinionated one way or another. This helped cut down our development time dramatically.

For a full video of our prototype in action, from signing up to watching 360 videos, click here.

In conclusion

This project forced us to work quickly, learn a lot about a new space in a short amount of time, and juggle various constraints (hardware, salesperson’s time, doctor’s time) all at once. User testing taught us a lot about how people experience VR for the first time and how to transition between mobile and VR quickly.

Ultimately though, VR was not the right path for Merck. Its newness was exciting but the unfamiliarity also gave salespeople an extra hurdle they didn’t need. Still, I’m glad our team explored this space. Both thoughtbot and Merck now have a clearer idea of when VR is appropriate and when it isn’t, and that’s invaluable!