A couple of weeks ago, we rolled out an update to Yesware that changed the position of our most-used feature: the Gmail dashboard.
It was a big risk to take.
With over 400,000 people relying on Yesware to help them get the job done, our product team must constantly juggle the needs of both our large pool of existing users and the feature needs of our evolving product. Suffice to say, changing any part of the user interface is not a decision we take lightly.
But being big does not excuse us from the necessity of evolving. It just has to be done in a measured way, using data-driven decision making.
Here’s an inside look at how and why we decided to change the Gmail dashboard.
We Heard You Loud And Clear: The Inbox Dashboard Had Its Limitations
The inbox dashboard is our most widely used interface. But despite its popularity, we heard from a number of you that it could be better. There were a handful of limitations:
- You had to navigate to the inbox view in order to view your tracking events.
- It did not support the use case of cross-referencing email content with tracking data.
- The interface did not provide ample space to accommodate extra functionality (like links to Mail Merge).
As regular users of Yesware ourselves, we knew what we wanted out of the dashboard. But it was not 100% clear yet whether others having the same problem.
Enter The Dashboard Positioning Experiment
If you roll out a feature to 100% of users only to find that it breaks a crucial workflow, you’ve needlessly sacrificed a lot of time and money. That’s why we use experiments (i.e. an interactive prototype written in actual code) to learn as quickly as possible from a small subset of users without upsetting a larger audience.
We ran an experiment on the inbox dashboard, starting the last week of April. In order to make the dashboard omnipresent across all of Gmail’s views, we pinned it to the top, just below the search container. In order to limit the number of variables tested, we changed very little about the visual design.
We then exposed the change to 1% of Yesware users and collected both subjective and usage-based feedback. The most dramatic results were gathered off of the usage for our most popular interaction: clicking on the Events tab to expand the dashboard.
What happened: There was a 30% bump in the number of users who performed the action as compared with what we’d expect from 1% of the audience. We saw a similar, though not quite as dramatic, increase in the total number of actions.
Real-Life Results: The New Dashboard Was A Change For The Better
These usage-based results, along with positive feedback (gathered through Google Form Surveys) gave us our answer: The new dashboard position was a success.
In the last week of April, we began pushing out the dashboard to more and more of you. Minor bugs notwithstanding, the rollout went smoothly. We continued to track usage carefully over the next few days. Below are some results.
The Events-Tab interaction which performed so well in experiments continued to perform well in production. From the start of our experiment to May 7, you can see that overall usage increased by 18.4%. Even taking into account natural growth from new Yesware users, that is a dramatic bump from a single user interface tweak.
Data-Driven Decision Making In Action
What may look like a small change is actually the culmination of a data-driven process, pointing us in the direction of what is going to help you work more efficiently. It’s how we knew rolling out the new dashboard positioning was a big risk worth taking. Even more recently, it informed our decision to redesign the Salesforce sidebar so that it’s always available, wherever you are in your Gmail workspace.
With enthusiastic coders on your team, quickly building features can seem easy. But it is rarely the best course of action. Rush-jobs will only end up costing you valuable time in the long run. Spending a week or two running experiments is a worthwhile investment that could save you from wasting months building the wrong feature.
Data-driven results are a big part of how sales organizations should operate. At Yesware, we think product teams should work the same way. In the coming months, we will be running many more experiments, so if your team would like to test drive exciting new features, please get in touch with us at firstname.lastname@example.org.