Benchmarking Training Products

Overview

Survey (n = 2,000); compared five live online training tools

Goals: compare satisfaction and usability across training tools; understand the needs and frustrations of trainers; quantify and triangulate learnings from interview studies

Measures: satisfaction, usability, feature preferences, and more
Statistics: ANOVA, linear regression, descriptive statistics


Lead researcher
Designed survey protocol; analyzed data; created key findings slide deck and readout

My role


Impact

Influenced the product roadmap: Survey findings informed the product strategy and the development of GoToWebinar-Training Edition

Influenced the design roadmap: Identified opportunity to delight trainers by improving an important (yet, difficult to use) training feature; collaborating with UX designers to improve feature

Increased empathy: educated UX, product, and engineering teams about trainer’s needs and frustrations


What I learned

One challenge was managing the tension between stakeholders’ expectations from past surveys and requests to use new survey measures. In this survey, I included a variety of measures in order to accomplish several goals at once (i.e., it was a lengthy survey). While this was an effective choice to ensure that lots of stakeholders received the information they needed, I had the sense that future surveys could benefit from a more focused and strategic approach.

This led me to be curious about stakeholders’ needs and expectations for quantitative data at LogMeIn. I wanted to know what was working well and what could be improved. To learn more, I initiated an ethnography of quantitative data use at LogMeIn. I spoke to 15 people from 7 teams who use quantitative data at LogMeIn. I learned that gaining access to data is a challenge and that there is a desire to learn from other data users across the company.

To help people at LogMeIn to be more data aware, I am facilitating Lunch & Learn sessions to open up opportunities for informal conversations about data across the company. I initiated a quarterly Research Pitch Hour meeting — an opportunity for stakeholders to share their research ideas and for UX researchers to increase transparency around their roadmaps and findings. Pitch Hour has helped stakeholders to identify overlapping research priorities and helped folks at LogMeIn to accomplish more by working together!