How customer insights saved us from the AI echo chamber

Looking back at 2023, some will remember it as the year AI went mainstream. For me, it was the year my original vision for Dovetail became possible.

Before I explain what I mean, I want to take you back and share the insight that led to the start of Dovetail.

Almost everything that’s successful starts with some kind of insight

We exist to improve the quality of everything—that is our mission—whether it’s a software product, an electric car, a power tool, or a government service.

We believe that great things are made by teams who talk to their customers. The problem is that in many organizations, it’s actually really difficult to get direct face time with customers.

Leveraging insights from teams who do talk to customers often, like sales, customer services, support, and research, is also quite difficult. This could be due to fragmented data silos, silos between teams, collaboration challenges, or compliance reasons.

When I started Dovetail, I felt like there were so many tools to help teams build products. Product managers had Confluence, Notion, Jira, and Asana. Designers had Sketch at the time, and now Figma. And engineers… well, they were super spoiled. They had everything from AWS and VS Code to Github, Linear, and Datadog—a real plethora of tools.

While there were so many amazing tools to help you build something, it felt like there weren’t many tools to help you figure out what to build. And after all, development is just one half of research and development. Building the right thing is super important. It’s not as simple as design, build, and ship. Almost everything that’s successful starts with some kind of insight—an insight on your users and their pain points, on the market, or on your competitors.

Before Dovetail, I worked at Atlassian as a product designer. I always wanted to know more about how our customers used our software so that I could do a better job. Since there was no obvious place for me to go for the information I needed to inform my design work, I asked a product manager, Sherif.

Sherif had been at Atlassian for a long time and had a lot of institutional knowledge. The problem is that when people like Sherif—project managers, researchers, or customer success professionals—leave a business, they take all of their knowledge with them. There’s no record system for their unique knowledge of the market, customers, buyers, and competitors. These are the insights that fuel the strategy, vision, and roadmaps—the business’s competitive advantage.

Rather than relying on the Sherif in your organization and all of that institutional knowledge, just ask Dovetail. This is what I always wanted to build—what I call a customer insights hub. This was my vision for Dovetail, but the technology just wasn’t there back in 2017.

Dovetail enables teams to uncover the why. You can use it to make sense of everything from customer calls to surveys and to implement strategy and roadmaps through insights. The consensus is that it certainly beats the spreadsheet!

Helping teams decide what to build next

We look at every project, every highlight, and every insight in our product as a step closer to our mission of helping you improve the products and services you’re working on. In the past year alone, our customers have been really busy with Dovetail, creating over 140,000 projects, 26 million highlights, and over 600,000 insights!

Last year, when AI exploded, I saw how it could unlock completely new possibilities. I saw people trying to use ChatGBT to analyze unstructured data and a lot of ChatGBT wrapper tools trying to help with product discovery and research. But honestly, for me, a lot of those felt like solutions looking for a problem, and the problem that we’re ultimately solving doesn’t really have anything to do with a specific technology. For me, it’s this: how do teams actually decide what to build next?

Last year, Dovetail ran a series of projects to uncover how teams leverage insights to make decisions. We spoke to product leaders, designers, CEOs, and founders. We also dived deep into how AI can enhance workflows, and we spoke to lots of our customers about their feelings around AI—and you have a lot of feelings around AI! We asked, what are the inputs that feed into strategy, roadmaps, and design? And, ultimately, how do you figure out what to build next?

We developed an overview of the types of insights an organization collects. We also mapped how product teams use these inputs to make decisions—how do they decide what to do next? It’s incredibly complex.

Teams need to leverage many different data sources, work with multiple stakeholders, and deal with many different kinds of inputs—research insights, what the business needs, goals, OKRs, you name it. To put it simply, insights are really just too slow and their shelf life is often limited.

So today, we’re here to solve that. We’re utilizing new technology like AI to streamline work, streamline workflows, and improve efficiency, and I want to run you through a few new features that we’re launching to help you decide what to build next.

Magic features, new and improved

Here at Inside Out 2024, we’re launching some of our biggest advancements yet. The theme for this launch is magic.

For us, “magic” means experiences that fit seamlessly into your workflow. So, rather than look to replace humans with AI entirely, our approach is what we call “human in the loop.”

78% of online and in-person attendees told us that the time it actually takes to get to insights hinders product development. But imagine if you could transform interviews, sales calls, market research, industry reports, and support tickets into insights quickly—like magic—so that you can make customer-backed decisions, motivate designers and engineers to build that next differentiator feature, and give leadership confidence in your strategy and approach.

Well, now you can.

Magic transcribe

When you drag and drop a customer call into Dovetail, it now transcribes your video automatically.

Off the back of user feedback, we’ve also improved speaker recognition by 25% for even more accurate transcripts in 41 languages, and we’re working toward further improvements to transcription this year.

Magic summarize

Magic summarize is a new feature that automatically summarizes any video or audio call, breaking it down into chapters and allowing you to quickly navigate the transcript.

Head to the right of the screen and open up the new sidebar. On the “Summary” tab, you can see that Dovetail has broken up the video into chapters and named each topic.

This feature enables you and your team to quickly jump to specific topics in any video, which cuts down the time it takes to get to insights. And because we don’t expect AI to fully take the reins, the summaries are editable, enabling you to add context and observations.

This is an example of what I said before: we like the “human in the loop” approach. We don’t want AI to fully replace workflows. We want you to still be in control.

You’ll also notice the new sidebar. A bunch of metadata, like fields, highlights, text, and comments, is easily accessible, but it won’t overwhelm new Dovetail users.

Magic summarize also works well on documents. You can drag any PDF, PowerPoint, or keynote deck into Dovetail, and we will extract all the content and summarize it for you. And it happens pretty quickly!

Don’t forget you can also highlight and tag documents in Dovetail.

We’ve been testing Magic summarize with a few customers, and the feedback has been great. Our customer Heather from Webflow mentioned that the new Magic features open up a whole new world of possibilities. The automatic timestamps make it easy for visitors to digest the most relevant insights when she shares something like a customer interview.

Magic highlight

Highlighting in Dovetail allows you to quickly highlight moments in the transcript and extract important things to turn them into reels.

Switching from the summary tab to the highlights tab in the sidebar, you’ll see one of our new features that we call Magic highlight. This feature automatically suggests highlights in transcripts, massively reducing the manual work.

Simply click “Suggest” and see what Dovetail has to suggest. When you get a suggestion, you’ll actually see a reason why Dovetail thinks it’s important. You can go through and accept or reject these highlights.

For even more control over the suggestions, you can actually tell Dovetail your learning goal. Simply type in a custom prompt and click “Suggest” again to see additional highlights.

Magic reels

Now, I know I don’t often have time to watch entire recordings of customer calls, and I’m sure a lot of your stakeholders don’t either. At the top of the video, click the “Highlights only” tab. This will shorten a video to just the key moments.

When you take Magic summarize, Magic highlight, and this new feature we’re calling Magic reels together, you can condense video from 30 minutes or an hour to the key moments in only a few seconds.

This is a great way for your team to consume the most important moments on calls without needing to watch the entire video. We’re also working on another tab that does this automatically based on the summary.

Magic redact

You can also Magic redact to select things in the transcript and remove them. You can also blur video and silence the audio. We know participant privacy is a really big deal for a lot of our customers, especially in enterprise. Magic redact is one of a long list of features we’ve launched over the past 18 months to support our largest customers with compliance.

Lastly, you can share specific highlights with colleagues. Your sales and CS teams can highlight interesting quotes from customer calls and share them with your product team, for example.

Magic cluster and the highlight canvas

Alright, let’s shift gears a little bit and go deeper and jump over to the “Highlight canvas.” Thanks to Magic highlight, you’ll have a whole bunch of highlights of important moments across a bunch of different customer calls and documents. You can look at those highlights from a different perspective, grabbing and dragging them onto the canvas to find patterns.

Then, we can use a new feature called Magic cluster. Magic cluster automatically groups highlights by theme. It uses machine learning and large language models (LLMs) to figure out these unstructured highlights’ classification.

As we know, AI isn’t perfect yet. But as always, you’re in control. You can now grab some of the highlights and move them to a different group. It works just like any other affinity mapping tool.

Our customers have also been asking us for the ability to duplicate highlights on the canvas. Now, you can do that by just copying and pasting a highlight. You can also add a duplicate from the reference picker.

The last thing I want to show you is that we’ve made it a lot easier to go from the Magic cluster view to a shareable and actionable insight, and also a highlight reel. Simply select a group and click “Add to insight.” Give your insight a name, and in just a few seconds, you’ll have created an engaging and shareable insight. You can also add context around the insight if that’s useful for your team.

Jumping over to the “Insights” tab, one of the cool things here is that we can actually use Magic summarize on insights, too. It summarizes all the clips from different calls, giving you a shareable insight and a reel. It’s easy to go from a bunch of different customer calls, documents, or audio recordings from multiple people to a shareable insight complete with direct video evidence, which you can share with your team.

At Dovetail, one of the ways that we use this feature is by having a monthly customer showcase. We all sit together and watch highlights from customer calls in reels just like this. It’s a great way to embed customers in our culture. We try to be very customer-centric and align our product team behind the biggest problems our customers face.

So, with Dovetail, not only can you find insights super quickly to drive decisions, but you can also drive a culture of customer centricity. It’s also great for uncovering patterns in interviews, feedback from sales discovery calls, and pain points in usability testing, and establishing themes from industry reports or market research.

These six new features in Dovetail enable you to go from a few calls to shareable insights in just a few minutes. And the good news is that all of these magic features are available now in Dovetail.

Magic search

But what if the answer already exists, and you just need to find it?

I’m super excited to introduce Magic search. It’s a whole new way to find answers to your customer questions.

So, you can jump in and type a question like, “What are the top challenges the customers have?” Rather than a simple keyword search, Magic search actually uses semantic search to return results based on meaning. With all your insights in Dovetail, this means you can actually ask your Insights hub questions and get answers. Dovetail answers your question through a Magic summary of the search results, so there’s no need to read through endless transcripts, documents, or survey responses.

I know what you’re thinking: what if the insights are no longer relevant? What if they’re in another team or department?

Well we’ve also redesigned search filtering. Now, you can really easily drill down and view things from a set time period or a specific folder that belongs to your department. This is really important for our bigger customers who have thousands and thousands of highlights and insights across multiple teams and departments in one workspace. Once you have applied some filters, simply rerun the summary to get a scoped-down answer.

Citations

You may notice there are some numbers in the summary. Well, we know how important it is that you can trust these summaries. That’s why we’ve launched citations. Citations actually allow you to trace the real data and insights back to their source.

All those transcripts, highlights, and documents we created earlier contribute to the answers in Magic search. And yes, you can also create an insight directly from the search results. Click “Create insight” at the bottom of the summary to take all those references and citations and apply them as evidence inside a shareable insight.

I personally believe that Magic search actually changes the nature of what an insight is. Rather than it being something that’s created at the end of a project for use in some uncertain future where you don’t really know what the context is going to be, it actually makes it an on-demand insight that stakeholders can use whenever needed. For example, they might use it to inform a design ideation session, a roadmap planning prioritization session, or even a board meeting.

Magic search really showcases the value of building an insights hub. And it enables self-service insights, which allows teams to figure out what to design, build, and ship. It’s also a great tool for go-to-market teams to influence the product roadmap.

A new way for product teams to turn user feedback into real-time, continuous, actionable insights

You don’t stop listening to customers after you ship something. Great product teams continue listening to user feedback and factor improvements into the product roadmap. I’m talking feedback in the form of support tickets, app reviews, posts on your community forum, or even social media. You might be getting thousands a month and not really know what to do with them.

So today, I’m super excited to introduce channels in Dovetail. It’s a new way for product teams to turn user feedback into real-time, continuous, and actionable insights.

Channels uses machine learning and LLMs to automatically and continuously classify high-volume unstructured data. It’s built for scale. It works on tens of thousands of data points.

From a single dashboard, you can now track your high-volume voice of customer feedback. Dovetail will continuously classify new data coming in and group it by theme. As a product manager, this helps you uncover patterns before they become problems. It keeps you in front of issues so that you can more effectively prioritize roadmaps. You no longer need to tag feedback manually in a spreadsheet.

Channels integrates natively with Intercom, Jira Service Management, Front, and Zendesk, and many more first-party integrations are coming, including the Google Play Store and the App Store. A channel can also support different data sources, so you can connect multiple integrations to one channel.

In line with our “human in the loop” approach, you’ll work hand in hand with our classification engine to surface the themes that are important to you.

It’s early days, and we’re still refining the way the themes are classified to make sure they’re as accurate as possible. As I’m sure many of you know, AI has plenty of limitations and quirks. That’s why we’ve made sure that you can rename, merge, and split the themes so you’re in total control of what the dashboard looks like.

Channels is currently available in beta to all paying customers. Join the waitlist here.

Escaping the AI echo chamber: finding real problems and solving them in a real way

Not only does Dovetail help you with product discovery at the beginning of the life cycle, giving you the insights to spark your next innovation, but it also helps you understand how everything went. You can now track your voice of customer feedback, support tickets, reviews, and so on.

Ultimately, we want to motivate your team by hearing customer problems straight from the source, and we want to help you figure out what to build next.

We had a lot of fun last year trying to figure out how to use AI in Dovetail and how to apply it. We did a lot of research, and we talked to a lot of people. We tried to escape the AI echo chamber by finding real problems and solving them in a real way—with intention, rather than just approaching it as a sort of ChatGBT wrapper.

So, thanks to everybody who contributed to our research in the past 12 months. I’m really excited to unveil all these new features.

Editor’s note: This article is a condensed overview of Benjamin Humphrey's keynote session at Insight Out 2024.