How we built AI in Dovetail—magic search and Ask Dovetail

Quick! A product or design decision needs to be made. Evidence to support a point about your customers’ use case needs to be gathered. Context about a pain point needs to be gained. Where do you start?

For many of our Dovetail users, this starting point has been the trusty search bar. And recently, with the addition of magic search in Dovetail and Ask Dovetail in Slack, we’ve made it easier than ever for teams to get instant, evidence-backed answers to natural-language questions.

How our search function works

We use a hybrid search process that combines full-text (keyword-based) search with semantic search. The semantic search allows the user to search and match on concepts rather than exact keywords. Full-text is useful when the right keywords are used, but semantic is better at guessing the intent behind a question and matching of what it means.

Using both together allows us to deliver more relevant results.

We’ve seen product managers utilize Ask Dovetail to do continuous discovery into how customers are using a product or feature. Designers have told us they start with magic search when looking for feedback on key experiences and workflows to inform their design decisions. Even marketing and sales teams have described how they look for feedback on how customers perceive their product or solution to refine messaging strategies.

Today, we thought we’d pull back the curtain and explain some of the technical decisions that go into scoping, designing and building these solutions—starting with one of the most common questions: what subprocessor are we using. 

There are a variety of (almost) household-name AI LLM processors available. ChatGPT by OpenAI, Claude by Anthropic, and Gemini by Google, just to name a few. 

We decided to go with Claude 3.5 Sonnet for a number of reasons. Some of which are the following: 

Superior summarization accuracy and clarity

Claude 3.5 Sonnet is renowned for its ability to generate highly accurate and coherent summaries. Its advanced natural language understanding ensures that key points are captured effectively without losing the essence of the original content. This makes it an ideal choice for answering customer questions and identifying key points from the insights repository. We have also configured the model in such a way that it is not allowed to come to its own conclusion, drastically reducing the likelihood of “AI hallucination”—where the AI generates incorrect, nonsensical, or fabricated information. Ask Dovetail summarizes based on your data, and your data only.

Additionally, we also use Wikipedia-style citations, complete with links to the relevant data point in Dovetail. This extra layer of transparency helps you continuously build trust with our AI model.

Seamless integration with existing workflows

In everything we do, we always prioritize providing a seamless experience for our users. When it comes to the subprocessor we use and configuring the model, the ease of integrating into our existing workflows is critical. This greatly enhances the safety and speed with which answers are delivered to customers on Slack or the Dovetail app. In the case of Ask Dovetail, this translates to a conversational chat experience for users—ask questions in Slack and instantly receive evidence-backed summaries from Dovetail. The model we use ensures this flow is seamless for the user.

Integration with other technologies    

Our commitment to seamless integration extends beyond workflows to encompass a robust technological ecosystem. In addition to Claude 3.5 Sonnet, we integrate Claude Haiku for handling certain types of queries and AWS Comprehend for customer language and sentiment detection. When designing audio digests, we worked with Amazon to implement Polly text-to-speech technology, which brings text-based customer insights to life audibly.

To provide a coherent AI-driven search experience that our customers can trust, ensuring that these technologies integrate cohesively is at the heart of our decision-making.    

Our principles for integrating AI into search

At Dovetail, we have a set of guiding principles for AI tools in our products. They are:

• Human-in-the-loop—we make sure there are always human intervention points throughout the generative process.

• Trust and transparency—in Dovetail, explanations are provided for why AI has made certain decisions.

• Match process, not replace it—AI is a research assistant, it doesn’t replace researchers.

When thinking about how we might integrate AI into our search experience, we made sure these principles were embedded. 

Human-in-the-loop

Teams always have control over AI involvement in Dovetail. For Ask Dovetail, the database of information that it can access in its responses is controlled by the workspace admin. The admin can choose which projects and channels should be included in results, and they can also determine which object types should be cited as sources. For example, if you don’t want the AI using your unpublished insights as sources for its answers in Slack, you can filter them out entirely. 

Trust and transparency

In terms of trust and transparency, every point generated by Dovetail in magic search or Ask Dovetail is backed up by evidence from your insights hub. To minimize hallucinations, Dovetail is programmed to tell the user when insufficient information is available to answer a question, rather than making up an answer that may be incorrect or incoherent.

Match process, not replace it

The goal with Ask Dovetail and magic search is simply to match—and hopefully amplify—the way people are already using Dovetail today, by making it faster to find answers to questions, gather information, and form insights. It can never, and should never, replace the important role that people play in the analysis and synthesis of customer feedback.  

Best practices for prompts—magic search and Ask Dovetail

Ask the right questions

Imagine you’re talking to someone new in your organization. They’re fresh-faced and hungry to learn, but there’s some lingo that they haven’t come across yet. Think of the AI search experience in a similar way. It can understand what you mean, but it can miss some context in your workspace. To avoid this, try to specify what you’re looking for in your query, and make every word count. 

So what are the best questions to ask? Depending on your role and core objectives, here are some example questions you could ask Dovetail to quickly get the insights you need:

• Ask “What do [persona type or job title] want to accomplish when they use [product or feature name]?” to understand the core tasks or goals your product needs to support for particular customers.

• Ask “How are users responding to [product or feature name]?” to get a pulse on what customers are thinking and to help you make quick course corrections.

• Ask “How can we improve customer satisfaction with our pricing model?” to highlight common frustrations and turn insights into decisions that improve customer satisfaction.

• Ask “What features do customers consistently request?” in order to prioritize features that you know your customers will love.

Integrate the voice of your customers into your team conversations

We know that customer needs are constantly evolving. We also know that for many teams who receive ongoing, regular feedback from customers, the ability to get an instant pulse on needs and pain points is incredibly helpful. This is why we built Ask Dovetail. With Ask Dovetail you can bring the voice of customers straight into your team’s conversations in Slack and Microsoft Teams. To keep your team continuously informed, you can also schedule regular audio or text digests. Set these up based on common queries to get a regular pulse on what your customers are thinking.

This is designed to enable you to share customer knowledge across your team without interrupting your workflow or even leaving your internal communications channels. Simply ask a question, and press ‘Follow for updates’ at the bottom of the response. You can customize your digest by selecting specific projects, setting the delivery frequency, and choosing where it gets shared.

Having this dynamic flow of insights can help empower decision-making that accurately reflects the latest customer insights across the entire team.

Learn more about Ask Dovetail or try Dovetail for free today.