Crafting Better Personas with AI
Part 4 of the AI vs. Human: A User Research Showdown series
Summary
There are numerous new platforms claiming to accelerate or even replace your typical user research with the use of AI. We put these claims to the test in a head-to-head study comparing the effectiveness of AI tools and human UX researchers throughout the generative research process. As part of this test, both human researchers and AI tools (and combinations of the two) created personas to help understand and empathize with the target audience. Leveraging AI tools significantly sped up the process of identifying appropriate personas, but we found that human researchers were generally more effective at building out the personas with relevant insights.
Study Recap
In the rapidly evolving world of user experience research, the emergence of novel artificial intelligence tools presents a new frontier. As we stand on the cusp of potentially transformative changes, the question arises: How might AI impact traditionally human-led endeavors like generative UX research?
At Brilliant Experience, we performed an empirical study to directly compare the effectiveness of AI models and human researchers in conducting qualitative interviews. Our study centered on parents of young children planning international travel, examined under four distinct conditions, each varying in AI and human involvement. Across these scenarios, the researchers (or AI tools) produced standard deliverables: a slide deck report of key insights, a set of personas, and corresponding experience maps for each persona. We evaluated each approach not only for output quality but also for efficiency and depth of understanding.
Read more about our method and goals in our Executive Summary.
This Edition: Persona Creation
In this post, we’ll continue to explore our findings from the synthesis phase, focusing this time on our experience creating personas in each condition.
Personas are detailed, fictional representations of target users or customers, created to help businesses and designers understand and cater to the needs, behaviors, goals, and challenges of specific audience segments. Though the specific attributes contained within the persona vary depending on the use case, they are generally structured like the sample below, providing key characteristics and common emotions and behaviors associated with the sub-segment of users.
It is important to mention that we did not find an AI tool suitable for the visual design aspect of persona creation. Therefore, a human researcher developed the persona template used across all conditions. Our comparison focuses specifically on the identification of distinct participant groupings and the content within the personas.
Choosing an AI Tool to Create Personas
There are three main types of tools you can use for data synthesis: general AI tools like ChatGPT, specialized research synthesis tools designed specifically for analyzing qualitative research data, and ideation tools with features meant specifically for creating personas.
Using General AI Tools for Persona Creation
General AI tools like ChatGPT are surprisingly powerful when it comes to assisting with persona creation. You can ask these tools to help you identify personas from a set of transcripts or interview notes, draft the persona content itself, and refine the copy.
As with all data synthesis tasks, it is essential to ensure that the AI tool you are using maintains data privacy within the confines of the conversation and does not use your data to train its model. For participant privacy reasons, we recommend only sharing research data with AI systems that guarantee data confidentiality.
Using Research Synthesis Tools for Persona Creation
Some tools like Next and Research Studio have specific features designed to draft personas and/or custom AI features that can help you work with your data in a number of ways, including generating personas from it.
Important features to consider when selecting a research synthesis tool for persona creation are:
Allowed File Types: Some tools allow both videos and transcripts, while others accept only one or the other.
Functionality: Some tools have one-click persona creation, while others come with more custom AI functionality. Either might be appropriate depending on your goals.
Pricing: As we mentioned in the Data Synthesis edition, tools specifically for research synthesis can be expensive, and often require an enterprise license to access the full feature set.
Using Ideation Tools for Persona Creation
Ideation tools like QoQo and UX Pilot can also generate personas, but are best used at the beginning of the research process to develop hypothesized proto-personas ahead of actual data collection. They draw on their existing world knowledge to draft a persona given a description of the target audience. So while they won’t be able to identify distinct personas given a dataset, they can help you begin to learn about an audience or build out an initial draft of a persona you’ve already identified from your own synthesis.
AI vs. Human Comparison: Persona Creation
Personas are one of the most common deliverables coming out of a generative interview study, designed to help teams understand and empathize with their users. This format effectively presents key insights from exploratory studies that reveal distinct sub-groups within a particular audience.
Initially, we were skeptical about using AI tools to generate personas. However, we found that AI can be a valuable partner for specific tasks in the persona creation process. As we outline below, some human intervention is still necessary to create a truly useful set of personas, but this remains an excellent area for AI-researcher collaboration.
AI Win: Identifying Likely Personas
Identifying meaningful, distinct personas from data collected in a generative interview study often requires extensive manual work and research expertise. Given the limitations of AI in extracting actionable insights from such datasets, we were initially skeptical about AI’s ability to synthesize the data into personas. However, our research revealed that identifying possible personas is actually an excellent use case for AI’s top-notch summarization capabilities.
We were pleasantly surprised that general AI tools like ChatGPT could generate a set of distinct personas that closely matched those identified by a trained researcher. Impressively, the AI achieved this using only the study transcripts, without any additional human synthesis.
AI Win: Refining Persona Content
Unsurprisingly, general AI tools were also great at refining persona copy drafted by a human researcher, ensuring that the writing was concise and felt unified throughout.
Human Win: Categorizing Individual Participants
Given AI's success in identifying distinct personas from transcripts, one might assume it would easily categorize study participants according to these personas. Unfortunately, this was not the case. The AI tools we tested struggled to map individual participants to the personas they identified, either providing inaccurate output or failing to provide mappings altogether. Luckily, this task is typically straightforward for human researchers, who we suggest handle this aspect of persona creation for the time being.
Human Win: Drafting Persona Content
Unfortunately, AI also struggled to draft accurate content to build out the personas. We found that AI tools drew too much on world knowledge instead of on the research data itself. This resulted in generic deliverables and content that was (erroneously) repeated across multiple personas. Research-specific AI tools that limited the model’s use of general world knowledge were slightly better at drafting persona content, but the added input of a human researcher greatly improved AI’s performance (see ratings from independent reviewers below).
Human Honorable Mention: Visual Design
As we mentioned earlier, we were not satisfied with any AI tool's ability to create a visually appealing persona template. However, we want to highlight that there are plenty of Figma plug-ins (e.g., AI Designer) and other AI tools available that can generate visual assets for your persona. Ultimately, we found the output to be too simplistic.
When to Involve AI in Persona Creation
Given our success in the AI Assisted condition of our study, our persona creation process has evolved to follow these steps:
Use a general AI tool to brainstorm distinct personas given a set of study transcripts
Have trained researchers associate individual study participants with the appropriate personas and draft persona content
Use a general AI tool to refine the language in personas
Have trained designers or researchers build the visuals for the personas
Conclusion
Personas are a great tool to help a product team better understand and empathize with their target audience. A variety of AI tools can be leveraged to create these deliverables, from general AI tools that can help with data synthesis and copy to design tools that can put the content into a visual format. In our head-to-head competition of these tools and human researchers and found that when it comes to creating useful personas:
Persona creation is an excellent area for collaboration between human researchers and AI
AI tools can greatly speed up the process by helping to quickly identify distinct personas and refine copy
Humans should continue to have a heavy hand in generating the content for personas and mapping individual participants to their associated personas
For these reasons, we consider AI a “trusted partner” when it comes to creating personas. AI tools can’t yet be left unattended for persona creation, but can make this phase of the research process far more efficient.
Interested to learn how AI tools stack up against human researchers when creating experience maps? Subscribe to our newsletter to make sure you don’t miss an issue of our AI vs. Human series – or our ongoing AI 4 UX video interview series featuring founders of some of the most popular AI tools for UX research.
