How AI is changing my role as a UX Researcher working with B2B clients

This year, the adoption of Generative AI as a more mainstream tool has altered my workflow and sped up my user research processes with our clients.

 The UX community, known for its vivid minds and passion for innovation, is of course enthusiastically embracing modern technologies, while keeping users at the core of our work. A recent 2023 study conducted by the NN group, surveyed 841 UX professionals about their use of AI. The findings were revealing: 92% of respondents had used at least one generative AI tool, and among those, 63% used them several times per week, if not daily. The most common application being text-content generation and editing, preparing desk-research summaries and specific UX-research documents. 

Perhaps it would be helpful if I told you about phases involved in UX Research and how I, as a UX Researcher, reorganised my process by leveraging AI tools.  

B2B UX research 

In terms of UX, Business to Business (B2B) projects traditionally differ from Business to Consumer (B2C) by having more complex buyer journeys, that often require a deep dive into unfamiliar areas, and particularly those involving intricate B2B problems. As a UX Researcher at Torpedo, some of these activities include: 

• Client workshops,  
• User and stakeholder interviews,  
• Usability studies, 
• Surveys and data analyses such as card sorting and tree testing,  
• Competitor analyses and benchmarking,  
• UX audits and expert reviews. 

As you might expect I use different research methodologies depending on the client’s brief, problem statement, time allocated, and resources available.  

In B2B it’s even more crucial to understand who the target users are and how they interact with and feel about the product or services. As McKinsey states:

“A strategy focused on improving the experience of existing customers can deliver breakthrough growth for incumbent companies—often more than double that of their industry peers. Companies start by defining their desired financial outcome and then prioritize the customer experience improvements that will deliver that outcome. Leaders in companies executing experience-led growth strategies take care to understand customer pain points standing in the way of growth—such as complicated buying processes, tracking and delivery issues that hinder recurring purchases, or a lack of integration between channels. “

Quicker, quality research and discovery

Today, the process has been transformed. I open ChatGPT or Bard AI, asking it to act as my user research partner, thereby giving me an extra pair of hands and speeding up my desk research. By prompting ChatGPT to assume a specific persona, I provide a specific context for the conversation. This can help narrow down the scope of the discussion and elicit responses that are relevant to the chosen identity. I can filter out irrelevant responses. When I need real data, I simply ask for a source, which it provides approximately half of the time. I filter responses based on my User Researcher experience, not heavily relying on AI. It is more a reflection exercise for me, where ChatGPT acts as the partner, whose point of view I can either agree with or disagree according to my knowledge and experience.  Working this way, I found myself engaging in more meaningful conversations with clients, since I could be better prepared for even last-minute meetings. 

AI as my new UX research assistant  

With my lovely AI research assistant onboard, AI can also help out on some of the more laborious tasks, or where I need a variety of ideas to test. Here are some examples of additional tactical research activities that AI assists me with:   

• Brainstorming ideas, suggest ice breaking activities for workshops, draft the timeline of the workshop based on the goals that I provide. I was surprised how creative AI can be while suggesting great activities that we can do online during our internal workshop or even workshop with clients. 
• Gathering preliminary insights, such as summaries and information architecture of web pages, to better understand the client’s digital landscape and user interaction points. In the past, I would manually replicate a sitemap, but now AI does it for me; my role is to validate its outcomes. 
• Helping with the categorisation and labelling of qualitative data to identify themes and analyse relationships within the data obtained from users. 
• Validating draft interview questions or usability scenarios to ensure they are clear and straightforward for users to respond to and follow. 
• Simulating user responses to accelerate the thought process and anticipate potential user reactions and feedback. 
• Reviewing and refining survey questions, particularly helpful as English is my second language, to ensure clarity and precision in communication. 
• Summarising data into graphs and quantitative outcomes, which I find valuable in reducing cognitive load when analysing and presenting large volumes of data. 
• Translating research findings into user or job stories and diagrams with the integration of plugins, so they can be used by development teams to back up their technical tasks. 

AI cannot do it all

In May 2023,another survey of just over a 1000 UX professionals was conducted to look at how they use AI in research. The findings report that: 

“The #1 concern is the potential for inaccurate and/or incomplete analyses mentioned by 29.7% of UX professionals. After inaccuracy, the biggest concerns were a lack of data privacy (19.7%) and the potential for introducing bias into study findings (14.1%).” 

While AI does indeed help me in my role, I want to highlight a few crucial limitations: 

• AI cannot build the entire research study on its own. It is limited to helping with preparatory work. As a result, editing the research plan, tag coding, emails, and questions is much more straightforward, making it easier to start a study and faster to complete preparation work. 
• AI cannot help evaluating usability of the website. According to Baymard Institute: “ChatGPT-4 has an 80% false-positive error rate and a 20% accuracy rate in the UX suggestions it makes.” 
• AI cannot create a real user persona. It often exhibits bias and usually lacks the specification of the ‘why’ behind human actions. However, it can aid in developing proto-personas and aid a better understanding of the target audience at an elevated level. AI’s responses are based on available data; without sufficient data, it cannot provide a complete overview. 
• AI does not give reliable answers. Since AI does not possess the full content context, it lacks accuracy. Two different ChatGPT message windows can generate different responses, let alone a different engine like Bard. 
• To better inform AI about my project, I need to constantly feed it with good data – the adage ‘garbage in, garbage out’ was never so true than using AI. Perhaps, this limitation can be addressed by creating a specific AI persona. Recently, OpenAI introduced custom versions of ChatGPT, and I am in the process of testing this based on my needs. 
• One area where AI tools are valuable, is for transcribing and summarising stakeholder kick-off calls, meetings or interviews. Documenting everything is crucial when onboarding a new project, as even minor details can be significant later in the research process. I can always search through transcripts to find the specific quote I need. However, the AI tool sometimes summarises information with a different meaning than I have captured. I find that I need to constantly chat with the AI tool to provide my feedback. AI still struggles with various accents, and it often fails to accurately capture product names, abbreviations, and specific technical jargon. This frequently may lead to misunderstandings or incomplete data, particularly in complex or niche subjects where precise terminology is crucial.