
AI IN ACTION
A Market Entry Research Case Study
by Arthur Weiss
As capabilities improve, gen AI’s role in various types of research will expand. However, it is crucial to avoid complacency; verifying sources and data remains essential. |
Can generative AI (gen AI) tools support rapid market research and analysis? A structured exercise at the Institute for Competitive Intelligence’s 2024 conference tested AI’s ability to conduct comprehensive market analysis under severe time constraints. The results demonstrate both the powerful potential and important limitations of AI-assisted research for information professionals. AI excels in tasks such as summarizing content and drafting reports—but can gen AI manage the complexities of an entire online research project?
Each year, the Institute for Competitive Intelligence (ICI; institute-for-competitive-intelligence.com), based in Germany, includes a preconference training case exercise at its annual conference (competitive-intelligence-conference.com/2024-1/challenge). I was a facilitator for the 2024 exercise. Usually, ICI includes all the material required for the case analysis, but the 2024 conference took a different approach. Could AI tools be used for a market entry research analysis, including establishing the initial research design, conducting online research, and producing management-level investment recommendations?
The case presented attendees with a (fictional) organization that needed urgent market research for a proposed new product line, with just a day to research and report before a board meeting the next day. (The case study explained that the short timescale was because your boss had forgotten to tell you about the research requirement.) Producing a report using traditional online research approaches in the time allowed would be impossible—even with a team of people. Could gen AI do the job?
The Case Study
The fictional scenario featured the CEO of Cold-Boomer Cooling Solutions (CBCS), a diabetic who struggled to maintain his insulin at optimal temperatures during air travel. CBCS believed there was a market for new products that would keep insulin pens (and similar self-administered injections) at the correct temperature when traveling, especially by air. The CEO, and his board, wanted to know these three things:
a) Was this a viable long-term market opportunity?
b) What existing products serve this market, as well as their strengths and weaknesses, plus what gaps could CBCS fill if it entered the market?
c) If a decision were made to enter the market, what approach should CBCS take?
Participants in the exercise were asked to prepare a report and a board presentation that answered these questions, working either alone or as part of a team.
The Challenge Exercise
Not all participants had experience with AI tools. Thus, apart from testing if it would be possible to do such a project, another objective was to teach gen AI tools to those who had not used them fully. The exercise was structured in three stages, with participants working individually or in teams. Each stage took around 2 weeks, allowing participants to complete the tasks along with their regular job requirements.
Preceding each stage was a webinar giving guidance on different AI tools, how to prompt effectively, what to do and what not to do, and similar instructions. Following each stage, ICI facilitators then assessed the team/individual participant results, giving feedback on their submissions.
Stage 1—Launch
The launch stage introduced participants to the case, requiring them to outline a research design created using gen AI tools to address the board’s questions.
All the AI tools used provided well-rounded suggestions on key areas to investigate:
- Market size and demand
- Travel behavior for diabetics globally
- Existing products and their features (competitive landscape)
- Market gaps
- Market entry strategies, including financial projections
- Future trends
- The tools also included suggestions that few of the participants had expected, although in hindsight, these
were obvious
- Regulatory approval requirements (air and medical)
- An intellectual property review, including patent and trademark searches
Stage 2—Carrying out the Research
The second stage took the requirements suggested in the AI-generated research design. Participants developed prompts to research each topic. For instance, participants explored global diabetes prevalence, focusing on insulin-dependent individuals and their travel behavior.
The AI tools were very good at finding data, broken down regionally, on the current numbers of diabetics, the proportion taking insulin, and their likelihood of travel, plus growth trends. Existing competitors were discovered, including product pricing and a brief strengths and weaknesses assessment for each one. Although the leading products were named by each AI tool, there were differences: Not all mentioned the same secondary players.
There were also language-dependent differences. For example, the websites given for products depended on the prompt language and the prompter’s location. Perplexity named competitors, highlighting features, but also included images of other products available on Amazon that were not in the main list. One of these products revealed a limitation of current AI tools. The facilitators were curious about the company that produced the Penguin Insulin Cooler Travel Case–Reusable Temperature-Controlled Diabetic Insulin Pen Cooling Case that Perplexity found on Amazon. Interestingly, none of the AI tools could identify the manufacturer behind this product despite locating promotional materials and the brand website (penguincoolcase.com). The actual company, 1922 Pty Ltd of Sydney, Australia, was easily identified using traditional OSINT Techniques (osinttechniques.com), in this case, checking the terms and conditions on the brand website, which the AI tools did not look at.
When assessing participants’ results, the facilitators also examined the data they provided to see if there were any “hallucinations” (false or made-up results). Were the source websites and data real? Some participants tried to short-circuit the process by combining parts with lengthy prompts that attempted to address multiple research topics. In general, their answers were weaker—and more likely to include hallucinations.
This issue also arose for topics such as trends, where prompts required significant drilldowns into the data for precise numbers across time. This showed how crucial it is to validate the research results by checking sources provided and proved to be a weakness when sources weren’t given (participants were told to ask for sources to be included in their prompts). Nevertheless, some of the same issues would have arisen with conventional research approaches when there was a lack of data. The difference is that a competent researcher would admit to not knowing while an AI tries to please by pretending to know.
Stage 3—Producing a Report
The next and final stage involved taking the data and creating a data dump in either Microsoft Word or Excel or as text files. These were then fed into the AI tools to summarize and to produce reports. Participants were tasked with creating reports tailored for senior management, limited to eight pages, including a summary and actionable recommendations. Typical would be a prompt such as the following:
Please produce a report on products available to keep insulin at the optimal temperature when traveling based on the attached documents. Include a report summary with recommendations and a table summarizing each product and key differences between them. Use bullet points as needed and text if this would be clearer. The report will go to senior management who need to assess the competitive landscape for this sort of product so they can identify any market gaps our company could fill. The report should be no longer than 8 pages.
Generally, all the reports were of an acceptable standard—the differences related to the data gathered at the research stage. Although token limits (which determine prompt input/result output length) have increased since this exercise, this was a limiting factor for some of the gen AI tools as too much data could not be analyzed in one go.
As well as producing a text report, saved as a Word document or imported as a text file, participants were asked to produce a presentation and, if they wanted, a video. In the webinar before this final stage, the AI video production tool, Invideo AI (invideo.io), was introduced as one tool participants could utilize. The facilitators also suggested asking the AI tools to produce slides that could be input into PowerPoint. Some participants produced excellent and captivating presentations using AI presentation generators such as Gamma (gamma.app) and Presentations.ai (presentations.ai). One group specified a prompt to produce slides and then asked the gen AI tool to output a VBA (Visual Basic for Applications) code that could be used to import and produce PowerPoint reports.
WHAT WE LEARNED
This exercise demonstrated gen AI’s potential to empower marketers by significantly accelerating comprehensive online research projects. The participants were surprised at how much information could be so quickly found and analyzed to produce the reports. Feedback was overwhelmingly positive. Of course, if this had been a real-life project, further work would be needed, including primary marketing research to follow up from secondary research. Although AI tools can provide guidance for this and even provide a questionnaire, this exercise did not test it.
In addition, only free sources were used—no commercial reports or news sources were accessed—as would be the norm if this had been a real project. However, as a first cut, gen AI appears to cut the mustard with ease, although there were limitations on what could be found when asked for an in-depth examination on a particular company. Hallucinations were also a significant danger. All results needed to be thoroughly checked. This was especially the case for overly complex and long prompts. Better results came when tasks were split into small, manageable chunks, with follow-up questions to drill down if the results weren’t initially clear.
There were also differences between those participants who had prior experience or had access to the premium/paid versions compared with those who used free versions of the gen AI products. Additionally, products such as Perplexity performed particularly well when searching for detailed, up-to-date data. In contrast, ChatGPT and Claude appeared better for the report and research design stages. The lesson here is that depending on just one product can limit usefulness.
Gen AI continues to evolve rapidly—this exercise relied on earlier versions of these tools. As capabilities improve, gen AI’s role in various types of research will expand. However, it is crucial to avoid complacency; verifying sources and data remains essential. While research tasks may shift with AI integration, skilled human researchers will remain indispensable for crafting effective prompts and critically evaluating results. |