Developers use AI to debug their code, test reports, and even create applications But they face bias and errors, according to Applause

Although 75% of people think chatbots are getting better at handling toxic or inaccurate responses, 50% of them have still experienced bias and 38% have seen inaccuracies, according to a new report.

The survey of more than 6,300 consumers, software developers and digital QA testing professionals, on behalf of digital QA and testing company Applause, reveals that 91% of respondents have used chatbots to perform searches, and 33% of them use them daily to carry out searches.

Since its release, it’s no surprise that ChatGPT has been the most used (91%), followed by Gemini (63%) and Microsoft Copilot (55%). Other chatbots used by less than a third of users: Grok (32%), Pi (29%), Perplexity (24%), Claude (23%) and Poe (21%).

Interestingly, 38% of respondents indicate that they use different chatbots depending on the specific task. 27% of people surveyed also say they have replaced one chatbot with another because of its performance.

Of the 1,539 respondents who use GenAI for software development and testing, the most common applications are writing or debugging code (51%), test reports (48%), building test cases (46%) and application creation (42%). GitHub Copilot is the most popular tool for helping with coding (41% of respondents), followed by OpenAI Codex (24% of respondents).

Privacy, however, remains a concern: 89% of respondents fear providing private information to chatbots, and 11% say they would never provide private information.

“The survey clearly shows that consumers are eager to use GenAI chatbots, and some have even integrated them into their daily lives for tasks such as research. Chatbots are getting better at handling toxicity, bias and inaccuracy, but concerns remain. It’s no surprise that it’s common to switch between chatbots to accomplish different tasks, as multimodal capabilities are now part of the table.“says Chris Sheehan, vice president of strategic accounts and AI at Applause. “To gain adoption, chatbots must continue to train models on quality data in specific domains and conduct extensive testing across a diverse user base to reduce toxicity and inaccuracy.“

The detailed results of the survey conducted by Applause are presented below:

When Applause surveyed its global community about generative AI last year, usage was increasing despite concerns about bias. Again this year, Applause surveyed the community to learn how consumers are using different generative AI tools, including chatbots, and how usage and user experiences have evolved as the technology has gained popularity. . This year, 6,361 consumers, software developers and QA professionals participated in the survey, revealing that while user satisfaction is increasing, there is still room for improvement.

The answers are becoming more and more useful

Last year, 7% of 2,110 people surveyed said the responses they received from GenAI chatbots were always relevant and appropriate, while 50% said this was the case most of the time. This year, more than twice as many people participated in the survey and they were more likely to find the chatbots’ responses helpful. Of 4,229 responses, 19% indicated that the GenAI tools they use understand their questions and provide helpful answers every time, and 58.5% said this is the case most of the time.

With over 2,900 respondents indicating that they use at least one GenAI chatbot daily, there is a lot of data to evaluate. Additionally, 37.5% of respondents said they use different chatbots for different tasks, while 26.5% said they replaced one service with another, usually due to performance issues.

The most common UX GenAI mistakes according to 4,174 people surveyed are:

• gave a general response that did not provide enough detail: 17.5%
• misunderstood my prompt: 16.8%
• gave a convincing but slightly incorrect answer: 10.7%
• gave obviously incorrect answers: 10.3%

Use cases are diversifying

For most users, chatbots are replacing existing search engines and tools – 91% of respondents have used chatbots to search, and 33% of them use GenAI to search daily. For 81% of survey participants, chatbots have replaced search engines for queries; 32% of them use chatbots for research daily. Other popular use cases include language translation, creative writing, and writing emails, proposals, or other similar business communications.

It’s no surprise that GenAI has a myriad of use cases in the software development and testing process. When respondents who indicated that they used chatbots to write or debug code, build test cases, or for test reports were asked to specify how they used chatbots for testing, 1,532 IT professionals QA managers reported that they most often used chatbots for generating test cases (19%), generating text for test data (17%), and creating test reports (16%). . GitHub Copilot and OpenAI’s Codex are the most popular tools.

Despite biases, hallucinations and other flaws, users are optimistic about the technology

This year, 38% of people surveyed said they had been victims of hallucinations, 50% saw content they considered biased and 19% saw content they considered offensive, i.e. 1 to 2 percentage points more than last year’s results. Although a greater proportion of users reported experiencing problematic responses this year, 75% of the 4,245 people surveyed said they thought chatbots were getting better at handling toxic or inaccurate responses.

How satisfied are users with GenAI experiences? More than a third (36%) say they are extremely satisfied, and 53.6% say they are quite satisfied. When asked about potential features they would like to see in generative AI chatbots, survey respondents cited better source attribution, more localized responses, support for more languages ​​and further personalization.

Despite improvements to the technology, some users are still not convinced of the value of GenAI: 1,001 survey participants indicated that they had never used the technology or had only tried it one or two times. When asked why they had not adopted GenAI, 28% of respondents said they did not want to use it. That’s good news for existing search engines and translation software for now, but near-daily advances in AI could attract more users in the coming year.

Source : “Survey Examines User Perceptions About Generative AI” (Applause)

And you ?

What is your opinion on the subject?
Do you find that the conclusions of this investigation carried out by Applause are credible or relevant?

See as well :

92% of developers would use artificial intelligence tools, according to a GitHub developer survey

OpenAI’s Codex AI is making its programming debut and can already code in around ten languages. Will AI soon be able to relieve developers of certain tasks, or even replace them?

Using GitHub Copilot AI Assistant for Programming Leads to Lower Overall Code Quality and Significant Amount of Redundant Code, Study Finds