The rise of generative AI tools like ChatGPT has sparked debate about their impact on creativity and the creation of novel ideas.
A new study by researchers from the University of College London School of Management and the University of Exeter explores the effect of generative models on creative writing. The study examined how access to story ideas generated by large language models (LLMs) affected the creativity of short stories written by humans.
The results are nuanced. While generative AI led to stories that were rated as more creative, engaging and better written, it also resulted in stories that were more similar to each other.
The study focused on short story writing. The participants were asked to write a short, eight-sentence story about a randomly assigned subject.
The researchers measured creativity based on novelty and usefulness.
Novelty measures “the extent to which an idea departs from the status quo or expectations.” On the other hand, usefulness is “the practicality and relevance of an idea.” For short stories, usefulness might translate to the story becoming “a publishable product, such as a book, if developed further.”
The researchers hypothesized that generative AI can affect creative writing in two ways. On the one hand, it may be used as a “springboard for the human mind, providing potential starting points that can result in a ‘tree structure’ of different storylines” or help writers overcome writer’s block.
“If this is the case, we would expect generative AI to lead to more creative written output generated by human writers,” they write.
On the other hand, generative AI may anchor the writer to a specific idea and “restrict the variability of a writer’s own ideas from the start, inhibiting the extent of creative writing.”
“If this is the case, we would expect generative AI to lead to more similar stories and potentially less creative written output generated by human writers,” they write.
To investigate the impact of generative AI on these aspects of creativity, the researchers designed a two-phase online experiment. In the first phase, 293 participants were asked to write a short story about a random topic. The participants were divided into three groups:
Human-only: This group received no assistance or input from generative AI.
Human with one gen AI idea: This group could receive a three-sentence story idea generated by OpenAI’s GPT-4.
Human with five gen AI ideas: This group could request up to five ideas from GPT-4.
The writers then self-evaluated their stories based on novelty, usefulness and various emotional aspects. In the second phase, 600 evaluators assessed the stories on the same criteria without knowing which group the writers belonged to.
The study found that access to generative AI ideas improved both novelty and usefulness.
“We find that having access to generative AI causally increases the average novelty and usefulness… relative to human writers on their own,” the researchers write.
Interestingly, the group with access to five AI-generated ideas showed the most significant improvement. Access to more ideas allows writers to break away from their initial assumptions and explore a wider range of possibilities.
The study also found that writers who scored lower on baseline creativity assessment benefited more from generative AI. These writers showed significant improvement in their stories’ novelty and usefulness when they used AI-generated ideas. The researchers observe that in this case, generative AI acts as an equalizer that removes “any disadvantage or advantage based on the writers’ inherent creativity.”
Accordingly, evaluators found the AI-assisted stories to be more enjoyable, better written and more likely to have plot twists.
“Having access to generative AI ‘professionalizes’ the stories beyond what writers might have otherwise accomplished alone,” the researchers write.
While generative AI enhances individual creativity, the researchers also found that stories based on AI-generated ideas were more similar to each other compared to those written by the control group.
This finding raises concerns about the potential homogenization of creative content if generative AI becomes widely adopted.
“In short, writers in the two generative AI conditions are anchored to some extent on the generative AI idea presented to them,” the researchers write.
If writers rely too heavily on similar sets of prompts and ideas from a limited number of generative AI models, “there is risk of losing collective novelty,” the researchers warn.
“Specifically, if the publishing (and self-publishing) industry were to embrace more generative AI-inspired stories, our findings suggest that the produced stories would become less unique in aggregate and more similar to each other,” they write. “In short, our results suggest that despite the enhancement effect that generative AI had on individual creativity, there may be a cautionary note if generative AI were adopted more widely for creative tasks.”
The findings can be significant as more companies are offering AI-powered writing tools and some organizations are using LLMs to create content en-masse. And the long-term impact will be when the web becomes filled with content that have similar distributions, which will in turn be used to train the next generation of language models.
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
The rise of generative AI tools like ChatGPT has sparked debate about their impact on creativity and the creation of novel ideas.
A new study by researchers from the University of College London School of Management and the University of Exeter explores the effect of generative models on creative writing. The study examined how access to story ideas generated by large language models (LLMs) affected the creativity of short stories written by humans.
The results are nuanced. While generative AI led to stories that were rated as more creative, engaging and better written, it also resulted in stories that were more similar to each other.
Measuring the impact of generative AI on creative writing
The study focused on short story writing. The participants were asked to write a short, eight-sentence story about a randomly assigned subject.
The researchers measured creativity based on novelty and usefulness.
Novelty measures “the extent to which an idea departs from the status quo or expectations.” On the other hand, usefulness is “the practicality and relevance of an idea.” For short stories, usefulness might translate to the story becoming “a publishable product, such as a book, if developed further.”
The researchers hypothesized that generative AI can affect creative writing in two ways. On the one hand, it may be used as a “springboard for the human mind, providing potential starting points that can result in a ‘tree structure’ of different storylines” or help writers overcome writer’s block.
“If this is the case, we would expect generative AI to lead to more creative written output generated by human writers,” they write.
On the other hand, generative AI may anchor the writer to a specific idea and “restrict the variability of a writer’s own ideas from the start, inhibiting the extent of creative writing.”
“If this is the case, we would expect generative AI to lead to more similar stories and potentially less creative written output generated by human writers,” they write.
To investigate the impact of generative AI on these aspects of creativity, the researchers designed a two-phase online experiment. In the first phase, 293 participants were asked to write a short story about a random topic. The participants were divided into three groups:
Human-only: This group received no assistance or input from generative AI.
Human with one gen AI idea: This group could receive a three-sentence story idea generated by OpenAI’s GPT-4.
Human with five gen AI ideas: This group could request up to five ideas from GPT-4.
The writers then self-evaluated their stories based on novelty, usefulness and various emotional aspects. In the second phase, 600 evaluators assessed the stories on the same criteria without knowing which group the writers belonged to.
Generative AI enhances creativity
The study found that access to generative AI ideas improved both novelty and usefulness.
“We find that having access to generative AI causally increases the average novelty and usefulness… relative to human writers on their own,” the researchers write.
Interestingly, the group with access to five AI-generated ideas showed the most significant improvement. Access to more ideas allows writers to break away from their initial assumptions and explore a wider range of possibilities.
The study also found that writers who scored lower on baseline creativity assessment benefited more from generative AI. These writers showed significant improvement in their stories’ novelty and usefulness when they used AI-generated ideas. The researchers observe that in this case, generative AI acts as an equalizer that removes “any disadvantage or advantage based on the writers’ inherent creativity.”
Accordingly, evaluators found the AI-assisted stories to be more enjoyable, better written and more likely to have plot twists.
“Having access to generative AI ‘professionalizes’ the stories beyond what writers might have otherwise accomplished alone,” the researchers write.
Individual creativity versus collective novelty
While generative AI enhances individual creativity, the researchers also found that stories based on AI-generated ideas were more similar to each other compared to those written by the control group.
This finding raises concerns about the potential homogenization of creative content if generative AI becomes widely adopted.
“In short, writers in the two generative AI conditions are anchored to some extent on the generative AI idea presented to them,” the researchers write.
If writers rely too heavily on similar sets of prompts and ideas from a limited number of generative AI models, “there is risk of losing collective novelty,” the researchers warn.
“Specifically, if the publishing (and self-publishing) industry were to embrace more generative AI-inspired stories, our findings suggest that the produced stories would become less unique in aggregate and more similar to each other,” they write. “In short, our results suggest that despite the enhancement effect that generative AI had on individual creativity, there may be a cautionary note if generative AI were adopted more widely for creative tasks.”
The findings can be significant as more companies are offering AI-powered writing tools and some organizations are using LLMs to create content en-masse. And the long-term impact will be when the web becomes filled with content that have similar distributions, which will in turn be used to train the next generation of language models.
Author: Ben Dickson
Source: Venturebeat
Reviewed By: Editorial Team
