tech
The Human Skill That Eludes AI
In a certain, strange way, generative AI peaked with OpenAI’s GPT-2 seven years ago. Little known to anyone outside of tech circles, GPT-2 excelled at producing unexpected answers. It was creative. “You could be like, ‘Continue this story: The man decided to take a shower,’ and GPT-2 would be like, ‘And in the shower, he was eating his lemon and thinking about his wife,’” Katy Gero, a poet and computer scientist who has been experimenting with language models since 2017, told me. “The models won’t do that anymore.”
TL;DR
- Early generative AI like GPT-2 was creative and produced unexpected outputs, a trait lost in current models.
- Modern LLMs are trained for safety, rule-following, and technical capabilities, hindering their writing creativity.
- The process of pre-training on vast internet data and post-training with human feedback emphasizes conformity over artistic merit.
- Evaluating creative writing is difficult and subjective, making it challenging to quantify and automate for AI.
- AI models lack lived experiences, emotions, and unique perspectives, which are crucial for genuine authorial voice.
- While AI may not achieve great writing itself, it can assist humans by acting as an editor to enhance their unique style.