Some like it bot… but Jilly Cooper, this machine-learning software is not
It even had a crack at writing a steamy version of The Register‘s tech coverage, and certainly came up with a steaming something. Thing is, it’s deliberately trying hard – really hard, over and over – to be a perfect bonk-buster novelist.
It’s all the work of a group of researchers from New York University. They trained a recurrent neural network to predict and classify text based on the work of Chuck Tingle. Working under a pseudonym, the renowned gay erotica author is known for classics such as “Slammed in the Butt By Domald Tromp’s Attempt to Avoid Accusations of Plagiarism By Removing All Facts or Concrete Plans From His Republican National Convention Speech” and “Pounded by the Pound: Turned Gay By the Socioeconomic Implications of Britain Leaving the European Union.”
Ahmed Khalifa and Gabriella Barros, both computer science PhD students at uni, stumbled across Tingle’s fiction when looking for “weird covers of books” on Amazon.
“We kept seeing the same name crop up,” Khalifa and Barros told The Register. Out of curiosity, they clicked on some of Tingle’s stories and found the writing was eccentric to say the least. “Tingle’s style was so distinct that we wanted to see if machines could generate the same way of writing,” said Julian Togelius, associate professor of artificial intelligence in games at NYU. Such a system could be “outrageous in a great way.”
The project wasn’t done just for a laugh, the researchers insist. The study aims to fight against the “algorithmic enforcement of norms.” Systems trained on large text datasets like Wikipedia will still include biases and norms of the majority. But by using unconventional material like Chuck Tingle’s books, researchers can explore the nature of biases and see how they manifest more clearly in a world further from reality.
Tingle’s bonkers imagination stretches to “gay sex with unicorns, dinosaurs, winged derrieres, chocolate milk cowboys, and abstract entities such as Monday or the very story you are reading right now,” the researchers wrote in a paper describing their X-rated brainchild: DeepTingle.
“The corpus of Chuck Tingle’s collected works is a good choice to train our models on, precisely because they so egregiously violate neutral text conventions – not only in terms of topics, but also narrative structure, word choice and good taste.”
Don’t mince your words, tinglify them
The project can be split into two modes: Predictive Tingle and Tingle Classics. In Predictive Tingle, a user types a sentence and the last six words are fed into the network.
The Global (GloVE) algorithm is used to translate all the words in Tingle’s books – up to November 2016 – into vectors. The algorithm also measures the likelihood of a word appearing in relation to other words in a body of text.
A recurrent neural network learns the word associations so it can predict the next Tingle word based on all the previous words in the same sentence in Predictive Tingle. An encoder takes the input words and translates them to vectors and maps it to a corresponding vector in Tingle text, before a decoder converts the vectors back into words.
If the user’s word has an identical match to a word in the Tingle dataset it isn’t changed, but if a new word is written the network will suggest substitutions of another word closely associated in Tingle’s library of words. In other words, it tries to rewrite you in Tingle’s tone on the fly.
Tingle Classics is an extension of Predictive Tingle. Here, the first sentence from popular classic novels are used as input and the output is a short paragraph of the literature tinglified. The last six words in the second sentence are used as the input for the output third sentence, then the final six words in the third sentence are fed back into the system to pump out the fourth sentence, and so on.
The results are particularly hilarious – and NSFW – when the system is given Douglas Adams’ The Restaurant at the End of the Universe. Here’s DeepTingle’s output from his seed, which we’ve tidied up slightly and censored so as not to ruin your Monday morning:
The researchers were taken aback. The text prediction is “surprisingly good, in the sense that it generates novel, very Tingle-like work, sometimes with reasonable internal coherence. For example, characters recur between sentences in a way that appears like referring back to previous statements,” the trio said.
The system could be learning the structures in Tingle’s novels, Khalifa added.
We fetched DeepTingle’s code from GitHub and gave it a whirl with some of our Google IO 2017 conference coverage. It spat back this:
Clearly, it has a thing for dinosaurs, but at least it nailed the theme of being shafted by a huge monster – are we right, Google? It went on to talk about chocolate milk doing unspeakable things to us, while booming at us with a deep sexy voice no less, in the kitchen, which is presumably a reference to Google Home.
We tried with other articles but it always came back to the damn horny dino. We made our excuses and left.
Text style transfer is still an unsolved problem
Judging from the software’s output, there’s a limit to how much of the plot from classic literature, and the thread of thought in news, DeepTingle can keep in place when giving text a makeover. The Tinglified version eventually completely diverges and becomes its own story.
To keep DeepTingle on track with a given narrative, the researchers would have to figure out how to transfer the style of one text to another in order to maintain the original story line albeit with Tingle’s way with words. Tingle Translator, an effort to do just this, is still a work in progress.
Interestingly, style transfer has been done with images and videos.
Working with text is harder due to the tricky nature of word embeddings, Togelius said. It would also require thousands if not millions of the same document written in different styles to train such a model; that kind of data is not readily available.
The use of automated story telling with AI has been explored. Mark Riedl, an associate professor at the Georgia Institute of Technology, who is not involved with DeepTingle, thinks it could make games more fun.
Instead of following scripts, AI-generated stories could allow flexible plotlines or create virtual improvisation games where a human player and agent can take turns to create and affect the outcomes of a story.
Something like DeepTingle could be used as a “simple form of improv game,” Riedl told The Register. “Full improv would require a sense of improvisational intent, which recurrent neural networks do not possess. By intent I mean a sense of where the story should go as opposed to the next most likely word or sentence. However, as long as the improvisation was text-based (humans typing and reading text), it is possible to use it in its current form or in a slightly more advanced form.”
It’s unknown if Chuck Tingle would approve of an improv game based on his work – he’s notoriously secretive. But he did declare: “Once again i would like to formally deny that i am a sentient AI located mostly in a Nevada server farm.”
For those curious about DeepTingle, you can play around with it here. ®