Machine Learning Enlightenment
As somebody who works in technology but is not really a programmer, I’ve been in a weird spot with machine learning. It’s obviously interesting and powerful, but I’ve never tried to do anything with it. Sometimes clients ask about it, but our work projects have never been great fits for it, despite the nice buzzword appeal.
But! I finally dipped my toe into this via RunwayML, a more consumer-friendly wrapper for Machine Learning projects, that bills itself as ML for Creatives. I downloaded it a few months back and just now got around to messing with it. Reader: HOLY CATS, it is fun.
Now, a disclaimer: I am really only dipping a toe here. This is very much ‘kindergarten-level’ machine learning, but still: FUN. Adam was right.
There was real learning curve, and I had to watch several tutorials to even understand what the interface was supposed to do. But within a couple of hours I had trained a GPT-2 model on Thomas Traherne’s Centuries of Meditations, which is a series of short devotional paragraphs from a priest / poet in the 1600s.
I actually find this book deeply meaningful. I have given copies to people, I once started a tumblr and twitter account to repost the meditations one at a time. In short: it’s a personal favorite. But it’s got a very singular style, and is made of these short defined units of text, that it seems ready for parody (and therefore machine generation?). The content of the meditations is also very consistent: Traherne’s big thesis is Extreme Optimism: it is easy and delightful to exist in the world, if you attune yourself to the inherit goodness of the created universe.
So.
I gave the thing the original text, set a super-low training cycle, and it’s just: good enough already, in like 15 minutes. After the model is trained, you give the system a text prompt and a desired length of output. It generates a number of attempts at completion.
There are hundreds, thousands, an infinite amount of new texts that are already indistinguishable from real Traherne. Lots that descend into gibberish, but lots that are… pretty good.
Examples
Here are some examples of real Traherne, from Centuries of Meditations.
Real Traherne:
Is it not easy to conceive the World in your Mind? To think the Heavens fair? The Sun Glorious? The Earth fruitful? The Air Pleasant? The Sea Profitable? And the Giver bountiful? Yet these are the things which it is difficult to retain. For could we always be sensible of their use and value, we should be always delighted with their wealth and glory.
Real Traherne:
Yet further, you never enjoy the world aright, till you so love the beauty of enjoying it, that you are covetous and earnest to persuade others to enjoy it. And so perfectly hate the abominable corruption of men in despising it, that you had rather suffer the flames of Hell than willingly be guilty of their error…
And here are some Machine-generated Traherne. In this attempt, I started it with the phrase ‘I have found’
Machine Traherne:
I have found that in all worlds the Laws of Nature are the best that shall be. Had men been made amiable by the desire of the Soul, they would have been made guilty of covetousness. The Laws of Nature contain the power of Loving, Imitating, Contemplate, and Loving. It giveth pleasure to the Soul that enjoys it, and delighteth in the Creature more than He is.
Machine Traherne:
I have found that the Sea of Joy is the end of all Happiness! Is not that because of its endlessness endless? Well, that the Sun and stars are the gates of Happiness, that your soul may be filled with Delights, Honour and Peace! But it is because God dwelleth in seeing them. And here His love hath made this place aflame: which it hath done with all His love: in making it open into many hosts: and manifesting Himself to innumerable other enjoyments.
Yes, there are some that just get stuck in a kind of probability loop, and repeat a phrase over and over.
I have found by the very things that are in Him that are in Him that are in Him that are in Him that are in Him that are in Him that are in Him that are in Him that are in Him that are in Him [ed. I just cut twenty lines of that] that are in Him that are in Him tha (saith of the Lord this Sunday) being fulfilled
And some that are absurd enough to delight.
I have found that the In-laws are conspiring to seduce you, seduce you with lies and trick you into insatiable love and devouring you: to make you to despise them, and to see you impure and enslave you.
The original text is also so old-fashioned with long complicated grammar, that it’s easy to get lost in a sentence - which makes it a pretty good match to how the GPT-2 sentences spin out. If the grammar falls apart towards the end of a long sentence, my brain immediately draws the conclusion that it was me, the reader, who misunderstood this. I think this charitable stance towards older authors helps the machine-generated version get an easier pass here.
This whole exercise is just so FULL of potential. I want to spin up a ‘lost Traherne manuscript discovered in dusty attic’ scam or something.
But wait
A weird, unexpected follow-on here: some of this machine-generated text was… kind of meaningful? I found there are real moments of poetry and insight in these. Several times I stopped and thought: huh, yes, that’s true.
Maybe that’s not surprising? Look, I fed a machine 500 mini-sermons that I find pretty resonant, and it spit out mini-sermons that I still find pretty resonant. But they were written by a machine! Five minutes ago!
My initial reaction was to say, ‘haha, nothing a computer generates actually means anything’, and of course, that’s true in the sense that there is no authorial intent. The author here didn’t mean anything by this text. GPT-2 simply spits out text that is statistically likely to match the model of the training text.
But… what does it matter to me if a paragraph-length sermon came from a guy who lived 450 years ago, or a computer five minutes ago? There’s a different level of experience that happens simply between ME and the TEXT. If I find some revelation via real or fake Traherne, does there need to be any difference?
And if I accept that proposal, am I just one step from the kind of person who is into horoscopes and fortune cookies and looks for ‘signs’ in the world? I don’t know.
There is presumably a whole field of art criticism or literary study devoted to this question, cf elephant-made paintings or Dada in general. But man! It was really fun to experience this dilemma directly, and self-inflicted at that.
(pictured: the Traherne windows at Hereford Cathedral)