Yesterday, I was able to share on GitHub a working GPT-2 model that ingested a 20 year corpus of my writing. Using the OpenAI 355 million parameter version of GPT-2 did produce some outputs that looked reasonably like something that I would write. Normally training would take the longest amount of time, but in this case the entire Jupyter notebook takes less than an hour to run to create a reasonable text creation engine. Part of what I’m going to work on today is cleaning up the input corpus file to make it better. A really good text encoding process might help move the needle from reasonable to mostly accurate. The idea that within an hour of training on my writing corpus the GPT-2 model could reasonably approximate my writing style is very surprising. Seriously, I was super surprised when it worked the first time yesterday. A victory lap was taken around the house. That pretty much means shuffling around the house arms in the air with victory at the forefront of my thoughts. 

Now that the experience of having victory at the forefront of my thoughts is fresh in my mind perhaps today will be the day that the next iteration of the Jupyter notebook will be shared on GitHub. Being able to share working notebooks in realtime is really satisfying. Part of my journey into natural language processing and machine learning models has always been about being able to reproduce the actions being taken. I do not like to write or produce one off code that requires hand holding to run successfully. Working with TensorFlow has to be about sharing and doing things that are repeatable. Over the last few years I have taken a ton of courses on how to use and work with TensorFlow. People have shared so much content online and GitHub is overflowing with different projects that people have undertaken. A lot of that is so very specifically targeted that generalizing it to a pattern others can repeat is a challenge. All of my efforts begin with the idea of sharing the pattern in the end. That helps me ensure that every step gets covered along the way and people can reasonably work through my examples. 

Interrupted. Breakfast.