R14: Better Language Models and Their Implications

Happy Valentines Day -- just today OpenAI released a technical system and research article that are causing a major stir Links to an external site. in the machine learning research community. As machine learning is one of the most interesting ways of programming without coding, and the field is evolving rapidly while you are in this class, I wanted to create a reading assignment about this very hot material. As of the moment I write this, the new result is only about 6 hours old.

Read through this blog post: https://blog.openai.com/better-language-models/ Links to an external site.

In the gray boxes within the post that show text generated by the system or its prediction on challenge problems, you can click the left and right arrows at the side of the page (or the strip of horizontal lines at the top of the box) to scroll through several different output samples. Previously, the best available language models were barely even able to assemble a single coherent sentence on a given topic. The evidence that this system can stay on topic, with readable phrasing, for paragraphs at a time is... well, it's extremely impressive.

This new result for text generation comes just a few months after an analogous published by Nvidia in image generation. Optionally, please check out this equally impressive video of the visual results: A Style-Based Generator Architecture for Generative Adversarial Network Links to an external site.

The nature of how we program machines is changing very rapidly in 2019. Increasingly, the programs we use to run our real-world systems will be partially coded by humans and partially learned from examples (using learning systems that were themselves only partially coded by humans). Entire generations of programmers who only understand the coding side of programming are being blindsided now by something they didn't anticipate, but you are part of the first generation who gets to see this stuff in your Introduction to Programming class.