Response 6

Read the following:

• Read Part 1 of The Illustrated GPT-2 (Visualizing Transformer Language Models) Links to an external site.. Jay Alammar. This is an illustrated blog post providing a high-level description of how GPT-2 works.
• Natural Language Processing for Programmers: World Models Links to an external site.. Liza Daly. Blog post arguing for the use of world models to add coherence to generated text.
• SHRDLU: A Game Prototype Inspired by Winograd's Natural Language Understanding Work (Extended Version) Links to an external site.. Santiago Ontañón. Technical paper describing the AI behind the SHRDLU experimental game. 

Submit a 1 to 1.5 page PDF answering the following questions. Include a header in your document for each question. You can work on this in pairs if you wish to facilitate conversation, but if you do, you should both submit the reading response so that I have it in canvas for each of you. Additionally, be sure to include both your names at the top of your response.  

Questions

• From the first blog post on GPT-2, describe an aspect of the architecture you find most confusing. Don't just say "X is confusing." Take the time to describe how you would think this aspect would work, but how it doesn't seem to, or to describe how you think some other aspects of the architecture work, and how this doesn't seem to fit in. 
• From the first blog post on GPT-2, describe an aspect of the architecture you find most interesting. 
• From the second blog post, describe how you think NLP-based text generation and world models could fit together. 
• From the third article, what was convincing? (Where did you see or learn something new with the author's help? Which examples or evidence was most effective?)
• From the third  article, what was *not* convincing? (Where did the authors overlook something? Where would you challenge them?)
• Any other questions or comments?