A large language model is what powers most text generators, such as ChatGPT, training the model to recognize language patterns from huge amounts of data.
In his book on creative prototyping with genAI, Pennefather explains that LLMs models “often have millions, if not billions, of parameters, allowing them to learn more complex patterns and improve their performance on a wide range of tasks. The size of a model is usually correlated with its capacity to learn; larger models can typically learn more complex representations but require more data and computational resources… it doesn’t understand the text, just like a parrot doesn’t understand what it’s saying, but GPT-4 can analyze patterns and context within the data and generate new text that closely mimics the data it has seen.” [1]
At its core, an LLM is not really thinking through the full context of the text it’s generating: it’s just predicting the next word in a (rather long and complex) sequence.[2] Learning the way an LLM is built will give you the confidence to evaluate tools’ outputs and their training methodology to judge for yourself if the creator avoided copyright infringement when building the model, and if your own input data is protected (for whatever context you are using the tool). The below video from Google Cloud Tech is a great introductory point:
Google Cloud Tech’s 15 min video on specifically on LLMs & GenAI
Google Cloud Tech’s 15 min video on specifically on LLMs & GenAI
<aside> 🗞️ Favorite Learning Resources
[1] Parra Pennefather, xxxi.
[2] David Foster, “Understanding LLMs” (panel at Gen AI Con, London AI Summit, London, June 15, 2023).
<aside> 🧭 Notebook Navigation 🔊 Listen in for an explanation (audio file) 🤖 Working with GenAI tools 💡 World ideation points 🎙️ Notes from interviews with experts ✍🏼 📹 🎭 Sample snippet of potential content in different forms
</aside>
Following this link to the next recommended page:
<aside> Create Your Own Work
</aside>
<aside> Learn about GenAI
</aside>
<aside> GenAI in Action
Artefact 2: Magical Knitted “Armor”
Artefact 3: Folk Song Turned Musical Theatre Concept
</aside>