Understanding LLMs
An ongoing series about how language models work, the technology behind them and where intuitive language can give us the wrong idea.
Chapter 01
How does an LLM work?
Training, text generation and the path from a model to an agent.
Chapter 02
Why do we talk about AI as if it were human?
Why technical language can sound more human than the processes behind it.
Chapter 03
Why is a GPU faster than a CPU?
Many similar calculations at once, and the limits of that advantage.
Chapter 04
Why does an LLM need so much memory?
Weights, working data and the KV cache that grows with the text.
Chapter 05
What is quantisation?
How approximate weights save memory and what can change as a result.
Chapter 06
What does model size mean?
What the parameter count tells us – and why it is not a quality score.
Chapter 07
What is a context window?
How a conversation grows and what compaction and session handoff aim to preserve.
Chapter 08
Is the LLM cold? Temperature and other controls for generating text
How temperature, top-p, top-k and min-p shape the choice of the next continuation.
Chapter 09
Why does an LLM hallucinate?
How plausible false claims arise and what the word “hallucination” obscures.
Chapter 10
What are embeddings and RAG?
Finding relevant passages and supplying them as context for an answer.
Chapter 11
Prompting, RAG or fine-tuning?
Coming soon
Chapter 12
What is a mixture-of-experts model?
Coming soon
Chapter 13
Can LLM benchmarks be trusted?
Coming soon
Chapter 14
Local or via an API?
Coming soon
1 / 14
My two cents on the OpenAI/Hugging Face incident
An examination of the attack on Hugging Face: the permissions granted, missing technical safeguards, inadequate oversight and the responsibility of those operating the system.