Learning patterns
During training, a language model adjusts numbers called weights. One important training task is predicting the next piece of text. Repeated adjustments help the model learn patterns in language and relationships between ideas.
Working with your words
Text is divided into tokens: pieces that may be words, parts of words or punctuation. In transformer models, attention helps connect information from different parts of the available context. The model produces an answer a token at a time.
This is why context matters. Your objective, source documents and explanation of a good result give it information to work with. During an ordinary conversation, using that context is different from permanently retraining the model.
What to do with that understanding
Give it the job and the background. Ask it to make assumptions visible. Check important facts and calculations. Where a task needs current information or access to a service, establish that the relevant tools are available.
The title Know Everything, Do Anything describes Nathan’s starting attitude: explore what is possible. The model’s information, tools and access still determine what it can carry out.
Adapted from How AI Actually Works. For the original transformer research, see Vaswani and colleagues, Attention Is All You Need.