Last updated on April 30, 2026
Image by Shubham Dhage on Unsplash
Imagine this: you open ChatGPT and type, “…help me understand thermodynamics,” and hit enter. Moments later, you have a nice review of work, energy, and entropy. But AI didn’t actually read your sentence. Well, not in the way you’re reading this one.
So how does it understand you at all? To understand, what is important to realize is that to an AI, every word is a list of numbers.
Vectors
For humans, the differences between the words “entropy” and “enthalpy” are pretty easy to notice. One is about disorder, and the other is about heat content. However, for an LLM to discern the difference, each word must first be assigned to a list of numbers, known as a vector. For the sake of simplicity, we will use 5 numbers for vectors. Note that in real systems, vectors can have up to billions of numbers.
For instance, when an AI encounters the word “entropy”, it might generate a vector like [0.91, 0.42, 0.88, 0.74, 0.83] and for “enthalpy”, a vector like [0.89, 0.95, 0.21, 0.78, 0.71]. These vector elements could represent categories like thermodynamics, heat, disorder, chemistry context, and physics, respectively, as shown. If you look at chemistry and thermodynamics, the numbers are both really close to 1, meaning a strong correlation related to those categories. Conversely, “entropy” has a low value for heat (0.42), and “enthalpy” has a low value for disorder (0.21).
Training The Model
Now, during training, a model looks at billions of words, and their vectors and groups create categories and group words together depending on their category. As it does this, it picks up on patterns. Dogs and cats become similar as they’re both house pets, mammals, but also contrast in that they’re different species.
Words slowly get pulled together, drift farther apart, and as this occurs, the AI is starting to understand the basis of human language. This can actually be visualized (well, a simpler version of it)!
Image by Altexsoft
From the graph above, you can see that this LLM has learned that certain words can go together and other words not so much.
Mathematically, this is called the cosine similarity. Simply, this is a measurement of how much two vectors point in the same direction. A score of 1 means identical meaning, and 0 means completely unrelated.
So what does this mean for your thermodynamics question?
When you hit enter on that prompt asking for help on thermodynamics, the algorithm took your entire sentence and used cosine similarity to determine a vector. Then, the LLM will look through other vectors and find words such as entropy, enthalpy, heat, work, and chemistry to be pretty related.
This is why AI can handle typos, vague prompts, and weird phrasing so well. It’s not because you said those words, but because it matched your meaning to whatever makes the closest sense.
Underneath all of the text AI generates and the beautiful explanation, it’s really just super fast, math happening.
Be First to Comment