How AI chatbots actually work — a plain-language explainer for teachers

You don’t need to understand engines to drive a car, but knowing roughly how brakes work makes you a safer driver. The same is true of AI chat tools. A little understanding explains almost every surprising thing they do — and tells you exactly where to be careful.

It’s a prediction machine for text

Tools like ChatGPT, Gemini, Claude and Copilot are built on large language models. A language model is trained on a huge amount of text — books, websites, articles, code — and learns one core skill: given some text, predict what text is likely to come next.

That sounds simple, but at enormous scale it becomes surprisingly capable. To predict the next words of a good lesson plan, the model has to have picked up patterns about how lessons are structured, how fourth graders are usually described, and what a learning objective sounds like. So it can produce a lesson plan that reads as if a teacher wrote it.

After that initial training, companies do further training so the model follows instructions, holds a conversation, and declines some harmful requests. That’s what turns a text predictor into a helpful assistant.

Why it can be confidently wrong

Here’s the key idea: the model is optimized to produce text that is plausible, not text that is checked against reality. Most of the time plausible and true line up. Sometimes they don’t.

That’s why AI tools can:

• invent a quotation that sounds exactly like something a historical figure would say,
• cite a research paper with a realistic title that doesn’t exist, or
• give an answer key where one answer is subtly off.

People call this “hallucination.” It isn’t lying, and it isn’t a rare glitch — it’s a side effect of how the system works. Newer models make these mistakes less often, and tools that search the web can point to real sources, but the habit of checking still matters. See our answer-key checklist.

What it “knows” — and when

A model learns from text collected up to a certain date, called its training cutoff. It doesn’t automatically know about anything after that. Some tools add a web search step so they can look up current information; when they do, they usually show the sources they used. If a tool doesn’t show sources, treat anything time-sensitive — policies, prices, recent events — as unverified.

What it remembers about you

Inside one conversation, the tool can “see” everything you’ve typed so far (up to a large but finite limit, sometimes called the context window). That’s why giving it your objective, grade and materials at the start produces much better results.

Between conversations, it depends on the product and your settings. Some tools have optional memory features or let you save custom instructions. And separately from memory, the company may store your conversations and — depending on the product and settings — use them to improve future models. That’s the main reason we say: keep student information out, and prefer education versions that commit not to train on your data.

Why the same question gets different answers

AI responses include some randomness by design, so asking the same thing twice usually gives two different drafts. That’s useful — ask for several versions and pick the best — but it also means one good answer doesn’t guarantee the next will be good.

Why your wording matters so much

Because the model is continuing from what you give it, the details in your request steer everything that follows. Compare:

• “Make a fractions worksheet.”
• “Make a one-page worksheet for grade 3 on comparing fractions with the same denominator, using pictures of pizzas and chocolate bars, 6 questions from easy to harder, with a separate answer key.”

The second gives the model far more to work with. This is all “prompting” really is: clear instructions plus context. Our prompt builder does it from a short form.

What this means in your classroom

• Use AI for drafts, variations and ideas — the things it’s naturally good at.
• Verify facts, numbers, sources and answer keys — the things it can get wrong.
• Give it context, because it knows nothing about your class unless you say so.
• Protect student data, because what you type may be stored.
• Teach students the same ideas. Understanding that AI predicts plausible text is one of the most useful pieces of AI literacy a student can have. See teaching AI literacy.