As a professional programmer my whole life, I’ve spent a good amount of time learning technical concepts. Tech concepts aren’t very difficult for me anymore, but I always like to ground myself in a baseline set of rudimentary concepts before moving into more complex ones.
Typically, the way we understand the world is by making associations: basing some new information on some old information that it is like.
Lately, I’ve been learning about LLMs. LLM stands for “Large Language Model,” and they are the primary thing most people understand AI to be: systems that take inputs in written language and return written results. AI does other things too — like producing images — but the vast majority of AI people are familiar with are LLMs, also known as just “language models.” (Common in the tech world of today are speech-to-text interfaces, where people talk to their LLM and their speech gets translated into words, which the LLM consumes).
In the last post, I talked about the importance of context in writing posts. In this post, I will introduce a handful of concepts and terms and give basic definitions of each one. Some of these concepts will receive a deeper dive in the next few articles.
Agentic vs. Workflow — Many people throw around the word “agentic,” and since it is such a new word, it’s hard to pin down exactly what it means. But Anthropic gives a pretty good definition: Applications that drive LLM calls from a set of predefined paths are called workflows (also called “workflow applications”). You can call them AI workflows or workflow applications built with AI and you wouldn’t be wrong. But the word agentic is reserved for applications where the AI itself (the model) is driving the process. That is, if the model acts on its own, it is agentic. Otherwise, it is workflow.
Prompt Chaining — Prompt chaining is a technique where you decompose a task into sequential subtasks. Each step then builds on the previous step. In other words, let’s say your three-step process will involve asking three LLMs: A, B, and C (arbitrary names). You give the initial prompt to A, then pass its result to B, then pass that result to C. You would do this for different reasons: to structure the process for consistency, you might be using different LLMs for different purposes or with different strengths, and to make each step easier to check when something goes wrong. The idea is: break one big ask into small steps. This often helps catch the error at the exact stage where something went wrong.
Parallelization — If you have a process that can be broken out into several steps, sending each step to different LLMs at the same time is called parallelization. You might use a technique like this when you don’t need a chained response (as in the previous example), but instead the tasks can be processed concurrently (at the same time).
Routing — Routing is the third primary technique that contrasts with chaining & parallelization. Routing is when you dynamically select which LLM to use based on the input. One good example of routing is an AI system built for handling customer service requests. In the examples Anthropic gives, a customer service system first evaluates the request to determine if it is a technical, account, or product-related ticket, then routes the request to a different LLM (or a different prompt) based on what type of request it is.
Chunk — When you feed a big document to an AI system, you rarely feed the whole thing at once. You split it into smaller pieces first. Each piece is a chunk. Think of chunking as a newspaper that has different sections— local, national, sports, entertainment, etc. Smaller pieces are easier to search, store, and pull back later. Chunk size is something builders think about a lot. If the chunk is too big, you lose precision. If the chunk is too small, you lose the surrounding meaning.
Drift — Drift is what happens when a model’s performance slips over time. The model didn’t change. Its training data became outdated. The inputs it sees in production start to look different from what it was built and tested against. Picture a spam filter trained on last year’s spam. The spammers change tactics, and the filter gets worse. That gap is called drift. To catch it, LLM builders measure performance over time.
AI Provider — OpenAI makes ChatGPT, Anthropic makes Claude, Google makes Gemini, etc. The provider is the company making the model.
Model — A model is the trained system. It’s the very large set of rules that takes your input and produces an output. When people say “the model,” they mean the specific AI doing the “thinking.” In Claude’s ecosystem, Haiku, Sonnet, Opus, and Fable are all models. For OpenAI, the models are named things like GPT-4o, GPT-4.1, GPT-5, etc.
Frontier Model — A frontier model means one of the more capable models available at any given moment. In other words, the latest models to be released (usually). “Frontier” points at the leading edge. These are the big, expensive, general-purpose models from the top labs. The frontier keeps moving. Today’s frontier model will be next year’s baseline.
LLM-as-judge — Sometimes you use one LLM to grade the output of another. That’s LLM-as-judge. Instead of paying a human to score a thousand responses, you hand the responses to a model and ask it to score them against your criteria. It’s faster and cheaper than human review. It’s also imperfect, so you spot-check the judge itself now and then.
Eval Harness — An eval harness is the scaffolding you build to test a model. It feeds a set of test cases in, collects what comes back, and scores the results. If you’ve written a test suite for regular code, it’s the same idea pointed at an AI. You run it whenever you change a prompt or swap a model, and you see if things got better or worse.
Offline evals — Offline evals run against a fixed set of test cases, away from real users. You do this before you ship changes to the code. You control the inputs, so you can compare versions cleanly. It’s the AI version of running your tests locally before you push.
Online evals — Online evals measure the model on live traffic, with real users, after you’ve shipped. You’re watching quality in the wild. Real inputs are messier than anything you’d write in a test file, so online evals catch problems your offline set never imagined.
RAG — Retrieval-Augmented Generation is a technique for grounding a model’s answer in your own data. Before the model responds, you fetch relevant information and include it in the prompt. Now the model answers using that material instead of relying on whatever it happened to memorize during training. This is how you get an AI to answer questions about your company’s docs, your product, or anything it wasn’t trained on.
RAG Context — RAG context is the outside information you pull in and hand to the model along with the question. It’s the “augmented” part of Retrieval-Augmented Generation. Without it, the model works from memory alone. With it, the model works from the material you gave it.
Retrieval Context — Retrieval context is the specific set of chunks your system pulled back for one particular query. It’s a narrower, more precise term than RAG context, and it shows up a lot in evals. When you measure whether retrieval is working, the retrieval context is the thing you grade: Did you fetch the right pieces or the wrong ones?
Retrieval observability — Observability means being able to see what your system is actually doing under the hood. Retrieval observability points that lens at the retrieval step. You log which chunks got pulled, how they scored, and what the model saw. When RAG gives a bad answer, this is how you find out whether the retrieval failed or the model did.
We understand new things by connecting them to things we already know. A while back, most of these terms were new to me too. Now it’s just part of my vocabulary. That’s why I like to start here before going deeper. By the time we get to RAG and evals, the words will already be familiar, and we can spend the time on the ideas instead.
This stuff doesn’t need to stay inside the AI labs. It’s showing up in the tools your company buys, the products you use, the way work is starting to get done. When someone tells you their app is “agentic,” you can now ask whether the model is actually driving or whether it’s a workflow with better marketing. When a vendor promises their AI answers questions about your own data, you know to ask how they’re handling retrieval, and how they’d know if it broke. The words are what let you ask the real questions.
I’m writing these for the person who’s technical enough to be curious and busy enough that nobody’s sat them down to explain it plainly. That was me a year ago. If it’s you now, the next few posts go deeper, one concept at a time.