[Generative AI] From LLM to Loop — Why AI Engineering Evolved the Way It Did

Introduction

The vocabulary we use for working with AI has changed quite a few times over the past few years. First everyone was talking about “writing better prompts,” then “context is what matters,” and now terms like “harness” and “loop” have started showing up.

These aren’t just buzzwords swapped in and out with the trends. Each one emerged to push past the limits of the stage before it. Context came along because prompts alone hit a ceiling. Harness came along because context alone wasn’t enough either. In other words, every time the name changed, there was a specific problem hiding behind it — one that the previous approach couldn’t solve.

This post follows the arc of LLM → Prompt → Context → Harness → Loop, asking for each stage: what is it, why did it emerge, and what problem eventually pushed things toward the next stage? Think of it less as a static taxonomy and more as a story of evolution.

If this post focuses on the evolutionary flow, the companion piece 4 Paradigms of AI Engineering puts the same concepts side by side as stacked layers, focusing on structure. Reading both together gives you the full picture.

LLM — Where It All Begins

The story starts with the large language model itself.

At its core, an LLM is a model that predicts the most plausible next token given the text so far. Trained on vast amounts of text, this predictor can handle tasks that previously each required a separate system — translation, summarization, code generation — using a single model driven by natural language instructions.

This is where the first fundamental question arose:

“How do you give instructions to this powerful predictor to get the output you actually want?”

Open-source smaller models took one route: fine-tuning on additional data. Large commercial models took another: injecting the relevant knowledge alongside the API call. But for most practitioners, touching the model itself wasn’t realistic. The only lever they had was the input text passed to the model.

That’s exactly where the first engineering discipline emerged.

Prompt Engineering — How Do You Say It

Prompt Engineering is the craft of how to write the instruction (prompt) you hand to a model. Since the model’s weights are fixed, the thinking goes: let’s get the most out of the one thing we can actually control — the input text.

Anyone who has used an LLM has felt how much the phrasing matters. “Review my code” gets you a generic response; “Review this from a senior backend engineer’s perspective, focusing on security issues” gets you something far more specific. As those experiences accumulated, a handful of structured techniques emerged:

• Few-shot: Provide a few examples of the desired output to establish a pattern.
• Chain-of-Thought: Ask the model to “think step by step” to elicit a reasoning process.
• Role Prompting: Assign a persona — “You are a backend engineer with 10 years of experience.”
• Output format specification: Lock in the shape of the output upfront — “Respond in JSON.”

For example, if you want to narrow the scope of an answer, you can be explicit about what’s in and what’s out:

Analyze only the What and Why of the requirements.
Do not discuss How (technical implementation).

The catch — a prompt is ultimately a request

Prompt engineering has a fundamental weakness: no matter how carefully you craft it, it’s a request, not a command.

Writing "IMPORTANT: Never do this" in all caps doesn’t guarantee the model will comply. Change a single word or reorder a sentence and the output can shift entirely. And crucially, no matter how good your prompt is, you can’t make the model actually open a file, check a live API response, or know the current state of your project.

“How you ask” and “what information you give it” are two different dimensions of the problem. That limit called the next stage into being.

Context Engineering — What Do You Show It

The focus shifts from “how to say it” to “what to show alongside it.” That’s Context Engineering.

Think of it this way: if a prompt is the question on the exam, context is the reference material you spread out beside it. The same question answered with the relevant textbook open looks very different from answering cold. Project background, related code, file structure, history of what’s been done so far — context engineering is the discipline of selecting and feeding in the right material for the model to make good decisions.

During this period, models began evolving beyond simple question-answering into agents that could use tools (tool calling), maintain memory, and determine their own next actions (ReAct). Since good decisions require good context, context engineering naturally became the core discipline of the agent era.

The core skill: curating what goes in

The context window isn’t unlimited. Too much information buries what actually matters and drives up cost. So instead of dumping an entire API response, you extract only the fields you need. Instead of loading one massive instruction file, you load only the modules relevant to the current situation.

# Feeding the entire response pulls in unnecessary data
curl -s https://api.example.com/comments | jq '.[].body'

# Selecting only what you need is far more efficient
curl -s https://api.example.com/comments \
  | jq '.[] | select(.tag == "review") | .body'

The catch — good information doesn’t guarantee good execution

No matter how well you design the context, as tasks get longer and more complex, success rates drop sharply. The characteristic failure modes look like this:

• Context Rot: As a conversation grows and exceeds the context window, earlier content gets forgotten.
• Session breaks: When a session ends, memory is gone — past work gets contradicted or the thread is lost.
• Rule fragility: Even if you write rules into the context, those rules are still ultimately just “prompts” — they don’t get followed consistently.

In the end, context engineering optimizes the material for judgment but can’t enforce that judgment or safely wrap the execution that follows. Giving someone good ingredients doesn’t guarantee the dish comes out right.

Harness Engineering — What Constraints Do You Put Around It

This is where Harness Engineering enters. A harness is the equipment used to safely control a horse’s power. The idea here is to build a structure that wraps around the model and steers it in the right direction. The concept is often summarized like this:

Agent = Model + Harness

The agent’s real-world performance depends not only on the model’s capability but equally on the design of the harness around it. The term gained visibility when Anthropic mentioned “effective harness” in late 2025, then spread widely after Mitchell Hashimoto discussed it in his AI adoption journey in early 2026.

Hashimoto defines harness engineering as:

“Every time the agent makes a mistake, engineering a solution so that mistake can never happen again.”

The core shift is from request to enforcement. Everything up through context engineering was “I’d like you to do it this way.” The harness is closer to “stepping outside this boundary is simply not possible.” In practice, that looks something like this:

Principle of least privilege — give the agent only the capabilities it needs and nothing more. Allow reads but block deletes and admin commands. Even if it makes a mistake, the blast radius is bounded from the start.

{
  "permissions": {
    "allow": ["Bash(ls:*)", "Bash(cat:*)", "Read(/src/**)"],
    "deny": ["Bash(rm:*)", "Bash(sudo:*)"]
  }
}

Automated verification (Hooks) — instead of prompting the agent to “please run the tests,” structurally enforce lint and tests at commit time. If they don’t pass, the process doesn’t advance.

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash(git commit:*)",
        "hooks": [{ "type": "command", "command": "npm run lint" }]
      }
    ]
  }
}

Providing verification tools — a point Hashimoto emphasizes: give the agent the means to check its own work. When an agent has a way to verify its output, it will generally catch its own mistakes and prevent regressions. You can already see the seed of a feedback loop here.

Documents like CLAUDE.md / AGENTS.md that explicitly record state outside the session, and approval steps before dangerous operations, are also part of the harness.

The catch — even a secure room needs someone to open the door

A harness builds a solid “room” for the agent to work in. But starting work in that room still requires a human. A person enters the prompt, the agent completes one cycle, and then the person reviews the output and enters the next prompt.

The longer and more repetitive the work, the more visible this bottleneck becomes — “a human has to type a prompt every single time.” The room is safe and the worker is capable, but the engine keeping things moving is still tied to someone’s keyboard. Loop Engineering is aimed squarely at this final bottleneck.

Loop Engineering — Who Enters the Prompt

Loop Engineering flips the assumption one more time.

Instead of “entering prompts well,” design the mechanism that enters prompts on your behalf.

The key idea is automating the back-and-forth between human and agent. The agent observes its current state → decides the next action → executes it → evaluates the result → decides whether to continue or stop — and this whole cycle runs without human intervention. The human’s role shifts from directing each step to defining what needs to be achieved (designing the loop).

You can think of it as taking the “self-verify → self-correct” feedback loop that appeared in harness engineering and promoting it to the central axis of the entire system. A loop in production typically involves elements like:

• Triggers (Automation): The loop starts itself on a schedule or in response to an event.
• Worktrees: Isolated environments so multiple tasks can run in parallel without colliding.
• Skills / Plugins: Procedure documents and external tool integrations the agent can reference.
• Sub-agents: Separating roles — one agent builds, another verifies.
• State / Memory: Persistent tracking of task state across restarts.

The catch — this isn’t a finished methodology yet

Loop Engineering is a work in progress. “Real, but not yet a complete methodology” is an accurate description. The more a human steps back from the loop, the greater the token cost — and the heavier the responsibility to verify that the output is actually correct. Most importantly, a new challenge emerges: comprehension debt, the accumulating cost of code that no human directly wrote or fully understands.

So humans haven’t become unnecessary — the role has shifted from executing each step to designing and overseeing the loop as a whole. And if the pattern so far holds, the next stage will probably be determined by where these new problems — cost and comprehension debt — hit their own wall.

The Big Picture

Putting the whole arc together in one place:

Two axes run through everything.

First, from request to enforcement. A prompt was a politely worded hope. A harness is a constraint you can’t escape.

Second, from “human intervenes every step” to “human only designs.” In the prompt era, a human typed every single input. In the loop era, a human designs the system and steps back.

One thing worth being clear on: later stages don’t retire earlier ones. Inside a loop, you still need good prompts, good context, and a solid harness. Later engineering disciplines are better understood as wrapping and amplifying what came before. That “stacked layers” structure is explored in more depth in the 4 Paradigms of AI Engineering post.

Wrapping Up

The names we use for working with AI haven’t changed with the trends — they’ve evolved in a chain, each stage’s limits giving birth to the next.

• LLM: A fixed predictor. The only thing we can touch is the input.
• Prompt: Refine the input. But it’s still just a request.
• Context: Give it information to work with. But you can’t enforce anything.
• Harness: Enforce through constraints. But humans still start the work.
• Loop: Automate the repetition too. But new debt accumulates in return.

Right now we’re at the point where Harness is maturing and Loop is just entering the conversation. Whatever name comes next — after this post — will probably be aimed at pushing past the limits Loop runs into. Every time you encounter a new term, asking “what problem from the previous stage is this trying to solve?” will help you keep your bearings in a sea of buzzwords.

References

• My AI Adoption Journey — Mitchell Hashimoto — Definition and background of harness engineering
• Loop Engineering — AI-Driven Lab (note.com) — The arc from prompt to loop, and what makes up a loop
• 4 Paradigms of AI Engineering (this blog) — Comparing each concept as a layered structure

Github Sponsor Buy me a coffee

Was my blog helpful? Please leave a comment at the bottom. it will be a great help to me!

App promotion

You can use the applications that are created by this blog writer Deku.
Deku created the applications with Flutter.

If you have interested, please try to download them for free.

word learning app for kids

BlaBoo

Korean TOPIK vocabulary app

TOPIK words

Japanese vocabulary app

Japanese words

Chinese HSK vocabulary app

HSK words