The Agent Loop Is a State Machine

If you cannot explain the agent loop as a state machine, you probably do not understand the system well enough to operate it. This prototype is useful because the control flow is explicit: gates, routes, tool execution, and termination conditions are all visible in code.

The bootstrap path tells you what this system actually depends on

main.py does four important things before the user types anything: it boots observability, builds the knowledge base, initializes the tools with the KB handles, and creates a session scope. That startup sequence is not accidental. It means the agent is assembled only after the environment around it is ready.

setup_observability()
_, collection, embed_model = build_kb()
init_tools(collection, embed_model)
scope = SessionScope(role=role, session_allowlist=session_tools)
agent = build_agent(scope)

I like this because it keeps the dependencies explicit. The downside is that startup work happens per session, which is fine for a demo but not for a shared service. Re-ingesting the KB each time is especially expensive once the document set grows.

The other detail worth calling out is that the runtime state is tiny on purpose. The graph is not dragging a giant nested object through every node. It carries just enough to resume the next step.

class AgentState(TypedDict):
    messages: Annotated[list, add_messages]
    pending_tool_call: dict | None
    hitl_reason: str | None
    awaiting_human: bool

The first security decision happens before the graph gets a vote

After the user enters text, the request hits check_injection() before it reaches LangGraph at all. That gives the app a belt-and-suspenders shape: one security check before the graph, and then another layer of safety later through scoped tool binding and approval gates.

sequenceDiagram
    participant User
    participant Main as main.py
    participant Guard as guardrails.py
    participant Graph as agent.py
    participant Tools as tools.py

    User->>Main: Submit helpdesk question
    Main->>Guard: check_injection(text)
    Guard-->>Main: safe / blocked
    Main->>Graph: invoke(state, config)
    Graph->>Graph: LLM reasoning node
    Graph->>Tools: search or action tool
    Tools-->>Graph: tool result
    Graph->>Graph: reason again
    Graph-->>User: final answer
                    

The graph is small, but it is doing real workflow orchestration

agent.py is small, but the structure is solid. There are three real execution nodes:

The routing logic is simple enough that you can inspect it mentally. If the LLM does not ask for a tool, the graph ends. If it asks for a high-stakes tool, the request detours into human approval. Otherwise, the tool runs directly and the graph loops.

def tool_router(state: AgentState) -> Literal["run_tool", "hitl_gate", "end"]:
    last = state["messages"][-1]
    if not isinstance(last, AIMessage) or not last.tool_calls:
        return "end"
    tool_name = last.tool_calls[0]["name"]
    if scope.requires_hitl(tool_name):
        return "hitl_gate"
    return "run_tool"

That function is doing most of the architectural work in the post. It is where the agent stops being a chat loop and starts behaving like a state machine. If I were reviewing this code in a design doc, this is one of the first blocks I would paste.

The loop matters because this is not just retrieval glued to an LLM

The important detail is the loop back from tool execution into the LLM. The graph is not just retrieve once and answer once. It can plan, act, observe the tool result, and then decide what to do next. That is a better fit for real support tasks where the first retrieval step may surface more context, or where a ticket creation action needs a follow-up message to the user.

graph.add_conditional_edges("agent", tool_router, {
    "run_tool": "run_tool",
    "hitl_gate": "hitl_gate",
    "end": END,
})
graph.add_edge("hitl_gate", "run_tool")
graph.add_conditional_edges("run_tool", after_tool_router, {
    "agent": "agent",
    "end": END,
})

I like this more than the usual blog-post pseudocode because it shows exactly where control loops back into reasoning and exactly where the workflow can terminate. That becomes important as soon as you add failure handling and resumability.

This lifecycle is clean locally, but the tradeoffs are obvious

The next step is not more agent logic, it is a real runtime

If I were extending this prototype, I would keep the graph structure and change the surrounding runtime. The first upgrade would be replacing terminal input with an external approval queue and resumable state store. The second would be moving startup concerns like KB ingest and tracing bootstrap into service initialization rather than per-user session setup.

Where this design breaks under multi-user load

The first break is not the graph logic. It is the fact that the runtime assumes a mostly single-user, single-operator world. The graph checkpointing is in-memory, the approval path blocks on stdin, and session startup eagerly rebuilds dependencies that should be shared service resources.

Once you have concurrent users, “simple and explicit” turns into head-of-line blocking. One operator cannot sit in a terminal approving actions for many sessions. One process-local checkpoint store cannot be your recovery story. And one per-session KB bootstrap becomes unnecessary repeated work instead of useful clarity.

That flow only works because retrieval is treated as a first-class step, not an optional add-on. In the next post I’ll go deeper on the knowledge side: how the app builds the KB, scores confidence, and where the PRD asks for a more serious freshness pipeline than the prototype has today.