A Prototype That Keeps the Hard Parts In

Most agent demos get easier by deleting the parts that make production systems hard. This prototype does the opposite. It keeps retrieval quality, tool scoping, approval gates, auditability, and fallback behavior in the design from day one, which is exactly why it is worth looking at.

I like that framing because it forces the right conversation early. If an agent can create tickets, escalate issues, and touch company workflows, architecture matters more than prompt polish. The interesting part of this repo is how it translates that bigger enterprise story into a small Python prototype.

This PRD is really asking for a control plane, not a chatbot

The PRD lays out four big ideas that drive the rest of the design:

That is a sensible enterprise baseline. It also means the architecture has to carry a lot more than a simple user -> LLM -> answer loop.

The code already commits to real architectural boundaries

The current repo is still a prototype, but it already has the right bones:

The useful part is that those pieces are actually wired together in a pretty disciplined order. The app does not construct a giant agent object and hope the dependencies sort themselves out later. The bootstrap path in main.py makes the runtime assembly visible.

def run_session(role: str = "analyst", session_tools: set | None = None):
    setup_observability()

    _, collection, embed_model = build_kb()
    init_tools(collection, embed_model)

    scope = SessionScope(role=role, session_allowlist=session_tools)
    agent = build_agent(scope)
    config = {"configurable": {"thread_id": "session-001"}}

That matters because it shows the real boundaries of the prototype. Retrieval and tool registration are process-local. Scope is computed before the graph is built. Observability is turned on before the first turn. Those are implementation choices, not just bullets in a PRD.

The PRD also includes a much more detailed architecture view. I wanted to keep that diagram intact here because it shows the actual control surfaces, not the simplified “agent plus RAG” version people usually stop at.

graph TD
    %% Node Definitions
    User(["👤 Employee / user input"])
    JWT["🔑 JWT Token
(Okta/Azure AD)"]
    
    subgraph KFP_Sub ["🔄 KB FRESHNESS PIPELINE"]
        Note_1["Freshness: Scheduled job re-ingests KB docs.
Uses Chunk Versioning v1.x -> v2.x for rollbacks."]
        Source[(Source Docs)] --> Ingest[Scheduled Ingest]
        Ingest --> Embed[Chunk & Embed]
        Embed --> Ver_Store["Versioned Store"]
    end
  
    Ver_Store -.-> SD_Search

    subgraph PIG_Sub ["🛡️ PROMPT INJECTION GUARD"]
        Note_2["Security: Heuristic & LLM-based scans check for adversarial intent."]
        HS_Scan[Heuristic scan] --> LC_Class[LLM classifier]
    end

    subgraph TS_Sub ["🔐 TOOL SCOPING"]
        Note_3["Identity: Role extracted from JWT maps to a set of allowed tool names."]
        Note_9["Entitlements: Permissions looked up from Okta FGA / OPA Service."]
        Note_8["Persistence: Scope cached in Redis for stateless multi-user clustering."]
        JWT --> Role["Role Extraction"]
        Role --> Auth_Svc[[Auth Service]]
        Auth_Svc --> Redis[(Redis Cache)]
        Redis --> Scope["SessionScope"]
    end

    subgraph RAG_Sub ["📚 RAG — CHROMADB"]
        Note_4["Reliability: If ChromaDB is offline, triggers Scripted Fallback."]
        EQ_Embed[Embed query] --> SD_Search[Vector similarity search]
        SD_Search -->|KB Empty| FB_1["Scripted Fallback"]
        Conf_Gate{"Confidence
≥ threshold?"}
        SD_Search -->|Found| Conf_Gate
    end

    subgraph HITL_Sub ["🙋 HUMAN IN THE LOOP"]
        Note_5["Asynchronous: Agent suspends state to DB while awaiting human approval."]
        UG_Gate[Uncertainty gate]
        AG_Gate[Authorization gate]
        State_DB[(💾 State DB)]
        UG_Gate --> State_DB
        AG_Gate --> State_DB
        Audit_Store[("📁 Audit Log")]
        State_DB --> Audit_Store
    end

    subgraph LOOP_Sub ["🤖 LANGGRAPH AGENT LOOP"]
        Note_7["Follow-up: Flow returns to reasoning node after tool results for next steps."]
        Note_10["Binding: LLM only sees authorized tools; others are logically invisible."]
        Scope --> Filter["Filter Tools"]
        Filter --> Bind["Bind to LLM"]
        Bind --> MS_Reason["Multi-step reasoning"]
        TC_Gate{"High-stakes?"}
        MS_Reason --> TC_Gate
    end

    subgraph OBS_Sub ["🔭 OBSERVABILITY"]
        Note_6["Monitoring: Tracking confidence drift to detect KB staleness."]
        FT_Trace[Full trace] --> LT_Metrics[Latency + Tokens]
    end

    %% Global Connections
    User --> PIG_Sub
    PIG_Sub -->|clean| TS_Sub
    PIG_Sub -->|injection| Block_Node(["🚫 Block + log"])
    TS_Sub --> LOOP_Sub
    LOOP_Sub --> OBS_Sub
    
    MS_Reason --> RAG_Sub
    Conf_Gate -->|low score| UG_Gate
    Conf_Gate -->|confident| LOOP_Sub
    
    State_DB -->|Resume| Tools_Node["🛠️ Exec Approved Action"]
    TC_Gate -->|write| AG_Gate
    TC_Gate -->|read| Tools_Node
    
    Tools_Node -->|Result Fed Back| MS_Reason
    OBS_Sub --> Done_Node(["📄 Final response"])

    %% Styling
    classDef container fill:#fff4dd,stroke:#d4a017,stroke-width:2px;
    classDef fallback fill:#ffebee,stroke:#c62828,stroke-width:1px;
    class PIG_Sub,TS_Sub,RAG_Sub,HITL_Sub,LOOP_Sub,OBS_Sub,KFP_Sub container;
    class FB_1 fallback;
                    

Keeping it monolithic is a feature, not a shortcut

I think this is the right first decision. The PRD describes eventual microservices, but the code stays monolithic for now. That looks like a compromise until you remember what we are trying to learn first: does the control flow make sense, do the approval gates feel sane, and do the boundaries hold when the agent starts calling tools?

The session scoping code is a good example of this. It captures a production-shaped idea without pretending the production dependencies already exist.

ROLE_PERMISSIONS: dict[str, set[str]] = {
    "analyst": {"search_knowledge_base", "get_ticket_status"},
    "supervisor": {"search_knowledge_base", "get_ticket_status",
                   "create_ticket", "escalate_to_manager"},
}

if session_allowlist is not None:
    self.allowed_tools = role_allowed & session_allowlist
else:
    self.allowed_tools = role_allowed

This is not fancy, but it is the right kind of prototype simplification. The code already models two permission layers: the role boundary and the session boundary. Later you can replace the hardcoded maps with JWT-derived entitlements or a policy service without rewriting the rest of the graph.

The prototype is useful because it is honest about what is fake

The good news is that the code mostly matches the spirit of the PRD. The less good news is that several enterprise pieces are still placeholders. Identity is hardcoded by role instead of coming from Okta or Azure AD. The audit log is SQLite, not Postgres with row-level security. The knowledge refresh pipeline is a static in-memory ingest, not a scheduled versioned pipeline.

KB_ARTICLES = [
    {
        "id": "kb-005",
        "title": "Laptop hardware failure and urgent repair",
        "content": "... critical hardware tickets within 1 hour."
    },
    ...
]

collection.add(documents=docs, embeddings=embeddings,
               metadatas=metas, ids=ids)

That snippet is exactly why I would call this a prototype and not a half-built platform. The repo is honest about where the KB comes from today. It is a curated in-memory corpus used to exercise the retrieval loop, not a real ingestion pipeline pretending to be production-ready.

What still has to get harder before this deserves production status

• Move identity, tool entitlements, and audit storage out of local process memory.
• Turn the knowledge base ingest into a scheduled pipeline with document versioning and rollback.
• Separate the approval workflow from the agent runtime so a human review does not depend on a terminal prompt.
• Expand fallback handling to cover more classes of provider and infrastructure failures.

What I would not ship from this prototype

I would not ship per-session KB ingestion, local-only role derivation, or a terminal-driven approval path. Those are reasonable prototype choices because they expose the architecture cleanly, but they become liabilities as soon as more than one user or one operator is involved.

I also would not ship the current boundary between “prototype control” and “production control” without making it explicit in docs. The code is good at modeling the eventual system shape, but a team can still fool itself if it treats these local implementations as merely temporary plumbing rather than design constraints that need real replacements.

That sets up the rest of the series. In the next post, I’ll walk through the request lifecycle in code, starting from run_session() and ending at the LangGraph routing decisions that make the whole app feel like an agent instead of a scripted assistant.