Module 1: Architecture and Composition

LangChain Architecture Overview

Learning objectives

Explain the core mental model behind LangChain Architecture Overview
Apply LangChain Architecture Overview within Architecture and Composition
Identify important boundaries, trade-offs, and failure modes
Produce concrete evidence from the practice exercise

Migration rule: when the vault reference uses AgentExecutor, ConversationBufferMemory, or imports from langchain.chains, translate the concept to create_agent, graph state with a checkpointer, or langchain-classic respectively.

Legacy and migration reference from the vault

Related: LangChain Core Concepts | LangChain Chains | LangChain Callbacks and Tracing | LangChain Index

This note is the block diagram. Every other note in the vault is a zoom-in on one of the boxes here.

The Big Picture

                    ┌────────────────────────────┐
                    │      Caller / UI / API     │
                    └────────────┬───────────────┘
                                 │ input
                                 ▼
┌──────────────────────────────────────────────────────────────────┐
│                         RUNTIME                                  │
│  ─ orchestrates invoke / batch / stream / async                  │
│  ─ attaches callbacks and metadata                               │
│                                                                  │
│   ┌──────────────────────────────────────────────────────────┐   │
│   │                     CHAIN / RUNNABLE                     │   │
│   │                                                          │   │
│   │   ┌─────────────┐    ┌─────────────┐    ┌────────────┐  │   │
│   │   │   Prompt    │───▶│    Model    │───▶│  Output    │  │   │
│   │   │  Template   │    │  (LLM/Chat) │    │  Parser    │  │   │
│   │   └─────────────┘    └─────────────┘    └────────────┘  │   │
│   │          ▲                                    │          │   │
│   │          │                                    │          │   │
│   │   ┌─────────────┐                      ┌────────────┐   │   │
│   │   │    Memory   │                      │  Tools     │   │   │
│   │   │  (context)  │                      │ (optional) │   │   │
│   │   └─────────────┘                      └────────────┘   │   │
│   │          ▲                                    │          │   │
│   │          │                                    │          │   │
│   │   ┌─────────────┐                      ┌────────────┐   │   │
│   │   │  Retriever  │◀─────────────────────│ VectorStore│   │   │
│   │   │  (optional) │                      │ (optional) │   │   │
│   │   └─────────────┘                      └────────────┘   │   │
│   └──────────────────────────────────────────────────────────┘   │
│                                                                  │
│   ┌──────────────────────────────────────────────────────────┐   │
│   │                    CALLBACK / TRACING                    │   │
│   │         LangSmith, stdout, custom handlers               │   │
│   └──────────────────────────────────────────────────────────┘   │
└──────────────────────────────────────────────────────────────────┘

Read the diagram top-to-bottom. The caller invokes a chain. The runtime manages execution mode (sync, async, batch, stream) and callbacks. Inside the chain, data flows left-to-right: prompt → model → parser. Memory feeds context into the prompt. Tools are invoked if the chain includes an agent or tool-binding step. Retrievers fetch documents from a vector store.

The Five Layers

1. Runtime Layer

The execution engine. It does not contain business logic; it manages how the chain runs.

Responsibilities:

Dispatch invoke, batch, stream, and async variants
Attach callbacks and propagate them through nested runnables
Collect metadata and tags for tracing
Handle streaming tokens and partial outputs

Engineering analogy: the event loop in Node.js or the executor in Python's concurrent.futures. The chain is the task; the runtime is the scheduler.

2. Chain / Runnable Layer

The composition layer. This is where you define the dataflow graph.

Building blocks:

Sequential — A | B | C
Parallel — RunnableParallel runs branches concurrently
Conditional — RunnableBranch or custom functions pick a path
Dynamic — agents build the path at runtime based on model decisions

Engineering analogy: a Makefile or a DAG workflow engine. Each step is a node; dependencies are edges.

See LangChain Chains.

3. Component Layer

The primitives that do the actual work.

Component	Role	Example
Prompt Template	Shape the input	`ChatPromptTemplate`
Model	Generate text	`ChatOpenAI`, `ChatAnthropic`
Output Parser	Structure the output	`JsonOutputParser`, `PydanticOutputParser`
Document Loader	Ingest external data	`PyPDFLoader`, `WebBaseLoader`
Text Splitter	Chunk long documents	`RecursiveCharacterTextSplitter`
Vector Store	Store embeddings	`Chroma`, `FAISS`, `Pinecone`
Retriever	Fetch relevant documents	`VectorStoreRetriever`
Tool	External function	`search`, `calculator`
Memory	Session state	`ConversationBufferMemory`

Each component is a Runnable (or can be wrapped into one). See individual notes for deep dives.

4. Integration Layer

The bridge to external systems.

Model providers — OpenAI, Anthropic, Google, Cohere, Ollama, HuggingFace
Vector databases — Pinecone, Weaviate, Qdrant, Milvus, pgvector
Document sources — S3, Azure Blob, Confluence, Notion, SharePoint
Tool platforms — SerpAPI, WolframAlpha, ArXiv

LangChain's value is not the integration itself; it is the uniform interface. Swapping Pinecone for Weaviate is a one-line change because both implement the same retriever contract.

5. Observability Layer

Callbacks and tracing.

Callbacks — fire on start, end, error, and new-token events
LangSmith — hosted tracing and evaluation platform
Custom handlers — log to Datadog, Prometheus, or your own backend

Every Runnable accepts a config with callbacks. This is how you get visibility without polluting business logic.

See LangChain Callbacks and Tracing.

Data Flow on a Single Invocation

Caller invokes chain.invoke({"topic": "RAG"})
Runtime looks up callbacks, starts the root span
Prompt Template renders variables into a list of messages
Memory (if attached) prepends past conversation to the messages
Model receives messages and returns a response
Output Parser transforms the response string into a typed object
Runtime calls on_chain_end on all callbacks, returns result

If the chain includes a retriever, step 3.5 fetches documents and injects them into the prompt.

If the chain includes tools, the model may emit a tool-call request; the runtime dispatches to the tool, feeds the result back, and re-invokes the model.

Why This Decomposition

Layer	Changes when...	Fails because...
Runtime	You need async or streaming	Event loop blocked, callback leaks
Chain	You add a new step or branch	Wrong input/output types between nodes
Component	You swap model or retriever	Bad prompt, wrong schema, irrelevant docs
Integration	You migrate vector DB	Connection failure, auth expiration
Observability	You need a new dashboard	Missing callbacks, broken trace propagation

Engineering analogy: this is MVC for LLM pipelines. The chain is the controller, components are models, and the runtime is the request handler.

What LangChain Doesn't Have

To set expectations:

No built-in model hosting. You call external APIs or local inference servers.
No production serving layer. LangChain is a library, not a server. For serving, wrap chains in FastAPI or use LangServe.
No built-in evaluation suite. LangSmith provides tracing; evaluation is your responsibility (see LangChain Evaluation).
No fixed DAG engine for agents. Agents are dynamic graphs. If you need static DAGs with guarantees, use LangGraph (see LangGraph What Is LangGraph).

LangChain Core Concepts — vocabulary used in the diagram
LangChain Chains — the composition layer
LangChain Callbacks and Tracing — the observability layer
LangChain Models and Providers — the model integration layer
LangChain Index — full topic map

Practice lab

Implement the smallest runnable agent workflow that demonstrates LangChain Architecture Overview. Trace inputs, state, model and tool calls, outputs, and cost; inject one failure and add a regression test that prevents it from returning. Add an operational constraint such as concurrency, recovery, security, latency, or cost, and defend the resulting design trade-off.

Review questions

What problem does LangChain Architecture Overview solve, and what assumptions does it rely on?
Which boundary or failure case is easiest to miss, and how would you expose it?
What alternative design would you consider, and what trade-off would change the decision?
What artifact, trace, test, or metric proves that your implementation is correct?

Completion evidence

A working artifact, annotated trace, or reproducible experiment
At least one normal case and one deliberately failing or boundary case
A concise explanation of the design choice and its trade-offs
Saved output showing how correctness was evaluated