Orchestration

LangChain's core abstraction is the Runnable: prompts, models, parsers, retrievers and plain functions all expose invoke, batch, stream and async versions. LCEL composes them with |: prompt | model | parser is itself a Runnable.

You get batching, streaming, async, retries and tracing hooks for free on any composed chain. RunnableParallel fans out to several branches; RunnablePassthrough carries inputs forward; RunnableLambda wraps any function.

The trade-off: another layer of abstraction to debug, and frequent API churn. Use it where it saves real work; drop to plain code where it obscures.

Sequential chains pass outputs to the next step. Routing sends a request to a specialised chain based on a classification (by a cheap model or rules): billing questions to one prompt, technical ones to another. Routing to a cheaper model for easy requests is one of the biggest cost levers.

Fallbacks (chain.with_fallbacks([backup])) retry with a different model or provider when the primary errors or times out: availability engineering for LLMs.

Keep routes observable: log which route handled each request so you can evaluate each one separately.

Output parsers convert raw model output into Python objects: strings, lists, JSON, Pydantic models. With modern APIs, prefer llm.with_structured_output(MyModel), which uses the provider's native schema enforcement or tool calling under the hood.

Parsing failures still happen. Wrap with retry logic that feeds the error back (the same loop as Project 4).

Keep schemas small and focused per step; one giant schema for a whole document is harder for the model and harder to validate.

Streaming shows text as it's generated, cutting perceived latency dramatically even though total time is unchanged. chain.stream() or astream() yields chunks; astream_events exposes intermediate steps (retrieval done, tool called) for richer UIs.

Streaming complicates things: you can't validate a full structured response until it's complete, errors can arrive mid-stream, and you need a protocol to the client (Server-Sent Events is the usual choice).

Callbacks fire on events (LLM start/end, tool start/end, errors), and they are how tracing tools like LangSmith and Langfuse capture each step.

LlamaIndex focuses on the data side of LLM apps: connectors for many sources, node parsers (chunkers), indexes, retrievers, query engines and response synthesisers. It is quick for document Q&A and has strong advanced-retrieval building blocks.

LangChain is broader and more general-purpose (chains, tools, agents via LangGraph). The two interoperate, and many teams use pieces of both, or neither.

Choose on fit, team familiarity and how much you're willing to depend on fast-moving APIs.

Every orchestration pattern is a few dozen lines without a framework: a prompt template is an f-string; a chain is function composition; routing is an if/else on a classifier; fallback is try/except; streaming is an async generator.

Frameworks earn their place with integrations, tracing and standard abstractions. They cost you in debuggability, version churn and hidden behaviour (hidden prompts, hidden retries).

A strong interview answer: 'I can build it either way; here's why I chose this one for this system.'

LangChain 1.0 consolidated agent-building into one function, create_agent(model, tools, system_prompt=...), which runs a tool-calling loop on LangGraph. You get persistence, streaming and human-in-the-loop support from the LangGraph runtime without writing a graph yourself.

Middleware hooks into the agent loop (before and after model calls, around tool calls) to add cross-cutting behaviour: summarising long histories, human approval for certain tools, PII redaction, retries, model fallbacks, call limits. It's the same idea as web-framework middleware.

When you need custom control flow (branches, parallel steps, multi-agent hand-offs), drop down to LangGraph directly (week 35). LCEL remains the tool for deterministic chains.