LangChain vs LlamaIndex vs Haystack: Choose a RAG framework

You are ready to build a serious RAG application and you are stuck comparing frameworks. LangChain, LlamaIndex and Haystack all claim to do RAG. Each has a different style, and half the tutorials you find use APIs that changed in the last year. Picking wrong costs you a rewrite when the abstraction stops fitting.

A RAG framework is a Python library that builds retrieval-augmented generation pipelines by giving you parts for loading, chunking, retrieving and prompting, plus a way to wire them. This guide compares LangChain vs LlamaIndex vs Haystack by how each one wires those parts. It shows current code for all three and maps each to the project it fits.

Key takeaways

• All three frameworks build the same pipeline: load, chunk, embed, index, retrieve, rerank and generate. They differ in how you wire it.
• LangChain composes parts with code, using LCEL pipes or an agent from create_agent, and its agents run on LangGraph.
• LlamaIndex is data-first: VectorStoreIndex.from_documents loads, chunks, embeds and indexes a folder in two lines, with defaults you tune later.
• Haystack builds explicit pipelines of typed components with named connections, which makes swapping a retriever or ranker a one-line change.
• LangChain 1.0 moved legacy chains to langchain-classic, and Haystack 3.0 removed OpenAIGenerator, so check tutorial dates before copying code.

What is the core RAG pipeline every framework builds?

Every RAG framework builds the same seven steps. Documents are loaded, split into chunks, embedded and indexed. At query time the retriever finds candidates, an optional reranker sorts them, and the LLM answers from the top chunks.

The difference is not what the frameworks do. It is how much of the wiring you write, and how easy it is to see and change each step.

How does LangChain build RAG?

LangChain gives you the parts and you compose them in code. For a fixed two-step flow, you pipe a retriever, a prompt and a model together with LCEL. For an agent that decides when to search, you wrap the retriever as a tool and pass it to create_agent.

LangChain 1.0 slimmed the core package. Legacy chains such as RetrievalQA and load_qa_chain now live in langchain-classic. LCEL itself lives in langchain-core and still works.

# filename: langchain_rag.py
# description: Two-step RAG chain with LCEL in LangChain 1.x.
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

# Assume vector_store is already built, for example InMemoryVectorStore.
retriever = vector_store.as_retriever(search_kwargs={"k": 5})
llm = ChatOpenAI(model="gpt-5-mini")

prompt = ChatPromptTemplate.from_template(
    """Answer the question based only on this context:

{context}

Question: {question}"""
)


def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)


rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

answer = rag_chain.invoke("What is our Q4 revenue?")

You can see and swap every piece. The trade-off is that you choose every piece: chunk size, retriever, k and prompt. LangChain fits best when RAG is one tool inside a larger agent built on LangGraph.

How does LlamaIndex build RAG?

LlamaIndex starts from your data and hides the wiring behind an index. VectorStoreIndex.from_documents chunks, embeds and stores everything, and as_query_engine gives you retrieval plus generation in one object.

# filename: llamaindex_rag.py
# description: Load, index and query a folder with LlamaIndex defaults.
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

documents = SimpleDirectoryReader("./data_folder").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine(similarity_top_k=5)
response = query_engine.query("What is our Q4 revenue?")

This is the fastest path from a folder to an answer. The defaults (chunk size, embedding model, prompt) are a starting point, not a tuned setup. When a bad chunk comes back, you open the query engine’s parts and set them yourself. For multi-step logic, LlamaIndex adds Workflows, its event-driven way to build agents.

How does Haystack build RAG?

Haystack builds a Pipeline of typed components and makes you connect each output to the next input by name. The graph you draw on a whiteboard is the graph in the code, which makes reranking and A/B tests easy to wire.

Haystack 3.0 removed the old OpenAIGenerator in favor of OpenAIChatGenerator. The Sentence Transformers ranker now ships in the sentence-transformers-haystack integration package.

# filename: haystack_rag.py
# description: Retrieve, rerank and generate with a Haystack 3.x pipeline.
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.dataclasses import ChatMessage
from haystack_integrations.components.rankers.sentence_transformers import (
    SentenceTransformersSimilarityRanker,
)

# Assume document_store is an InMemoryDocumentStore with documents written.
template = [
    ChatMessage.from_system("Answer using only the documents provided."),
    ChatMessage.from_user(
        "{% for doc in documents %}{{ doc.content }}\n{% endfor %}"
        "Question: {{ query }}"
    ),
]

rag = Pipeline()
rag.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
rag.add_component("ranker", SentenceTransformersSimilarityRanker())
rag.add_component(
    "prompt_builder",
    ChatPromptBuilder(template=template, required_variables=["query", "documents"]),
)
rag.add_component("llm", OpenAIChatGenerator(model="gpt-5-mini"))

rag.connect("retriever.documents", "ranker.documents")
rag.connect("ranker.documents", "prompt_builder.documents")
rag.connect("prompt_builder.prompt", "llm.messages")

question = "What is our Q4 revenue?"
result = rag.run({
    "retriever": {"query": question, "top_k": 20},
    "ranker": {"query": question, "top_k": 3},
    "prompt_builder": {"query": question},
})
print(result["llm"]["replies"][0].text)

Every component and connection is visible. Trying a different ranker means changing one add_component line. Haystack also has an Agent component for tool calling, and agents can run inside pipelines.

LangChain vs LlamaIndex vs Haystack: How do they compare?

LangChain is the widest toolkit, LlamaIndex is the fastest path from documents to answers, and Haystack is the most explicit pipeline. All three are open source and work with the major model providers.

Framework	Style	Strongest at	Agent layer	License
LangChain	Compose parts in code	Integrations, agents with tools	LangGraph, create_agent	MIT
LlamaIndex	Index and query engine	Ingestion and query defaults	Workflows	MIT
Haystack	Typed pipeline graph	Explicit wiring, swappable parts	Agent component	Apache 2.0

If you would rather not write the pipeline at all, compare these with batteries-included platforms like Dify and RAGFlow.

How should you choose your RAG framework?

Pick by what already exists in your project and where your hardest problem sits. Four cases cover most teams.

You already run LangGraph agents and need to add RAG. Use LangChain. Wrap your retriever as a tool and the agent decides when to search. State and memory stay in the same graph, as in LangGraph persistence and thread models.

Your whole app is Q&A over thousands of PDFs. Start with LlamaIndex. Its readers, node parsers and query engines get you to a baseline fast. For questions that need several lookups, see multi-hop RAG with query decomposition.

Your team needs to A/B test retrievers and rankers. Use Haystack. Each variant is a pipeline with one component swapped, and the named connections keep the comparison honest.

You need one agent that searches PDFs and the web. Combine them. Build the orchestration in LangGraph and expose a LlamaIndex or Haystack pipeline as one of its tools. The build a team of AI agents lab covers the orchestration side.

What to do next

1. Write down your hardest problem: agent logic, document ingestion, or retrieval experiments.
2. Pick the framework that matches it, using the cases above.
3. Build the smallest pipeline from the code here against 20 of your real documents.
4. Build a labeled set of 50 questions and measure retrieval hits, as in tuning retriever top-k.
5. Pin framework versions in your lockfile, and check any tutorial’s date against the LangChain 1.0 and Haystack 3.0 changes.

Frequently asked questions

Should I use LlamaIndex or LangChain for RAG?

Use LlamaIndex when RAG over documents is the main job, and LangChain when RAG is one tool inside a larger agent. LlamaIndex gets a folder of files to answers in a few lines with sensible defaults. LangChain asks you to wire each step, which gives you control and fits naturally into LangGraph agents. Both are MIT licensed, and you can call one from the other.

Is LlamaIndex free?

Yes, the LlamaIndex framework is open source under the MIT license and free to use. You pay for the model and embedding APIs it calls, unless you run local models. LlamaParse and LlamaCloud are a separate hosted platform from the same company, sold as an enterprise product. You do not need them to build RAG with the open-source library.

What is the difference between LangChain and LangGraph?

LangChain is the high-level framework with model integrations and a ready agent loop called create_agent. LangGraph is the lower-level orchestration library that LangChain’s agents run on. Use LangChain to get an agent running quickly. Use LangGraph directly when you need custom control flow, branching, persistence or human review steps that combine fixed and agent-driven logic.

Is Haystack better than LangChain for production RAG?

Haystack is a strong fit when you want an explicit, inspectable pipeline that a team can review and change one component at a time. LangChain is a better fit when RAG sits inside agents with many tools. Both run in production. Choose by which wiring style your team reads faster, and test retrieval quality on your own data.

Do LangChain 1.0 and Haystack 3.0 break old RAG tutorials?

Some of them. LangChain 1.0 moved RetrievalQA, LLMChain and other legacy chains to the langchain-classic package, so old imports from langchain.chains fail. Haystack 3.0 removed OpenAIGenerator in favor of OpenAIChatGenerator. LCEL pipes and LlamaIndex’s VectorStoreIndex starter code still work. Check each tutorial’s date before you copy it.

Sources

• LangChain overview, LangChain, October 2026: create_agent, and that LangChain agents are built on LangGraph.
• LangChain v1 migration guide, LangChain, October 2026: legacy chains, retrievers and the hub moved to langchain-classic.
• LlamaIndex repository, LlamaIndex, October 2026: MIT license, the VectorStoreIndex starter, Workflows and LlamaParse.
• Haystack 3.0.0 release notes, deepset, 2026: OpenAIGenerator removed in favor of OpenAIChatGenerator.
• SentenceTransformersSimilarityRanker, deepset, October 2026: the integration import path and the retriever to ranker pipeline.