LLM app frameworks: SDKs, LangChain, LlamaIndex and more
How to build applications with large language models: calling models with the official SDKs from OpenAI, Anthropic and Google, then the popular frameworks: LangChain, LlamaIndex, Haystack, Semantic Kernel, DSPy and the Vercel AI SDK. Simple examples and a guide to which to use for which job.
🎯 By the end of this lesson you will be able to:
- Call a model with a provider SDK and understand each line
- Explain what LangChain, LlamaIndex, Haystack, Semantic Kernel, DSPy and the Vercel AI SDK are for
- Choose the right framework for a given application
In this lesson
Step 1: the provider SDKs
Every model provider has an official SDK. You create a client with your API key (a secret password for the service), send a message and read the answer. The three below are the most common; the pattern is almost the same in each.
Anthropic (Claude) in Python
import anthropic
client = anthropic.Anthropic() # reads the ANTHROPIC_API_KEY environment variable
message = client.messages.create(
model="claude-...", # choose a current model from Anthropic's model list
max_tokens=500, # the longest answer you allow
messages=[{"role": "user", "content": "Summarise this complaint in 3 bullets: ..."}],
)
print(message.content[0].text)Model names change as new versions are released, so the examples use placeholders. See our AI model releases page for current models.
OpenAI and Google Gemini in Python
# OpenAI from openai import OpenAI client = OpenAI() # reads OPENAI_API_KEY response = client.responses.create(model="gpt-...", input="Summarise this complaint: ...") print(response.output_text) # Google Gemini from google import genai client = genai.Client() # reads GEMINI_API_KEY response = client.models.generate_content(model="gemini-...", contents="Summarise this complaint: ...") print(response.text)
Three providers, the same idea: client, request, answer. Keep your API key out of your code, for example in an environment variable or a secrets manager.
LangChain: the big toolbox
LangChain (Python and JavaScript) is the most widely used LLM framework. It gives you one standard way to talk to almost any model provider, reusable prompt templates, and hundreds of integrations with databases, document loaders and tools. Its agent and workflow features build on LangGraph (Lesson 42).
A provider-neutral prompt with LangChain
from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate
model = init_chat_model("anthropic:claude-...") # change to "openai:gpt-..." with no other code changes
prompt = ChatPromptTemplate.from_messages([
("system", "You summarise customer emails in 3 bullet points."),
("user", "{email}"),
])
chain = prompt | model # the | joins steps into a chain
print(chain.invoke({"email": "My parcel arrived broken..."}).content)Best for: projects that need many integrations or easy switching between providers.
LlamaIndex: your documents, answered
LlamaIndex specialises in connecting LLMs to your own data. It loads documents from files and many sources, splits them into chunks, creates embeddings, stores them and answers questions with sources: the whole RAG pipeline from Lessons 9 and 15, in a few lines.
Questions over a folder of policy documents
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
documents = SimpleDirectoryReader("policies/").load_data() # read PDFs, Word files, text...
index = VectorStoreIndex.from_documents(documents) # chunk, embed and store them
query_engine = index.as_query_engine()
print(query_engine.query("How many days of annual leave do new staff get?"))Best for: document question answering, knowledge assistants and anything RAG-heavy.
By default LlamaIndex uses OpenAI models; you can configure Anthropic, Google, local models and other vector databases.
Haystack, Semantic Kernel, DSPy and the Vercel AI SDK
- Haystack (by deepset): builds search and RAG systems as clear pipelines of components. Strong for production search applications.
- Semantic Kernel (by Microsoft): connects LLMs to C#, Python and Java applications with plugins. Microsoft's Agent Framework reached version 1.0 in 2026 and is the direct successor to Semantic Kernel and AutoGen (Lesson 42), with migration guides for existing projects, so start new Microsoft projects there.
- DSPy (from Stanford): instead of hand-writing long prompts, you describe what goes in and out, and DSPy can automatically improve the prompts using your examples and a metric.
- Vercel AI SDK (TypeScript): the most popular choice for AI features in web apps built with Next.js, React and other JavaScript frameworks, with streaming chat interfaces built in.
DSPy and the Vercel AI SDK, side by side
# DSPy (Python): describe the task, not the prompt
import dspy
dspy.configure(lm=dspy.LM("openai/gpt-..."))
classify = dspy.Predict("email -> category")
print(classify(email="My parcel never arrived").category)
// Vercel AI SDK (TypeScript) in a web app
import { generateText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
const { text } = await generateText({
model: anthropic("claude-..."),
prompt: "Write a friendly reply to this customer: ...",
});Which framework for which job?
| You are building… | Good choice | Why |
|---|---|---|
| One or two model calls in a feature | Provider SDK | Least code, full control, no extra dependency |
| Q&A over your documents | LlamaIndex (or LangChain) | Built for loading, chunking, indexing and retrieval |
| An app using many tools and data sources | LangChain | Hundreds of ready-made integrations |
| A chat or AI feature in a JavaScript web app | Vercel AI SDK | Streaming UI, React hooks, many providers |
| AI inside a Microsoft / .NET application | Microsoft Agent Framework (successor to Semantic Kernel) | First-class C# support and Azure integration |
| Production search over large collections | Haystack | Clear, testable pipelines for search and RAG |
| Prompts you want to optimise with data | DSPy | Improves prompts automatically against a metric |
| Multi-step agents with tools | An agent framework (Lesson 42) | Control over loops, state and approvals |
What this means for your role
Developer
Write your first feature with the provider SDK, then rebuild it with the framework your team is considering. You'll see exactly what the framework adds.
Tester / QA
Frameworks change between versions; pin versions and re-run your evaluation set when upgrading, as default prompts and behaviour can change.
Business Analyst / PO
Knowing the main options helps you estimate effort: document Q&A with LlamaIndex is a well-trodden path; a novel multi-agent system is not.
Tech Lead / Architect
Choose a small number of frameworks for the organisation, wrap them behind your own interfaces, and track their release notes.
Project / Delivery Manager
Framework upgrades are recurring work; plan for them like other dependency updates.
Scrum Master / Agile
Use spikes to compare frameworks on a real task with agreed criteria such as code size, quality and debuggability.
Try it: one task, two ways
- Get a free or trial API key from one provider and run its SDK example with your own prompt.
- Install LangChain and run the same prompt through
init_chat_model. - Compare: lines of code, what you can see happening, and how easy it would be to switch providers.
Key takeaways
- Provider SDKs (OpenAI, Anthropic, Google) are the simplest way to call a model: client, request, answer.
- LangChain offers integrations and provider-neutral code; LlamaIndex specialises in RAG over your documents.
- Haystack suits production search pipelines, the Microsoft Agent Framework (successor to Semantic Kernel) suits .NET shops, DSPy optimises prompts, and the Vercel AI SDK suits JavaScript web apps.
- Pick by the job you're building, start simple, and turn on tracing to see what frameworks do.
Key terms
- API key
- A secret code that identifies your account when calling a service. Never put it in public code.
- Integration
- Ready-made code that connects a framework to another service, such as a database or file store.
- Chain
- A sequence of steps, such as prompt then model then output parser, joined together.
- Streaming
- Sending the answer to the user word by word as it is generated.
Check your understanding
Answer all 5 questions. Score 3 or more to complete the lesson.
1. You want users to ask questions about 300 internal PDF policies. Which framework is built for this?
2. What does LangChain's init_chat_model help with?
3. Which tool is most popular for AI chat features in a Next.js web app?
4. Where should your API key live?
5. What makes DSPy different?