Nishaglobal Education logo

LLM app frameworks: SDKs, LangChain, LlamaIndex and more

How to build applications with large language models: calling models with the official SDKs from OpenAI, Anthropic and Google, then the popular frameworks: LangChain, LlamaIndex, Haystack, Semantic Kernel, DSPy and the Vercel AI SDK. Simple examples and a guide to which to use for which job.

Lesson 41 of 4422 min readQuiz: 5 questions

🎯 By the end of this lesson you will be able to:

  • Call a model with a provider SDK and understand each line
  • Explain what LangChain, LlamaIndex, Haystack, Semantic Kernel, DSPy and the Vercel AI SDK are for
  • Choose the right framework for a given application
In this lesson
  1. Step 1: the provider SDKs
  2. LangChain: the big toolbox
  3. LlamaIndex: your documents, answered
  4. Haystack, Semantic Kernel, DSPy and the Vercel AI SDK
  5. Which framework for which job?
  6. What this means for your role
  7. Key takeaways
  8. Quiz

Step 1: the provider SDKs

Every model provider has an official SDK. You create a client with your API key (a secret password for the service), send a message and read the answer. The three below are the most common; the pattern is almost the same in each.

Anthropic (Claude) in Python

import anthropic

client = anthropic.Anthropic()          # reads the ANTHROPIC_API_KEY environment variable
message = client.messages.create(
    model="claude-...",                 # choose a current model from Anthropic's model list
    max_tokens=500,                     # the longest answer you allow
    messages=[{"role": "user", "content": "Summarise this complaint in 3 bullets: ..."}],
)
print(message.content[0].text)

Model names change as new versions are released, so the examples use placeholders. See our AI model releases page for current models.

OpenAI and Google Gemini in Python

# OpenAI
from openai import OpenAI
client = OpenAI()                                  # reads OPENAI_API_KEY
response = client.responses.create(model="gpt-...", input="Summarise this complaint: ...")
print(response.output_text)

# Google Gemini
from google import genai
client = genai.Client()                            # reads GEMINI_API_KEY
response = client.models.generate_content(model="gemini-...", contents="Summarise this complaint: ...")
print(response.text)

Three providers, the same idea: client, request, answer. Keep your API key out of your code, for example in an environment variable or a secrets manager.

LangChain: the big toolbox

LangChain (Python and JavaScript) is the most widely used LLM framework. It gives you one standard way to talk to almost any model provider, reusable prompt templates, and hundreds of integrations with databases, document loaders and tools. Its agent and workflow features build on LangGraph (Lesson 42).

A provider-neutral prompt with LangChain

from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate

model = init_chat_model("anthropic:claude-...")   # change to "openai:gpt-..." with no other code changes
prompt = ChatPromptTemplate.from_messages([
    ("system", "You summarise customer emails in 3 bullet points."),
    ("user", "{email}"),
])
chain = prompt | model                            # the | joins steps into a chain
print(chain.invoke({"email": "My parcel arrived broken..."}).content)

Best for: projects that need many integrations or easy switching between providers.

LlamaIndex: your documents, answered

LlamaIndex specialises in connecting LLMs to your own data. It loads documents from files and many sources, splits them into chunks, creates embeddings, stores them and answers questions with sources: the whole RAG pipeline from Lessons 9 and 15, in a few lines.

Questions over a folder of policy documents

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

documents = SimpleDirectoryReader("policies/").load_data()   # read PDFs, Word files, text...
index = VectorStoreIndex.from_documents(documents)            # chunk, embed and store them
query_engine = index.as_query_engine()
print(query_engine.query("How many days of annual leave do new staff get?"))

Best for: document question answering, knowledge assistants and anything RAG-heavy.

By default LlamaIndex uses OpenAI models; you can configure Anthropic, Google, local models and other vector databases.

Haystack, Semantic Kernel, DSPy and the Vercel AI SDK

  • Haystack (by deepset): builds search and RAG systems as clear pipelines of components. Strong for production search applications.
  • Semantic Kernel (by Microsoft): connects LLMs to C#, Python and Java applications with plugins. Microsoft's Agent Framework reached version 1.0 in 2026 and is the direct successor to Semantic Kernel and AutoGen (Lesson 42), with migration guides for existing projects, so start new Microsoft projects there.
  • DSPy (from Stanford): instead of hand-writing long prompts, you describe what goes in and out, and DSPy can automatically improve the prompts using your examples and a metric.
  • Vercel AI SDK (TypeScript): the most popular choice for AI features in web apps built with Next.js, React and other JavaScript frameworks, with streaming chat interfaces built in.

DSPy and the Vercel AI SDK, side by side

# DSPy (Python): describe the task, not the prompt
import dspy
dspy.configure(lm=dspy.LM("openai/gpt-..."))
classify = dspy.Predict("email -> category")
print(classify(email="My parcel never arrived").category)

// Vercel AI SDK (TypeScript) in a web app
import { generateText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
const { text } = await generateText({
  model: anthropic("claude-..."),
  prompt: "Write a friendly reply to this customer: ...",
});

Which framework for which job?

What are you building?1–2 model callsProvider SDKQ&A over docsLlamaIndexMany integrationsLangChainJS web appVercel AI SDKMulti-step agentsAgent frameworkSimple features need no framework; add one when it solves a real problem
Start from what you are building. Most projects need only one of these, and simple features need none.
You are building…Good choiceWhy
One or two model calls in a featureProvider SDKLeast code, full control, no extra dependency
Q&A over your documentsLlamaIndex (or LangChain)Built for loading, chunking, indexing and retrieval
An app using many tools and data sourcesLangChainHundreds of ready-made integrations
A chat or AI feature in a JavaScript web appVercel AI SDKStreaming UI, React hooks, many providers
AI inside a Microsoft / .NET applicationMicrosoft Agent Framework (successor to Semantic Kernel)First-class C# support and Azure integration
Production search over large collectionsHaystackClear, testable pipelines for search and RAG
Prompts you want to optimise with dataDSPyImproves prompts automatically against a metric
Multi-step agents with toolsAn agent framework (Lesson 42)Control over loops, state and approvals

What this means for your role

Developer

Write your first feature with the provider SDK, then rebuild it with the framework your team is considering. You'll see exactly what the framework adds.

Tester / QA

Frameworks change between versions; pin versions and re-run your evaluation set when upgrading, as default prompts and behaviour can change.

Business Analyst / PO

Knowing the main options helps you estimate effort: document Q&A with LlamaIndex is a well-trodden path; a novel multi-agent system is not.

Tech Lead / Architect

Choose a small number of frameworks for the organisation, wrap them behind your own interfaces, and track their release notes.

Project / Delivery Manager

Framework upgrades are recurring work; plan for them like other dependency updates.

Scrum Master / Agile

Use spikes to compare frameworks on a real task with agreed criteria such as code size, quality and debuggability.

Try it: one task, two ways

  1. Get a free or trial API key from one provider and run its SDK example with your own prompt.
  2. Install LangChain and run the same prompt through init_chat_model.
  3. Compare: lines of code, what you can see happening, and how easy it would be to switch providers.

Key takeaways

  • Provider SDKs (OpenAI, Anthropic, Google) are the simplest way to call a model: client, request, answer.
  • LangChain offers integrations and provider-neutral code; LlamaIndex specialises in RAG over your documents.
  • Haystack suits production search pipelines, the Microsoft Agent Framework (successor to Semantic Kernel) suits .NET shops, DSPy optimises prompts, and the Vercel AI SDK suits JavaScript web apps.
  • Pick by the job you're building, start simple, and turn on tracing to see what frameworks do.

Key terms

API key
A secret code that identifies your account when calling a service. Never put it in public code.
Integration
Ready-made code that connects a framework to another service, such as a database or file store.
Chain
A sequence of steps, such as prompt then model then output parser, joined together.
Streaming
Sending the answer to the user word by word as it is generated.

Check your understanding

Answer all 5 questions. Score 3 or more to complete the lesson.

  1. 1. You want users to ask questions about 300 internal PDF policies. Which framework is built for this?

  2. 2. What does LangChain's init_chat_model help with?

  3. 3. Which tool is most popular for AI chat features in a Next.js web app?

  4. 4. Where should your API key live?

  5. 5. What makes DSPy different?

Answer all 5 questions to check.

Practise more