Most people meet large language models through a chat window. That works fine until you want the model to do the same job a hundred times, against your own data, with output your code can actually use. At that point you need structure around the model, and LangChain is one of the more popular ways to get it.

This is the first of three parts. Part 1 covers models, messages, and the settings that control generation. Part 2 will cover prompt templates and output parsers. Part 3 will cover document loaders, embeddings, vector stores, and retrievers, which together give you retrieval augmented generation.

Every example runs on your own machine with Ollama. No API keys, no per-token billing, and nothing leaves your laptop. The versions used throughout are langchain 1.3.16, langchain-core 1.6.0, and langchain-ollama 1.1.0.

What LangChain Actually Does

A language model has a narrow interface. You give it text, it gives you text back. Everything that makes an LLM application useful sits around that call, and that surrounding work is remarkably repetitive.

Every provider ships a different SDK. OpenAI, Anthropic, Google, and Ollama each have their own client library, their own message format, and their own way to request structured output. Swapping providers means rewriting your integration layer. LangChain puts one interface in front of all of them, so switching from a local 3B model to a hosted frontier model is usually a one-line change.

Beyond that, LangChain gives you composable pieces for the work that always shows up: templating prompts, parsing replies into real Python objects, chunking documents, and wiring retrieval into a prompt. You could write all of it yourself. Most people write about 60% of it, badly, before deciding a framework was a reasonable idea.

Three things LangChain is not. It is not a model, so you still need something to actually generate tokens. It is not a server, so it does not host anything for you. It is also not required, and for a single API call in a small script you should skip it.

The three layers you meet first are chat models, prompts, and chains. This post covers the first one.

Setting Up Ollama

Ollama runs models locally and exposes an HTTP API on port 11434. Install it from ollama.com, then pull the two models used in this series:

ollama pull llama3.2:3b
ollama pull nomic-embed-text

llama3.2:3b is a three billion parameter chat model. It fits in about 2 GB and runs at usable speed on a laptop CPU. nomic-embed-text is a dedicated embedding model that Part 3 uses. Confirm both are present:

ollama list

NAME                       ID              SIZE      MODIFIED
llama3.2:3b                a80c4f17acd5    2.0 GB    2 minutes ago
nomic-embed-text:latest    0a109f422b47    274 MB    20 hours ago

Then install the Python packages. Use a virtual environment:

python3 -m venv venv
source venv/bin/activate
pip install langchain langchain-ollama

A note on the package split, because it confuses people. langchain-core holds the base abstractions such as messages, prompts, and parsers. langchain-ollama holds the Ollama integration. The top-level langchain package in version 1.x is much smaller than it used to be, and now contains only agents, chat_models, and tools. If you find a tutorial that imports PromptTemplate from langchain, that tutorial predates version 1.0.

Your First Call

Three lines gets you a working model call:

from langchain_ollama import ChatOllama

model = ChatOllama(model="llama3.2:3b")
response = model.invoke("Name the seven OSI layers, bottom to top. List only.")
print(response.content)

1. Physical
2. Data Link
3. Network
4. Transport
5. Session
6. Presentation
7. Application

The important detail is that invoke() does not return a string. It returns an AIMessage object, and the text lives on .content. Print the whole object and you can see why that matters:

print(repr(response))

AIMessage(content='1. Physical\n2. Data Link\n3. Network\n4. Transport\n5. Session\n6. Presentation\n7. Application', additional_kwargs={}, response_metadata={'model': 'llama3.2:3b', 'created_at': '2026-08-21T21:58:24.261024Z', 'done': True, 'done_reason': 'stop', ...

The message carries metadata alongside the text: which model answered, when it finished, why it stopped, and how many tokens it used. You can read the token accounting directly:

print(response.usage_metadata)

{'input_tokens': 38, 'output_tokens': 29, 'total_tokens': 67}

That done_reason field becomes useful later in this post.

The Message Classes

Passing a bare string to invoke() is a convenience. Underneath, LangChain wraps it in a HumanMessage and sends a list. Once you want to set the model’s behavior or hold a conversation, you build that list yourself.

There are three classes you use constantly, all from langchain_core.messages:

• SystemMessage sets the model’s role, rules, and tone. It goes first in the list and you normally send exactly one. This is where you put instructions like “answer in one sentence” or “you are a network engineer” or “never invent a command you are not certain about”.
• HumanMessage is what the user said. Everything you would type into a chat box becomes one of these.
• AIMessage is what the model said. You get one back from every call, and you append it to the list yourself when you want the model to remember its own previous answer.

Here is all three in one script:

from langchain_ollama import ChatOllama
from langchain_core.messages import SystemMessage, HumanMessage, AIMessage

model = ChatOllama(model="llama3.2:3b")

messages = [
    SystemMessage(content="You are a terse network engineer. Answer in one sentence."),
    HumanMessage(content="What is the difference between a VLAN and a subnet?"),
]

response = model.invoke(messages)
print(response.content)

A VLAN is a virtual network that shares the same physical infrastructure, while a subnet is a division of a physical network into two or more distinct networks with their own IP address ranges.

Now the part that trips up almost everyone the first time. The model has no memory. Each call to invoke() is completely independent. The model does not remember the previous question, and Ollama is not holding a session for you. The list of messages is the memory, and you are responsible for maintaining it.

To continue the conversation, append the reply you received and then append the next question:

messages.append(response)
messages.append(HumanMessage(content="Which one does a router care about?"))

follow_up = model.invoke(messages)
print(follow_up.content)

A router primarily cares about subnets to route traffic between them, but VLANs are also important as they allow a single router to switch between VLANs to route traffic.

The model answered a pronoun question correctly because the earlier exchange was in the list. Notice the cost of that memory. The first call in this exchange sent 48 input tokens. The second sent 90, because it carried the system message, the original question, and the model’s own answer along with the new question. Every turn resends the entire history, which is why long conversations get slower and, on a paid API, more expensive.

There is a fourth class worth knowing about: ToolMessage carries the result of a function the model asked to call. Tool calling deserves its own post, so this series leaves it alone.

Controlling the Output

Two settings on ChatOllama do most of the work. Both are passed at construction time.

temperature

temperature controls how the model picks each next token. At 0.0 it always takes the highest-probability option, so the same prompt gives the same answer. Raise it and the model samples from a wider set of candidates, which produces more variety and more mistakes.

Run the same prompt three times at each end of the range and the difference is obvious:

from langchain_ollama import ChatOllama

prompt = "Give a name for a network monitoring tool. Reply with the name only."

cold = ChatOllama(model="llama3.2:3b", temperature=0.0)
for _ in range(3):
    print(cold.invoke(prompt).content.strip())

hot = ChatOllama(model="llama3.2:3b", temperature=0.9)
for _ in range(3):
    print(hot.invoke(prompt).content.strip())

Vigilix
Vigilix
Vigilix
Vigilix
PulseWatch
Sentrio

Pick the value from the job, not from taste:

Task	Temperature	Reason
Data extraction, classification	0.0	You want the same answer every time
Generating code or config	0.0 to 0.2	Creativity is a defect here
Summarizing	0.2 to 0.4	Slight variation reads better
Drafting prose	0.5 to 0.7	Repetitive phrasing is the failure mode
Brainstorming names or ideas	0.8 to 1.0	Variety is the whole point

If you are debugging a chain and the output keeps changing, set temperature=0 first. You cannot fix what you cannot reproduce.

num_predict

num_predict caps how many tokens the model generates. A token is roughly three quarters of an English word, so 40 tokens is about 30 words. If you want the full explanation of what a token is and why models count them that way, I wrote about tokenizers previously.

The critical thing to understand is that this is a hard stop, not an instruction. The model does not shorten its answer to fit. It writes normally and generation is cut off mid-thought:

capped = ChatOllama(model="llama3.2:3b", temperature=0.0, num_predict=40)
response = capped.invoke("Explain BGP route reflectors.")

print(response.content)
print("stop reason:", response.response_metadata["done_reason"])

BGP Route Reflectors (RRs) are a type of BGP (Border Gateway Protocol) configuration used to distribute routes between multiple Autonomous Systems (ASes). They are used to improve the scalability
stop reason: length

The sentence ends in the middle of a clause, and done_reason reports length instead of the usual stop. Always check that field when you set a cap. It is the difference between “the model finished” and “the model was interrupted”.

If you want a short answer, ask for one in the SystemMessage. Use num_predict as a safety limit that stops a runaway generation from burning ten minutes of CPU, and set it generously above what you expect.

Both settings default to None, which means ChatOllama sends nothing and Ollama applies its own defaults:

model = ChatOllama(model="llama3.2:3b")
print(model.temperature, model.num_predict)

None None

Streaming

A 3B model on a laptop CPU takes several seconds to write a paragraph. With invoke() the user stares at nothing for the whole duration. Swap in .stream() and tokens arrive as they are produced:

model = ChatOllama(model="llama3.2:3b", temperature=0.0, num_predict=30)

for chunk in model.stream("What does MTU stand for?"):
    print(chunk.content, end="", flush=True)
print()

Each chunk is an AIMessageChunk with a small piece of .content. Printed as raw values, the granularity is clear:

'MT' 'U' ' stands' ' for' ' Maximum' ' Transmission' ' Unit' '.' ' It' ' refers' ' to' ' the' ' largest' ' amount' ' of' ' data' ' that'

Total time to the final token is the same. Time to the first token drops to almost nothing, and that is what users perceive as speed. Every chain you build in the next two parts supports .stream() in place of .invoke().

That is the foundation. You can call a local model, control how it generates, and hold a conversation by managing the message list yourself. What you cannot do yet is reuse a prompt with different inputs, or get anything back other than a blob of text. Part 2 will cover PromptTemplate and ChatPromptTemplate for building prompts you can parameterize, MessagesPlaceholder for slotting conversation history into a template, and the output parsers that turn a model’s reply into a validated Python object.