8 Agents, One Local LLM — Ollama Concurrency Measured
I fired 8 agents at one local model expecting a speedup. Default Ollama serializes requests, so eight at once matched one. I benchmarked OLLAMA_NUM_PARALLEL.
Tags
Ollama
10 posts
Local Reasoning Model Token Cost — thinking ON/OFF Measured
I ran 13 questions on gemma4:12b with thinking ON and OFF. Reasoning got one more right while spending 68x the output tokens and 19x the wall-clock.
Ollama num_ctx Silent Truncation — Why My Agent Forgot Its Prompt
My local agent kept ignoring its system prompt on long inputs. Past num_ctx, Ollama silently trims the front of the prompt — no error. I measured where it breaks.
Local LLM Cold Starts — Why First Replies Take 10 Seconds
After idling, my agent's first reply dragged. I pulled Ollama's load_duration across model sizes: 1.5s for 2GB up to 9.7s for 9.6GB, and split it by keep_alive.
Why Local LLMs Slow Down in Long Chats — Prefill vs Generation
A 9,700-token prompt took 55s to its first token, then 65ms on the identical second call. I split Ollama's timings into prefill vs generation to see why.
Local LLM Reproducibility: Testing Temperature and Seed
I sent the same prompt to local Gemma 4 dozens of times. temperature=0 was deterministic, and even at higher temperature a fixed seed collapsed output to one line.
Ollama Structured Outputs with Pydantic — Type-Safe JSON
A hands-on guide to Ollama's JSON schema enforcement with Pydantic for type-safe local LLM responses. Measured: 6x faster with near-100% parse success.
Ollama + FastAPI Local LLM Server — Docker Production Guide
Build a production LLM API with Ollama and FastAPI. Covers SSE streaming, health checks, Docker Compose. Llama 3.2 and Mistral execution logs included.
Private MCP Server with Gemma 4 + FastMCP — Fully Offline
Run an offline AI tool pipeline with Ollama, Gemma 4, and FastMCP, no internet needed. Built for medical, legal, and finance where data stays on the premises.
$30 Radio + Local AI = Internet-Free Smart Home
Analyzing a real-world project that achieves voice control and smart home automation without internet using just a Mac mini and a $30 LoRa radio. A deep dive into local AI × IoT implementation and costs.