8 Agents, One Local LLM — Ollama Concurrency Measured
I fired 8 agents at one local model expecting a speedup. Default Ollama serializes requests, so eight at once matched one. I benchmarked OLLAMA_NUM_PARALLEL.
Tags
Local LLM
2 posts
Ollama + FastAPI Local LLM Server — Docker Production Guide
Build a production LLM API with Ollama and FastAPI. Covers SSE streaming, health checks, Docker Compose. Llama 3.2 and Mistral execution logs included.