Use Go for LLM Orchestration, Not Model Research

I like Go for AI systems. I would not use it to build every part of one.

The split I have used in production looks like this:

• Python for model research, evaluation, and the ecosystem around machine learning
• Go for orchestration, APIs, tool execution, and high-volume production services

Python has the model ecosystem. Go is very good at the less glamorous work around the model: serving requests, calling tools, moving typed data, and staying alive in production.

Go rarely fixes the real LLM bottleneck

“Go can handle thousands of concurrent connections” sounds impressive in a slide deck. In an LLM application, it may be irrelevant. The request is often waiting on:

1. the model
2. the database or retrieval layer

Making the Go code faster will not make inference faster. Choose Go when the surrounding service is the problem you need to solve: many requests, parallel tool calls, typed contracts, networking, and deployment.

Production AI is mostly orchestration

An agent service often needs to:

• call models and tools concurrently
• validate structured inputs and outputs
• expose HTTP, streaming, or WebSocket interfaces
• connect internal systems through an MCP server
• emit traces and operational metrics
• deploy as a small, predictable service

None of this is mysterious AI work. It is production engineering, and Go is good at it.

I especially want types at tool boundaries. The model is already unpredictable. There is no prize for making the rest of the system unpredictable too.

Human approval is an architecture feature

An agent that can execute tools can also execute damaging tools.

During an internal demonstration, I found it uncomfortably easy to push an agent toward destructive actions involving a database and local files. An agent with tools is effectively a new operator in your system, except this operator can misunderstand a sentence at machine speed.

Separate operations into classes:

• safe read-only operations that may run automatically
• sensitive reads, such as secrets or environment files, that require stronger controls
• destructive writes, deletes, or shell operations that require explicit human approval

Treat the approval step as authorization, not as a friendly confirmation modal.

Keep the model replaceable

Models change too quickly for me to bury business workflow inside one provider’s API or one elaborate agent graph.

I prefer model-agnostic boundaries:

• a clear interface for model calls
• explicit tool contracts
• ordinary control flow where possible
• tracing around every consequential action
• business decisions outside model-specific prompts

I want to replace a model with the same level of drama as replacing another external API. That is not always possible, but it is a useful design pressure.

Use each ecosystem for its strength

Use Python where the Python ecosystem is excellent. Use Go where a boring, typed, deployable service is the desired result. The system does not care whether the repository looks perfectly consistent.

For broader technical decision support, see technical strategy coaching and architecture review mentoring.