Self-Hosted AI Agents for Business: A Practical 2026 Guide

A self-hosted AI agent runs on servers you control, so prompts, documents and ERP data stay on your own network. Here's the open-source stack, the hardware, a security checklist, what drives the cost, and how to connect it to ERPNext safely.

A private AI agent kept inside the company's own server: a lit vault room holds a rack running n8n workflows, an open-weight model chip with the AI orb above it and a read-only ERPNext cabinet, a log card records every step, a person with a checklist stands at the open vault door to approve, and a public cloud AI floats outside, apart from the data
Published 10 min read
Memory gpt-oss-20b runs within
16 GB
Single GPU gpt-oss-120b is built to fit
80 GB
Port Ollama listens on, localhost only by default
11434
OWASP risks for LLM apps, 2025 list
10

The short answer

What is a self-hosted AI agent?

In short

A self-hosted AI agent is an AI assistant that runs on servers you control, so your prompts, documents and ERP data never leave your own network. For most companies the right setup is a private agent for sensitive work (finance, HR, customer records) and a cloud AI plan for everything else.

An agent is a loop. A language model decides what to do, a tool runs the action, and the result goes back to the model for the next step. Oxide's engineers put it plainly: inference runs on GPUs, tool calls run on CPUs, and a harness ties the two together (Oxide, 29 Oct 2026). Self-hosted means you run all three parts yourself:

The model

An open-weight language model served on your own GPU or server.

The agent runtime

The workflow or framework that holds the loop, memory and tool calls: n8n, LangGraph, CrewAI and similar.

The tools

Your own systems: ERP, document store, email, ticketing.

You'll also hear it called a local AI agent, or simply private AI. If any one of those three parts is a public cloud API, the agent is only partly self-hosted. That's often fine. What matters is knowing exactly which data crosses which line.

Why now

Why are businesses moving AI agents in-house?

Two things changed in 2026. Agents got good enough to act, not just chat. And companies started asking where those actions happen.

On 11 November 2026, United Airlines said it would block unauthorised third-party AI agents from booking and changing tickets. It pointed to customers handing over "account credentials", plus unintended purchases and account changes (The Record, 12 Nov 2026). The same report notes Bank of America has told customers not to share login details with AI agents.

That's a customer story, but it carries over to your staff. When someone pastes a supplier contract into a personal AI app, or hooks a cloud agent up to the company mailbox, your data and credentials now sit with a third party you never vetted. That's the main reason owners ask us about private AI for business.

  1. Nov 2025

    India notifies the DPDP Rules, 2025

    Most duties for data fiduciaries apply 18 months after notification.
  2. 29 Oct 2026

    Oxide publishes its self-hosted agent setup

    Each agent session runs in its own short-lived VM.
  3. 11 Nov 2026

    United blocks third-party AI agents

    It cites the risk of customers handing over account credentials.

Regulation points the same way. India's Digital Personal Data Protection Rules, 2025 were notified in November 2025, and most duties for data fiduciaries apply 18 months later (PTI via The Week, 14 Nov 2025). A private agent won't make you compliant on its own. It does make "where did this personal data go?" much easier to answer.

Decide

Self-hosted AI vs a cloud AI plan: which is right for you?

Short answer

Neither wins every time. Self-host when the data is sensitive, the volume is high and steady, or a client or regulator asks where the data lives. Stay on a cloud plan when you need the strongest model for occasional hard tasks and the data isn't sensitive.

QuestionSelf-hosted AI agentCloud AI plan (ChatGPT, Claude, Gemini)
Where does data go?Stays on your serversSent to the vendor; business plans say they don't train on it by default
Model qualityGood open-weight models, a step behind the best cloud models on hard reasoningBest available models
Upfront costHardware or a GPU server, plus setup timeNone
Running costPower, hosting, upkeep, staff timePer-seat or per-token fees
Who patches and monitors itYour team or your partnerThe vendor
Works offline or in a closed networkYesNo
Best forFinance, HR, payroll, customer records, regulated data, high-volume repetitive tasksDrafting, research, general office work, low-sensitivity data

If you're choosing between the cloud plans themselves, our Claude vs ChatGPT for business comparison covers seats, data terms and ERP fit.

The stack

What does a private AI agent stack look like?

You don't need to build anything from scratch. A practical open-source stack for a mid-size business has four layers.

1. Model server

Ollama on one machine, vLLM for a shared server.

2. Open-weight model

A model whose licence allows business use.

3. Agent runtime

n8n for business workflows, LangGraph or CrewAI for code-heavy agents.

4. Your data and tools

ERPNext, documents, email, a vector store for search.

Model server

Ollama is the easiest way to run open-weight models on one machine. Its docs say "Ollama runs locally" and that the company doesn't see your prompts or data when you do. For a shared server with many users, vLLM gives you an HTTP server compatible with OpenAI's API, so most tools that talk to OpenAI can talk to your own model instead.

Open-weight model

Pick a model whose licence allows business use. One example: OpenAI's gpt-oss models ship under the Apache 2.0 licence. gpt-oss-20b runs within 16 GB of memory, and gpt-oss-120b is built to fit a single 80 GB GPU such as an NVIDIA H100 (gpt-oss on GitHub).

Llama, Qwen and Mistral come with their own licences. Read them before you commit.

16 GB
gpt-oss-20b runs within this much memory
80 GB
gpt-oss-120b fits a single GPU of this size

Agent runtime and workflows

n8n is our usual pick for business agents. The same tool runs the agent and the dull workflow around it: triggers, approvals, retries, logs. You can self-host it "on your own infrastructure, on-premises, or in a private cloud," and its docs name Docker Compose as the best option for production. n8n uses the Sustainable Use License, which allows use "for your own internal business purposes". Among open source AI agent platforms, it's the easiest for business teams to read and maintain. See how we orchestrate AI agents in n8n. For code-heavy agents, LangGraph or CrewAI fit better; we compared them in our guide to open-source AI agent frameworks.

Your data and tools

This is where the agent earns its keep: ERPNext, a document store, shared drives, email, a vector database for search over your own files. Every connection is also a door, which is why the security section matters more than the model you pick. If you want an agent that acts on its own schedule, see our wiki on the proactive AI assistant OpenClaw.

Hardware

What hardware do you need to run a self-hosted AI agent?

Short answer

The model drives the hardware. The agent runtime (n8n, a database, a vector store) runs fine on an ordinary server; the model is the heavy part.

WorkloadWhat it needsFits
Pilot: one team, simple Q&A and draftingA smaller model like gpt-oss-20b (within 16 GB of memory)A workstation or Mac with enough unified memory, or a single mid-range GPU
Department agents, document reading, ERP lookupsA mid-size model, several users at onceA dedicated GPU server on-premises or in your private cloud
Company-wide agents, larger model, heavy loadA large model like gpt-oss-120b (single 80 GB GPU)Data-centre GPU (H100 class) on-premises or a private cloud GPU instance

Three practical points from our projects

Pilot on one machine first

Measure real response times with your own documents before you buy a GPU server.

Count concurrent users

Ten people asking at once need more GPU memory than a hundred people asking through the day.

Split model and runtime

Past the pilot, the GPU box serves the model. n8n and the database live on a normal server you already know how to back up.

Security

How do you secure a self-hosted AI agent?

Self-hosting removes the third-party risk. It doesn't remove the agent risks. The OWASP Top 10 for LLM Applications (2025) lists them, and four matter most for business agents.

  • LLM01 Prompt injection
  • LLM02 Sensitive information disclosure
  • LLM06 Excessive agency
  • LLM10 Unbounded consumption

The checklist we use before any agent touches real data

Keep the model server private

Ollama binds to 127.0.0.1 on port 11434 by default (Ollama FAQ). Leave it there, or put it behind an authenticated reverse proxy. Never expose it to the internet as-is.

Its own user everywhere

No shared admin keys, no personal logins.

Read-only first

Let the agent look things up for a few weeks before it can create or change anything.

Human approval for actions

n8n can pause an agent before a chosen tool runs and ask a person to approve (n8n docs). Use it for anything that posts, pays, emails a customer or deletes.

Treat documents as untrusted

A supplier invoice or an email can carry hidden instructions. An agent that only reads can't be talked into writing.

Log every tool call

Who asked, what the agent did, which records it touched.

Set limits

Cap tokens, run time and tool calls per request, so a stuck loop can't burn the server for hours.

Patch on a schedule

n8n, the model server and the OS all need updates. Our self-hosting n8n guide covers hardening.

ERPNext

How do you connect a private AI agent to ERPNext safely?

ERPNext holds most of a company's sensitive data, so this is the connection to get right. We set it up the same way every time.

  1. 1Create a dedicated ERPNext user for the agentFor example "AI Agent - Sales". Frappe's own guide suggests a separate user for API calls.
  2. 2Give it a narrow roleOnly the doctypes it needs, read-only to start. Frappe checks API calls against the roles of the user who owns the key, and logs the request against that user (Frappe token authentication).
  3. 3Add User PermissionsIf the agent should see one company, branch or territory only.
  4. 4Generate an API key and secretStore them in n8n's credential store, never in a workflow node or a prompt.
  5. 5Route writes through approvalThe agent saves a Sales Order or Journal Entry as a draft and a person submits it. ERPNext's draft-and-submit flow does half the work.
  6. 6Review the log weekly for the first monthYou'll quickly see which questions the agent handles well and where it guesses.

A typical first use case: a sales or finance assistant that answers "what's overdue from this customer?" or "which purchase orders are pending receipt?" from live ERPNext data, without anyone exporting a spreadsheet to a public chatbot. More patterns are in our guide to deploying enterprise AI agents on ERP.

Cost

What does a self-hosted AI agent cost to run?

There's no single number. Anyone quoting one without seeing your workload is guessing. The cost has the five parts listed here.

Put them next to what you pay today in per-seat or per-token cloud fees for the same work. Self-hosting tends to make sense when usage is heavy and steady. Cloud plans tend to win when it's light or spiky.

  • Hardware or GPU hosting: the biggest line, set by model size and concurrent users
  • Power and cooling if the GPU sits in your office or data centre
  • Setup: model choice, workflows, ERP connection, testing
  • Upkeep: updates, monitoring, backups, and someone who owns it
  • Staff time to review agent output, mostly in the first months

Limits

When should you not self-host?

No one to run it

An unpatched AI server with ERP access is worse than a well-run cloud plan.

You need the very best model

For hard reasoning, long coding tasks or complex analysis, today's best cloud models are still ahead of what most companies can run in-house.

Usage is small

A handful of people using AI a few times a day rarely justifies a GPU server.

You can't wait

A cloud plan is ready today. A private stack takes planning and testing.
1

Private agent

For ERP, HR, payroll and customer data.

2

Cloud business plan

With no-training terms, for everyday writing and research.

That hybrid is what we recommend most often. FinByz was founded in 2016 and has delivered 200+ projects across 15+ countries with a 50+ member in-house team. We set up self-hosted n8n, private AI agents and the ERPNext connection end to end, with read-only users, approvals and logs from day one, through our custom AI development and ERPNext AI teams. Tell us about your data and workload first, and we'll say honestly whether self-hosting fits.

Talk to our Experts

Ready to Scale with Smart Tech? Let's Talk.

Transform your business processes with AI-powered automation and expert implementation.

200+ projects since 2016
Questions buyers ask

Frequently Asked Questions

Find answers to common questions about our services