Custom AI Development: Architectures, Costs, and the Path to Production

Custom AI development is the work of building AI into your own software: applications on large language models (LLMs), retrieval-augmented generation (RAG) over your data, agents that act through tools, fine-tuned models, and classic machine learning trained on your records. In BD Emerson's 2026 planning ranges, a production feature on a prompted API model costs $50,000 to $200,000 to build, a RAG assistant $150,000 to $500,000, and an agent that acts inside a regulated workflow $300,000 to $1.2 million. First-year run cost depends on model tier, volume, and the evaluation and oversight you keep in place, and the model tier alone can move inference cost by a factor of 100.

Most custom AI solutions run on bought models: you rent a model through an API or host an open-weight one, and you build the retrieval, tools, evaluation, guardrails, and integration around it. That build work holds the cost, the risk, and the differentiation.

When should you build AI instead of buying it?

Buy an AI product when the use case is common across companies and a vendor covers it under contract terms your data rules allow; build when the AI has to work inside your systems and decisions. BD Emerson's decision rule counts four tests:

• Systems of record: the AI must read and write your core systems, such as claims, policy, loan servicing, or care management, through your APIs and permissions.
• Regulated decisions: its outputs feed decisions an examiner reviews, so you need evaluation evidence and audit trails you control.
• Data boundary: the data cannot leave your environment, or retrieval must respect entitlements a vendor product cannot see.
• Differentiation: the workflow is where you compete, so a product your competitors can buy gives you parity at best.

If two or more tests apply, build on bought models. If none applies, buy the product. If exactly one applies, buy and extend the product through its APIs, or pilot a build and the product side by side on the same cases. Meeting notes, coding assistants, general chat over documents, and standard customer service channels usually pass none of the tests, and vendors spread the engineering for them across thousands of customers.

Custom AI architecture options compared

Five architecture patterns cover nearly every custom AI solution, and the right one depends on whether the model needs your knowledge, your behavior, your systems, or your boundary.

Retrieval changes what a model knows and fine-tuning changes how it behaves, so most programs should build retrieval first and fine-tune only when format or tone remains the bottleneck; our RAG vs fine-tuning guide covers that choice in depth. Agents reach your systems through tools, and the Model Context Protocol (MCP) is the common way to expose them. Anthropic introduced MCP in late 2024 and donated it to the Agentic AI Foundation, a directed fund under the Linux Foundation, on December 9, 2025, and the MCP versioning page lists 2026-07-28 as the current specification revision as of October 2026. Classic machine learning remains the better fit for structured prediction from labeled history, such as claim severity or transaction fraud scoring, and it costs less to run.

How much does custom AI development cost?

Custom AI development costs $50,000 to $1.5 million to build in BD Emerson's 2026 planning ranges, and first-year run cost adds 20 to 35 percent of build for evaluation, monitoring, model updates, and support, plus inference and hosting that scale with volume. AI systems run above the 15 to 25 percent we plan for conventional software because model versions change underneath you and every change needs re-evaluation.

The software around the model, its screens, integrations, identity, and data pipelines, prices like any build, and our custom software development team estimates it with the same team, rate, and duration formula.

What do AI models cost to run?

Standard API pricing at OpenAI, Anthropic, and Google runs from $0.10 to $10 per million input tokens and $0.50 to $50 per million output tokens, checked against each provider's pricing page on October 15, 2026. A token is roughly a short word or part of one, and every request pays for the tokens it sends and receives.

The figures come from OpenAI's pricing page, Anthropic's pricing page, and Google's Gemini API pricing page. Cached input and batch processing cut them further: OpenAI lists cached input for GPT-6.1 Sol at $0.10 per million tokens, Anthropic lists cache hits for Claude Sonnet 5.5 at $0.10, and Anthropic and Google price batch requests at 50 percent of standard. Google's page already lists increases for Gemini 4 Argon on December 1, 2026 and Gemini 3.8 Flash on January 1, 2027, so build price changes into the business case. Embeddings cost little: OpenAI lists text-embedding-3-small at $0.02 per million tokens, so embedding a 50-million-token corpus costs $1.

A worked example shows why tier choice matters. A RAG assistant that answers 5,000 questions per business day handles 105,000 a month over 21 business days, and each answer sends 6,000 input tokens and returns 500 output tokens.

The mid-tier line works out as (6,000 × $2 + 500 × $10) ÷ 1,000,000 = $0.017 per answer, and 105,000 answers × $0.017 = $1,785 a month. Self-hosting changes the unit from tokens to GPU hours. Google Cloud lists its a3-highgpu-8g machine, with eight NVIDIA H100 GPUs, at about $88.49 an hour on demand and $38.86 an hour on a three-year commitment as of October 2026, so one machine running all year costs about $775,000 on demand or $340,000 committed (8,760 hours at each rate). A small open-weight model on one NVIDIA L4 GPU (g2-standard-8, about $0.85 an hour) costs about $7,500 a year. Self-hosting pays when volume is high and steady or when data cannot leave your boundary.

How much does it cost to build an AI agent?

An AI agent that reads from two or three systems and drafts work for a person to approve costs $75,000 to $250,000 to build in BD Emerson's 2026 planning ranges. An agent that writes to systems of record behind human approval costs $300,000 to $700,000, and multi-agent systems or agents with more autonomy over regulated decisions run $700,000 to $1.2 million or more. Tool count, the systems touched, approval design, and evaluation depth set the position in each range.

Tokens per task are the agent's unit cost. An agent task that makes 20 model calls, each sending 8,000 input tokens and returning 500 output tokens, uses 160,000 input and 10,000 output tokens. At mid-tier prices that costs $0.42 per task (160,000 × $2 + 10,000 × $10, divided by 1,000,000), or about $0.21 when 70 percent of input tokens come from cache at $0.10. At frontier prices the same task costs $2.10, or about $1.09 with caching at $1.00. At 10,000 tasks a month, the bill runs $2,072 to $21,000 depending on tier and caching, and in our planning the evaluation, monitoring, and human review around the agent cost more than the tokens.

Who you need on a custom AI team

A production AI team mixes software engineering, AI engineering, data, evaluation, security, and domain judgment, and the domain expert is the role a pilot can skip and production cannot. Allocations are BD Emerson planning figures for a regulated workflow agent.

From pilot to production: a realistic timeline

A regulated workflow agent takes 5 to 9 months from use case to full rollout, and the stages that make it defensible take longer than the prototype. The durations are BD Emerson planning figures.

The stages add up to 22 to 40 weeks. Skipping the baseline is the most expensive shortcut, because without it nobody can prove the system worked.

Evaluation, guardrails, and red teaming

Evaluation makes every other control measurable, so build the test set before the system and run it on every prompt, retrieval, or model change. A useful set holds 200 to 500 real cases with expected outcomes written by domain experts, and it scores task accuracy, groundedness against retrieved sources, policy adherence, correct refusals, latency, and cost per task against thresholds the business owner signs. Rerun it whenever a vendor updates a model, because hosted models change on the vendor's schedule.

Guardrails enforce in code what the evaluation measures: input screening for injected instructions, output validation against schemas, allow-listed tools with scoped credentials, human approval for consequential writes, and rate and spend limits. Red teaming then attacks the finished system the way an adversary would, before launch and after major changes, and our guide to AI red teaming covers how those engagements run.

Securing custom AI against the OWASP Top 10 for LLM Applications

The OWASP Top 10 for LLM Applications 2025 lists the ten risks OWASP's GenAI Security Project ranks highest for LLM applications, and each maps to a control you can build into a custom system.

OWASP's GenAI Security Project added a Top 10 for Agentic Applications in December 2025, with risks such as agent goal hijack (ASI01), tool misuse (ASI02), and identity and privilege abuse (ASI03), and agent builds should test against both lists.

Governance: ISO 42001, NIST AI RMF, and sector rules

Governance makes a custom AI system defensible to auditors, examiners, and your board, and two frameworks do most of the work. ISO/IEC 42001:2023, published in December 2023, specifies an AI management system: the policies, roles, risk and impact assessments, and controls an organization runs for AI across its life cycle. An accredited certification body issues ISO 42001 certificates; BD Emerson implements ISO 42001 and performs internal audits against it, and it is not a certification body. The NIST AI Risk Management Framework, released January 26, 2023, organizes the work into four functions, Govern, Map, Measure, and Manage, and NIST's Generative AI Profile, NIST AI 600-1, released July 26, 2024, applies them to generative AI. NIST says AI RMF 1.0 is being revised under the White House AI Action Plan.

Sector rules add specifics. Banks apply the Federal Reserve's SR 11-7 model risk guidance of April 4, 2011, which expects validation independent of development and "effective challenge" from objective, informed reviewers, whether a model is built or bought. Insurers answer to the NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers, adopted December 4, 2023, which expects a written AI systems program with governance, risk management, internal controls, and oversight of third-party AI vendors and data.

Why AI pilots stall and how to measure ROI

AI pilots stall on the organization around the model: ungoverned data, no named model risk owner, no ROI baseline, a security review that starts too late, and integration work nobody scoped. Our analysis of why AI pilots stall covers each failure point. Measure return against the baseline you set in the first stage, and count two kinds: the P&L return in cost, cycle time, and loss or error rates, and the change in intangible assets the program creates, such as software, data, and redesigned processes, which our AI ROI framework shows how to value.

Frequently asked questions

What is custom AI development? Custom AI development is building AI into your own software: large language model applications, retrieval-augmented generation over your data, agents that act through tools, fine-tuned models, or classic machine learning trained on your records. Most custom AI runs on bought models, and the custom work is the retrieval, tools, evaluation, guardrails, and integration that make the model useful and safe in your environment.

How much does it cost to build an AI agent? In BD Emerson's 2026 planning ranges, an agent that reads from a few systems and drafts work for approval costs $75,000 to $250,000 to build, an agent that writes to systems of record behind human approval costs $300,000 to $700,000, and multi-agent or more autonomous systems in regulated workflows run $700,000 to $1.2 million or more. Tool count, systems touched, approval design, and evaluation depth set the position.

How much does AI cost to run each year? Plan first-year run cost at 20 to 35 percent of build for evaluation, monitoring, model updates, and support, plus inference. Inference ranges widely: a RAG assistant answering 105,000 questions a month costs about $1,071 a year on an efficient-tier model, $21,420 on a mid-tier model, and $107,100 on a frontier model at October 2026 list prices. Self-hosting replaces token costs with GPU hours.

Should we use RAG or fine-tuning? Use retrieval-augmented generation when the model needs your knowledge, such as policies, contracts, or records that change, and fine-tuning when it needs consistent behavior, such as a fixed format, tone, or narrow task at high volume. Build retrieval first in most cases, measure it against a real test set, and fine-tune only if format or behavior is still the bottleneck in production.

How long does it take to build a custom AI solution? A prompted feature inside an existing application takes 6 to 12 weeks, and a RAG assistant takes 3 to 6 months. A regulated workflow agent takes 5 to 9 months from use case to full rollout, because the baseline, shadow pilot, security review, red teaming, and model risk sign-off take longer than the prototype. Those durations are BD Emerson planning figures.

Do we need ISO 42001 certification to deploy custom AI? ISO/IEC 42001 certification is voluntary, and you can deploy custom AI without it. Customers, partners, and examiners ask for evidence of an AI management system, and certification against ISO 42001 is one way to provide it. An accredited certification body issues the certificate; BD Emerson implements ISO 42001 and performs internal audits, and it does not certify.

If you're scoping a custom AI build, our AI implementation team can run the baseline and an evaluation-first prototype, then give you the architecture, the build range, and the run cost before you commit to production.