Short answer: you almost never need to rebuild your app to add AI. In most cases AI arrives as a new service that your existing backend calls over an API, the same way you already call a payment or email provider. Start with one narrow feature, such as a summary, a classification or a draft reply, and only add retrieval, background processing or tool access when that feature actually needs it.

Key points

  • Your current frontend, database and auth usually stay as they are. The AI calls go through your backend, never straight from the browser with your API key.
  • Five patterns cover most features: a direct API call, structured outputs, retrieval over your own data (RAG), background jobs, and tool calling.
  • If you run Postgres, you may not need a separate vector database. The open-source pgvector extension adds vector search to the database you already have.
  • The real new work is around the model: a test set of real examples, logging, cost limits, and a fallback for when the AI is slow or wrong.
  • Give the model the least access it needs. Read-only first, and human approval for anything that sends, pays or deletes.

The five integration patterns

Anthropic's engineering team, writing about how its customers build with language models, recommends "finding the simplest solution possible, and only increasing complexity when needed," and notes that for many applications "optimizing single LLM calls with retrieval and in-context examples is usually enough" (Anthropic, Building effective agents). We agree, and the patterns below are ordered from simplest to most involved.

PatternGood forWhat changes in your stackEffort
1. Direct API callSummaries, rewrites, translations, draft repliesOne new backend endpoint, an API key in your secrets storeLow
2. Structured outputsPulling fields from emails, PDFs or forms; tagging and routing ticketsSame as above, plus a JSON schema and validationLow
3. Retrieval (RAG)Answers based on your own documents, help center or product dataAn indexing job, a vector column or store, a search step before each callMedium
4. Background jobs and batchProcessing thousands of records, nightly reports, re-tagging old dataA queue or scheduled worker, result storageMedium
5. Tool callingLetting the AI look up an order, create a ticket, or check stockNarrow internal APIs the model may call, permission checks, an audit logMedium to high

1. Direct API call

This is the starting point for most teams. A user clicks "Summarize" or "Draft reply", your backend sends the relevant text to a model provider's API with a short instruction, and you show the result. Nothing else in your architecture moves.

Keep two habits from day one. First, call the model from your server, never from the browser, so your key stays secret and you can add limits per user. Second, wrap the call in a small internal service or module. When you want to switch providers or models later, you change one place.

2. Structured outputs

Many useful AI features aren't chat at all. They read messy input and return clean data your app can store: an invoice total, a support ticket category, a lead's company name.

Major providers now let you require the response to follow a schema. OpenAI's documentation describes Structured Outputs as a feature that "ensures the model will always generate responses that adhere to your supplied JSON Schema" (OpenAI, Structured model outputs, checked October 2026). You still validate the result, because a well-formed answer can still be the wrong answer. But you no longer write fragile code that parses free text.

3. Retrieval over your own data (RAG)

If the feature has to answer from your documents, policies or product catalog, you add a retrieval step. Your content is split into passages and indexed. When a question comes in, the app finds the most relevant passages and sends them to the model along with the question. This is retrieval-augmented generation, or RAG.

The index doesn't have to be a new system. pgvector, an open-source Postgres extension, lets you "store your vectors with the rest of your data," including adding a vector column to an existing table (pgvector on GitHub). For a team already on Postgres, that means no new database to run, back up and secure.

Most RAG problems come from the data, not the model. Outdated or conflicting documents produce confident wrong answers. Plan time to clean the sources and to decide who keeps them current.

4. Background jobs and batch processing

Some AI work doesn't need to happen while a user waits: tagging every past support ticket, summarizing yesterday's calls, generating product descriptions for a catalog import. Move it to a queue or a scheduled worker, the same way you would handle sending a large email campaign.

Providers price this differently too. OpenAI's Batch API offers a "50% cost discount compared to synchronous APIs" with each batch completing within 24 hours (OpenAI, Batch API, checked October 2026). Anthropic's Message Batches API also reduces costs by 50%, and its docs say most batches finish in under an hour (Anthropic, Batch processing, checked October 2026). If a feature can wait, running it in batch is one of the easiest ways to cut its cost.

5. Tool calling

Here the model can ask your app to do something: look up an order, check a calendar, create a ticket. You define a small set of functions, the model decides when to call them, and your code runs them and returns the result.

This is powerful and it's where most of the risk sits. The OWASP Top 10 for LLM applications lists "excessive agency" as a vulnerability and names its root causes as "excessive functionality; excessive permissions; excessive autonomy" (OWASP GenAI Security Project, LLM06). In practice: expose only the functions the feature needs, run them with the signed-in user's permissions, and put a human approval step in front of anything that changes data or contacts a customer. If you are weighing whether the feature needs this level of autonomy at all, our guide to AI agents vs chatbots vs automation can help you decide.

What actually changes in your stack

For a typical web or mobile app, the list is shorter than people expect:

  • A model provider account and key, stored with your other secrets and never shipped to the client.
  • One AI module or service in your backend that holds prompts, schemas and provider calls.
  • Optionally, a vector column or index if you use RAG.
  • Optionally, a worker or queue for background and batch work. Many apps already have one.
  • Logging and cost tracking that records which feature called which model, with how many tokens, and what came back.
  • Limits, such as maximum requests per user, maximum spend per day, and timeouts.

Your UI changes too, in small ways. AI responses take longer than a database read, so plan for loading states or streaming text. Show users when something was written by AI and let them edit it before it is saved or sent.

Data and privacy questions to settle first

Before you send customer data to any provider, read its data policy and decide what you are comfortable sharing. For example, OpenAI states that "data sent to the OpenAI API is not used to train or improve OpenAI models (unless you explicitly opt in)," and that abuse monitoring logs are by default "retained for up to 30 days" (OpenAI, Data controls, checked October 2026). Other providers publish similar pages, and the terms differ, so check the one you plan to use.

Then decide what leaves your system. Often you can strip names, emails and account numbers before the call and add them back afterward. If your customers are in regulated sectors or specific countries, involve whoever owns compliance early rather than after launch.

Test it like any other feature, with real examples

The step teams skip most often is evaluation. OpenAI's documentation calls writing evals "an essential component to building reliable applications," especially "when upgrading or trying new models" (OpenAI, Working with evals).

You don't need a platform to start. Collect 20 to 50 real inputs, write down what a good output looks like for each, and run them every time you change a prompt, a model or the data. Keep the set in your repo next to the code. When the provider releases a new model, you'll know in an afternoon whether it's better for your feature or just different.

Keep running costs under control

Every call costs money, and costs scale with usage in a way your hosting bill may not. A few habits help:

  • Send only the context the model needs, not the whole record or document.
  • Use a smaller, cheaper model for simple steps like classification, and a larger one only where quality demands it.
  • Use prompt caching for long instructions or documents you send repeatedly. Anthropic's docs describe it as a way to "cut costs and latency" (Anthropic, Prompt caching).
  • Move anything that can wait into batch jobs.
  • Set a daily spend limit and an alert before launch, not after the first surprising invoice.

For how the build and running costs add up, with a worked sample calculation, see how much an AI agent costs to build and run.

A sensible rollout order

  1. Pick one feature with a clear owner and a clear "good result", such as a ticket summary for support agents.
  2. Build it with the simplest pattern that works, usually a direct call or structured outputs.
  3. Ship it to internal users first, with logging on and a way to flag bad answers.
  4. Build the test set from what they flag.
  5. Open it to customers behind a feature flag, with a fallback if the AI call fails or times out.
  6. Add retrieval, batch or tools only when the feature clearly needs them.

FAQ

Do I need to rewrite my app to use AI?

Usually not. AI features are added as backend calls to a model provider, much like a payment or email integration. A rewrite only makes sense if the app has other problems, such as no API layer at all or a stack nobody can maintain.

Do I need a vector database?

Only if the feature answers from your own documents or data (RAG). Even then, if you already use Postgres, the pgvector extension may be enough before you consider a dedicated vector database.

Which AI provider should I choose?

Pick based on quality for your specific task, data terms, region and price, and test two options against your own examples. Keep provider calls in one module so switching later is a small change.

How long does it take to add a first AI feature?

It depends on the feature and the state of your codebase. A single summary or extraction feature in a well-structured app is a much smaller job than a RAG assistant over messy documents. The fastest way to find out is to scope one feature in detail before committing to a roadmap.

Is it safe to let AI take actions in my app?

It can be, with limits. Give it the minimum functions and permissions, run actions as the signed-in user, log everything, and require human approval for anything that sends, pays, deletes or changes customer records.

Want a second opinion on your app?

Tell us what your app does and which task you want AI to help with in a free 30-minute call. We'll suggest the simplest pattern that fits and what it would change in your stack. See how we work on existing products on our AI integration page.

Sources