AI features inside existing products

Adding AI to something already in use is mostly not a model problem. The model is a hosted API call. The problem is that there are users with expectations, screens that are currently fast and should stay fast, a database with a shape, and a support process that will receive the complaints when an answer is wrong. A greenfield demo has none of those and is therefore not evidence about anything.

The first question is narrow and it decides the rest of the design: which decision is this feature allowed to make? A feature that drafts something a person confirms can be wrong occasionally at almost no cost. A feature that decides something and acts on it turns every wrong answer into an incident that somebody has to discover, explain and undo. Those two need different architectures, and the second one needs a much better reason to exist.

Djob is a two-sided recruiting SaaS built over about six months on PostgreSQL and Base44, live at djob.agency with public plans starting at $29/month. Its matching layer is the AI feature, and three decisions inside it are the ones worth reusing anywhere.

The first is what gets embedded. Candidate and job records are not stored as one document each and embedded as a blob. They are broken into structured statement parts, and OpenAI's text-embedding-3-small runs over those parts. Embedding a whole CV gives you one number and no way to say what it was responding to. Embedding separate statements gives you something you can point at when a recruiter asks why a candidate surfaced.

The second is that similarity does not decide. Cosine similarity produces a closeness score, and a score is an input to a decision, not the decision itself, because a candidate can read as extremely close to a role and still fail a hard requirement. So plain pass or fail business rules sit around the score, and the reason is kept with the result. A rule written in code can be read, argued with and changed by whoever owns the business logic. A threshold buried in a prompt cannot.

The third is where the work happens. Match views read from snapshot tables rebuilt daily rather than recomputing every candidate against every job when a page opens. Same inputs, same outputs, computed once on a schedule instead of once per view, which is what makes the screen a database read.

One more thing worth copying is how the models are split. Two different jobs get two different models: text-embedding-3-small handles similarity, and a separate path on gpt-4o-mini handles the analysis and drafting work — reading a record and producing prose a human then reviews. Keeping them apart is deliberate. Similarity has to be cheap, stable and comparable across records, which is exactly what a small embedding model is for. Drafting has to read as language, which embeddings cannot do at all. Teams that reach for one general-purpose chat model to do both end up paying chat prices for a numeric comparison and getting a score they cannot reproduce next week.

This is the pattern that makes AI features safe to leave switched on in a product with paying users. The model does the part it is genuinely better at: reading unstructured input, finding candidates, proposing text, spotting similarity. Deterministic code does the part that has to be correct and defensible: eligibility, thresholds, ordering, permissions, anything with a number attached that somebody will be held to.

It also changes how the feature appears in the interface. Output presented as a suggestion, in a draft state, with an obvious way to accept, edit or discard it, sets expectations honestly and gives you a stream of corrections to learn from. Output presented as an authoritative answer invites the user to trust it exactly as much as it does not deserve.

One more piece belongs in the first version: a path where the feature declines. Returning nothing, or returning the plain rules-based result, is nearly always better than producing a confident answer built from too little signal. Products that cannot say they have nothing are the ones that generate the support tickets.

It will be wrong, and the plan for that is a feature, not an operational afterthought. Four things cover most of it. Log enough to reproduce: the input, the output, and which model, prompt version and embedding version produced it, so a complaint three weeks later is investigable rather than a shrug. Bound what a wrong answer can touch: draft states, confirmation steps, and limits on what the feature is allowed to write.

Then give the product a correction path that a normal user can find, and record that a correction happened instead of silently overwriting the original. Corrections are the most valuable data the feature produces, and a system that discards them can only be improved by guesswork.

The fourth is unglamorous and gets skipped: a rebuild you can actually run. Changing an embedding model, a prompt or a scoring rule invalidates everything precomputed under the old one, and mixed-vintage data is a genuinely nasty class of bug because it looks like intermittent model flakiness. A scheduled snapshot rebuild, as in Djob's daily job, means the answer to that is running it again rather than writing a migration under pressure.

Two numbers should exist before anything is built: what one use of the feature costs, and how many times a day it will happen. Multiply them and the design either survives or it does not, and finding out at that point is free. Prototypes that were abandoned as too expensive were usually not expensive because of the model. They called it on the read path, once per page view, per user, forever.

The fixes are architectural. Embed on write rather than on read, so the cost is paid once per record instead of once per view. Precompute rankings on a schedule. Cache aggressively where inputs have not changed. Use the smallest model that actually separates your data, which for Djob's structured statements is text-embedding-3-small rather than anything larger.

Latency deserves the same treatment, because it lands on somebody doing their job. A screen backed by a snapshot table is a database read. A screen that calls a model when it opens is a network round trip plus a model wait, sitting on the critical path of a recruiter working through a list. And since the provider will eventually be slow or unavailable, the feature needs a defined behaviour for that: serve the last good snapshot, or fall back to the rules-only result, rather than showing a spinner and blocking the work.