AI integration for existing products — not a demo chat
I help companies integrate large language models and related AI capabilities into software they already run. The default is still deterministic code. A model enters only where the task is ambiguous enough that probability earns its latency, cost, and error rate. That usually means structured outputs, retrieval over the company's own data, tool use behind validation, and a human path when the happy path does not hold. I have shipped AI-native product surfaces and an enterprise LLM benchmarking platform — the work is about choosing the model against a use case, not decorating a landing page with 'GPT'. If you need a wrapper around a chat API, you do not need me.
Who this is for
- SaaS teams adding AI to an existing product, not starting an 'AI company'.
- Internal platform teams who want retrieval or copilots on company data.
- CTOs who need a build-vs-buy and model-selection decision before procurement.
- Product companies that already have a prototype and need it to survive traffic and bad input.
Problems this work is for
- A chatbot was demoed. Now it has to write into the same systems the rest of the product uses.
- Hallucinated fields are reaching users or downstream APIs.
- Inference cost appeared in the bill before anyone defined success.
- The team is arguing RAG versus fine-tuning without a task definition.
What I actually implement
- Use-case validation: whether the task is ambiguous enough to justify a model.
- Model selection against quality, latency, cost, privacy, and hosting constraints.
- RAG design: chunking, retrieval evaluation, citations, and when not to retrieve.
- Tool use and agents only where a workflow has steps a model should not hard-code.
- Structured outputs, validators, and prompt architecture that can be tested.
- Observability: traces, evals, cost, and failure taxonomies.
How I integrate a model
Start with the workflow, not the model. Name the input, the required output shape, the cost of being wrong, and who is accountable. Prove the risky cases on a vertical slice with representative data. Only then connect tools, retrieval, and production traffic — with a circuit breaker and a human fallback. Deterministic logic stays in front of anything that can be expressed as rules.
Seven factors before adding an LLM to an existing product
If you cannot answer these, you are not ready to put a model on the request path.
1. Task ambiguity
If a rules engine or a form can do it, do not pay for tokens. Models earn their keep on classification, extraction, drafting, and retrieval over messy text — not on calculating tax.
2. Cost of being wrong
A wrong summary is not the same as a wrong payment instruction. High-cost errors need validators, dual control, or a human in the loop. Do not 'prompt better' your way around that.
3. Data boundary
What may leave the VPC? Customer content, secrets, and regulated records often cannot go to a hosted API. That decision precedes model quality.
4. Grounding
If the answer must come from your corpus, you need retrieval and citations, not a larger base model. Fine-tuning is for style and task format, not for facts that change weekly.
5. Output contract
Production software consumes JSON, tool calls, and database writes. Free-form chat is a UI, not an interface. Schema, repair, and reject paths are part of the design.
6. Latency and cost envelope
A 8-second, $0.04 call in a background job is a different product from the same call on page load. Measure both before you pick a frontier model.
7. Operability
You need traces, eval sets, and an owner. A prompt in a dashboard with no tests is not an integration.
How AI integrations fail in production
- Chat UI bolted onto a product with no write-path design.
- RAG over a dump of PDFs with no retrieval evaluation.
- Trusting chain-of-thought as a security boundary.
- Fine-tuning to 'remember' facts that belong in a database.
- No fallback when the provider has an outage or starts refusing the domain.
Deliverables
- A go / no-go on the use case, with the cheapest architecture that could work.
- A production path: model, retrieval, tools, schema, and fallback.
- Eval cases for the failures you actually care about.
- Instrumentation for quality, latency, and cost after release.
Selected work
- Genie Platforms — AI-native sales copilots aligning outbound, nurture, and RevOps choreography without abandoning instrumentation.
- TIVA — Ambient voice and text retrieval that meets teams where they work — not another siloed search screen.
- Kwizie — Generative quizzes from any video corpus — lecture-length MOOCs or quick smartphone captures.
Related services
- AI agents — Tool-using agents for bounded workflows, with permissions and evals — not an autonomous employee.
- AI automation — Remove the manual bottleneck in a live operation: rules first, models only where the input is messy.
- Software architecture — System design for live products: what should exist, where the boundaries sit, and how the system fails.
- Software engineering consulting — Senior engineering judgement for companies that need a principal-level partner, not a staffed agency team.
Related insights
Questions I am asked
RAG or fine-tuning?
If the facts live in documents or systems that change, retrieve them. Fine-tune when you need a stable output format, a domain style, or a smaller model to follow a task — not as a knowledge base. Many products need neither: a schema, a good prompt, and a validator are enough.
How much does an AI integration cost?
The engineering cost is usually a bounded discovery plus a slice, then production hardening — not a six-month 'AI transformation'. Inference cost is separate and dominates if you put a large model on a high-QPS path. I will estimate both; I will not quote a fake average.