Build vs. Buy for AI: A Decision Framework for Non-Technical Founders

Founders hear “proprietary AI model” and immediately think: competitive moat. The reasoning goes: if we train our own model on our data, competitors can't replicate it. That's sometimes true. More often it's not, because the data isn't as unique as it sounds, the model isn't as good as a foundation model trained at 100x the scale, and the team doesn't have the ML infrastructure expertise to maintain it. The result is 6–18 months of engineering time spent on a model that underperforms an API call that costs $0.01.

Training a model from scratch is an ML infrastructure project, not a product feature. It requires data at the right scale and quality, a team with model training and evaluation expertise, compute infrastructure, and an ongoing maintenance commitment because models degrade without retraining as the world changes. Most startups are not equipped for any of these things, and even the ones that are often find that the model they trained doesn't justify the investment compared to what they could have built on an API in a fraction of the time.

The founders who get this right start with a strong buy-first default and raise the bar for building as high as it will go. That doesn't mean you never build — it means you only build when you've exhausted the buy options and the case for custom is overwhelming. The framework below gives you the criteria.

The three options are: use an API (buy), fine-tune an existing model (buy and adapt), or train from scratch (build). Using an API means calling a foundation model — GPT-4, Claude, Gemini, Mistral, or similar — and engineering the prompt to get the output you need. The barrier is low, the iteration cycle is fast, and the models are extraordinarily capable for a wide range of tasks. The tradeoffs are cost at scale, a dependency on a third-party API, and limits on how much you can customize model behavior.

Fine-tuning takes an existing foundation model and continues its training on a narrower dataset to adjust its behavior for a specific task or domain. The actual requirements are higher than most founders realize: you typically need 1,000 or more high-quality labeled examples, the ability to evaluate model output consistently (which requires either human raters or an automated evaluation pipeline), and engineering capacity to run and monitor the training process. Fine-tuning doesn't give you a fundamentally different model — it adjusts a model's behavior in constrained ways. If the base model can't do the task at all, fine-tuning rarely fixes that.

Training from scratch requires everything fine-tuning requires, at orders of magnitude more scale: millions or tens of millions of training examples, significant compute budget, and a team with genuine ML research and infrastructure expertise. This option is appropriate for a small number of companies with genuinely novel data in a domain where foundation models have limited coverage. For everyone else, it's optimizing the wrong layer of the stack.

The right default is buy-first. Use the cheapest API model that meets your quality bar on your actual data, not on benchmark data. Run a real evaluation: take 100 examples from your production inputs and measure how well the cheapest available model handles them. Buy up to a more capable model only if the cheaper one genuinely fails. In most cases, GPT-3.5 class models handle the task adequately, the step up to GPT-4 class costs 10–30x more, and the quality difference isn't worth it for the specific task at hand.

Consider fine-tuning only when two things are simultaneously true: the general model consistently fails at your task on real inputs despite genuine prompt engineering effort, and you have more than 1,000 high-quality labeled examples to train on. If either condition is missing, the investment is premature. “Consistently fails” means failure rate is above your acceptable threshold on representative inputs — not just on edge cases you found in testing. Fine-tuning to reduce a 10% failure rate to 7% is probably not worth the investment. Fine-tuning to reduce a 40% failure rate on a core task might be.

Consider training from scratch only if your data is genuinely proprietary and unavailable to foundation model providers — meaning a task in a domain with no public training data, where the major foundation models consistently fail despite fine-tuning attempts, and you can hire or have the team to maintain the model over time. If the answer to “why can't you use an API” is anything other than “our data is genuinely novel and private and we've proven the API can't do the task,” you're probably building a distraction.

The decision follows from four questions: What is the task? What data do you have? What team do you have? What does the API evaluation show? If the task is text generation, summarization, classification, or extraction — use an API. If the API fails on your task and you have 1,000+ labeled examples and engineering capacity — consider fine-tuning. If you have millions of examples in a genuinely novel domain with no public training data coverage and a team that includes ML researchers — training from scratch is on the table. Everything else is buy, until proven otherwise.

The fake moat trap is worth naming explicitly: training a model on publicly available data in your domain does not give you a competitive advantage over a foundation model that was trained on the same public data at 100x the scale. Your 1-million-example healthcare dataset doesn't compete with a foundation model that ingested most of PubMed, every medical textbook, and clinical notes from dozens of institutions. The data volume asymmetry is too large. The only proprietary data that constitutes a genuine training advantage is data that is unavailable to the major foundation model providers — genuinely private, genuinely novel, and genuinely voluminous enough to shift model behavior.

The practical implication is that for most startups, the AI moat isn't in the model — it's in the workflow, the data flywheel, and the product experience built around the AI. A company that uses GPT-4 as a commodity component and builds an exceptional product around it is harder to displace than a company that spent a year training a model that performs comparably. The model is the means, not the moat.