In my previous article, 9 Things I Really Hate About AI, I complained that everyone suddenly seems to be an AI expert, and that all those experts together make a lot of noise. So I'm going to spend the next few posts explaining some AI basics for business people. When a manager goes looking for this stuff, what they find is mostly wrong or written for engineers. Neither helps when you have a budget decision to make on Thursday.
I'm starting with a big one, which is how (and why) you get your company's data into a large language model.
A general LLM knows nothing about you. I picture it as a very smart new hire who has read half the internet but has never seen your price list, your contracts or one single customer email. Hand that person your internal documents, your customer conversations, product sheets and industry reports, and suddenly they're useful. With the model it's the same story. Give it your data and it can answer questions about your business, take over parts of the work and spot things a generic chatbot would never see.
You've probably done a small version of this already. Ever pasted some background into a ChatGPT question, or dropped a PDF in there? Then you've fed it your data. For personal use, fine. When you build something for customers or staff, you need to be a bit more grown up about it, and there are seven ways people do that today. Roughly from easy to hard:
- Adding information directly to your question
- Adding a file to the model (uploading a document)
- Function calling (connecting an API)
- Retrieval Augmented Generation (RAG)
- Cache Augmented Generation (CAG)
- Fine tuning an existing model (create your own LLM)
- Training a new foundational model (compete with OpenAI, Anthropic, etc.)
Everybody starts here. You type a question and paste some context underneath so the model has more to go on. Quick and dirty, and it works. If you need something fast and don't care much about security or reuse, go for it.
It does run out of road. Past a certain amount of text the model starts skipping bits, and it pays most attention to whatever came last. Paste in 20 pages with your question at the bottom and I'd bet the first few pages get ignored.
And you're shipping possibly sensitive data off to a third party. Yes, even if they promise not to store it. Brainstorming is fine. Board reports and customer data, no.
There's a technical ceiling too, the context window. That's the maximum number of tokens (chunks of words or characters, roughly) a model handles in one request, and it limits this method and the next three. On some models it's quite small, so "just dump all our data in" is not an option. You also pay per token, so every fat prompt costs you, and at hundreds or thousands of requests a day that adds up fast. RAG and CAG were invented partly for that reason.
For most people the next step is to upload a PDF, a policy or a user manual and start asking questions about it. ChatGPT (Pro or Enterprise) and Claude make it very easy. Drop the file in the chat and the model appears to "read" it.
It doesn't actually read anything the way you and I do. Under the hood the text gets cut into chunks of usually 200 to 500 words, those chunks become vectors, and when you ask something the system fishes out the ones that look most relevant. You don't see any of that happening.
People feel this is safer than pasting text. I don't see why. Your file still travels through ChatGPT's servers, or whichever third party tool you use, and unless you pay for enterprise privacy controls (and actually read the fine print) your confidential documents shouldn't go there.
Great for reviewing or summarizing a document on the fly. If you want a company wide assistant or an internal knowledge bot, you'll want function calling or RAG.
Of everything that happened to LLMs lately, this is one of my favorites. Instead of having the model guess from plain language, you give it structured access to your own systems and tools, and it can actually do stuff.
In my head the LLM is the brain and your APIs are the hands. You describe which functions exist, things like "get customer order history" or "calculate monthly revenue", and the model figures out when to call which one and with what.
It doesn't need to understand your backend, you only describe the interface. The classic demo is someone asking "What's the weather in New York City?" and the model calling a weather function for that location. Plug in your own company systems and it works the same way, only with order histories instead of weather.
OpenAI's GPT, Google's Gemini and Anthropic's Claude models all support this out of the box, so hooking one up to your CRM, ERP or support desk is very doable. Microsoft Copilot does something similar inside Excel, Outlook and Teams. With open source models you'll do a lot more engineering yourself, although LangChain or LlamaIndex take a good chunk of that off your plate.
RAG is the industry standard for this at scale, and most enterprise AI you'll run into is built on it. You take your documents, policies, manuals and data and cut them into chunks of typically 200 to 500 words. Each chunk is turned into a vector embedding, which is just a numerical representation, and stored in a vector database like Pinecone, Weaviate or Qdrant. When a user asks a question, the question becomes a vector too. The system looks up the most similar vectors in the database, feeds those chunks to the LLM as context, and the LLM writes its answer from there.
The reason I like it is that your data stays outside the model. Nothing gets trained. The model just looks things up in real time, so you can change your data whenever you want without retraining anything, and you stay in control.
Most enterprise chatbots, document Q&A tools and internal knowledge assistants run on it. It's secure because you hold the data, it scales to millions of documents, and the cost is reasonable.
Newer, less common, picking up speed. The idea behind CAG is to remember prompts and answers that keep coming back, so the model doesn't write the same reply from scratch a hundred times.
Say your support team gets the same 50 questions every day. There's no reason to push each of them through the whole LLM pipeline. Store the answers once and serve them straight away when a similar question shows up. You make fewer API calls (cheaper), skip the generation time (faster) and give everyone the same answer. Someone does have to look after that cache and decide when an answer has gone stale, which is more work than it sounds.
Anthropic and some other providers now offer prompt caching as a built in feature, which helps.
Here you take a pretrained model like GPT-4, Llama or Mistral and train it further on your own data. More work than RAG. For specialized tasks it can pay off.
There are a few flavors. Light fine tuning (LoRA, QLoRA) leaves the model itself alone and adds small adapter layers that learn your patterns, which is a lot cheaper and quicker. Full fine tuning retrains the whole thing, needs serious compute (dozens of GPUs) and people who know what they're doing, and most companies have no need for it. Instruction tuning teaches the model to follow your instruction formats or patterns from your domain; you see that a lot in customer service bots and internal tools.
The last flavor, deep customization, is nearly building your own model. You take a base checkpoint (LLaMA, Mistral, DeepSeek) and keep training on huge datasets, think hundreds of millions of tokens or more. Now you own a model variant, and with it the MLOps, the evaluation pipelines and the guardrails. AI native companies live here. If AI isn't your product, you probably shouldn't.
Unless you are OpenAI, Google, Mistral, Meta or Anthropic, just don't. We're talking tens (or hundreds) of millions, plus GPU infrastructure, research and talent most companies will never have. OpenAI pays its AI engineers more than some companies pay their CEOs.
You also need ridiculous amounts of data. Bloomberg went for it anyway with BloombergGPT. They collected 363 billion finance specific tokens from their own database and added 345 billion general tokens from public sources like Wikipedia.
Plenty of companies say they built their own model. Most of them fine tuned an open one, which is fine, but let's not pretend that's building from scratch.
There's no neat answer, but this is roughly how I'd decide. If you need it fast and cheap, paste context into the prompt or upload the document, and live with low safety and mediocre quality. For enterprise quality and safety, start with RAG and function calling. Fine tuning is for AI native companies and very specialized domains. And unless you're OpenAI, DeepMind or Meta, don't build your own model.
Questions about any of this? Drop them in the comments, or keep following my tech updates. I try to write these for business people wading through all the AI noise, not for other engineers.
In the next one I'll look at image and video generation, how it works, what you can do with it today and what's mostly hype.
Updated on 12 October 2026.
