07 mins

RAG in Laravel: Finally, an AI That Actually Knows Your Business

A guide to RAG in Laravel: embeddings, vector storage, semantic search with the AI SDK, and the chunking strategy most tutorials skip. Real code, real trade-offs.

Dan Newns

06.10.26

If you've read our overview post, you know the pitch for RAG: give the AI access to your data before it answers, so it's grounding its responses in reality rather than confident invention.

It's a simple concept. Genuinely powerful in practice. And - with the Laravel AI SDK - much more approachable than you might expect from the jargon surrounding it.

In this post, we're going deep. We'll walk through a complete RAG implementation from scratch: setting up embeddings, storing vectors, querying semantically, wiring it to an agent, and the thing that most tutorials quietly skip over - chunking strategy, which turns out to be the difference between RAG that works and RAG that actually works.

Real example code, real tradeoffs, honest assessment of where it gets difficult.

What Problem Are We Actually Solving?

Let's be precise about this, because the answer shapes every decision that follows.

LLMs are brilliant at reasoning and generation, but they only know what they were trained on. That training data has a cutoff date, doesn't include your private data, and knows absolutely nothing about your client's specific product, policies, or history.

When a user asks your support assistant "what's the return window for items purchased during a promotional period?", you don't want the AI guessing. You don't want it generating a plausible-sounding policy that happens to be wrong. You want it to find the actual policy document, read the relevant section, and answer from that.

That's RAG. And it's not magic - it's structured retrieval followed by generation, with the retrieved content passed as context. The AI doesn't "know" the answer; it reads your documents and synthesises one.

The Four Components You Need.

Before any code, understand the moving parts:

Embedding model. Converts text into vectors - numerical representations of meaning. You'll use an API for this (OpenAI' text-embedding-3-small is cheap and excellent; Gemini and Cohere have good options too). The same model must be used for both indexing your content and querying at runtime.

Vector store. Where your embeddings live. For most Laravel applications, this is your existing database with a vector column - PostgreSQL with pgvector, or MariaDB 11.7+. No new infrastructure needed.

Retrieval. At query time, the user's question is used to find the nearest neighbours in your vector store. Pass a plain string and the SDK embeds it automatically.

Generation. The retrieved documents are passed as context to the LLM, which generates an answer grounded in them. Your system prompt instructs it to answer only from what was retrieved, not from general knowledge.

The Laravel AI SDK handles all of this cleanly. Let's build it.

Setup.

1composer require laravel/ai
2php artisan vendor:publish --provider="Laravel\Ai\AiServiceProvider"
3php artisan migrate

copied!

The publish step creates your config/ai.php file where you configure your providers and default models. Set your API keys in .env

1OPENAI_API_KEY=sk-...

copied!

The SDK supports OpenAI, Anthropic, Gemini, Cohere, and others for embeddings - configure your preferred provider in config/ai.php.

Step 1: Prepare Your Database.

Add a vector column to the table you want to make searchable. Call Schema::ensureVectorExtensionExists() first - this enables the pgvector extension on PostgreSQL automatically:

Then add AsVector cast to your Eloquent model - the SDK requires this to read and write vector columns correctly:

 1<?php
 2// app/Models/KnowledgeBaseArticle.php
 3 
 4use Illuminate\Database\Eloquent\Casts\AsVector;
 5 
 6class KnowledgeBaseArticle extends Model
 7{
 8    protected function casts(): array
 9    {
10        return [
11            'embedding' => AsVector::class,
12        ];
13    }
14}

copied!

Step 2: Generate and Store Embeddings.

When content is created or updated, generate its embedding and store it. Do this in a queued job -embedding generation makes an API call and you don't want it blocking your request cycle.

 1<?php
 2 
 3// app/Jobs/GenerateArticleEmbeddingJob.php
 4 
 5namespace App\Jobs;
 6 
 7use App\Models\KnowledgeBaseArticle;
 8use Illuminate\Contracts\Queue\ShouldQueue;
 9use Illuminate\Support\Str;
10 
11class GenerateArticleEmbeddingJob implements ShouldQueue
12{
13    use Dispatchable, InteractsWithQueue, Queueable;
14 
15    public function __construct(
16        protected KnowledgeBaseArticle $article
17    ) {}
18 
19    public function handle(): void
20    {
21        // Combine title and content for a richer embedding
22        // The title carries significant semantic weight — don't leave it out
23        $textToEmbed = trim("{$this->article->title}\n\n{$this->article->content}");
24 
25        $this->article->update([
26            'embedding' => Str::of($textToEmbed)->toEmbeddings(),
27        ]);
28    }
29}

copied!

Trigger this from a model observer:

 1<?php
 2 
 3// app/Observers/KnowledgeBaseArticleObserver.php
 4 
 5class KnowledgeBaseArticleObserver
 6{
 7    public function created(KnowledgeBaseArticle $article): void
 8    {
 9        GenerateArticleEmbeddingJob::dispatch($article);
10    }
11 
12    public function updated(KnowledgeBaseArticle $article): void
13    {
14        // Only regenerate if the content actually changed
15        if ($article->isDirty(['title', 'content'])) {
16            GenerateArticleEmbeddingJob::dispatch($article);
17        }
18    }
19}

copied!

1<?php
2 
3KnowledgeBaseArticle::observe(KnowledgeBaseArticleObserver::class);

copied!

Step 3: Semantic Search at Query Time.

This is where it gets elegant. The SDK's whereVectorSimilarTo method accepts either a pre-computed embedding array or a plain string - when you pass a string, it generates the embedding for you automatically:

 1<?php
 2 
 3// app/Services/KnowledgeBaseSearchService.php
 4 
 5namespace App\Services;
 6 
 7use App\Models\KnowledgeBaseArticle;
 8use Illuminate\Database\Eloquent\Collection;
 9 
10class KnowledgeBaseSearchService
11{
12    public function search(string $query, int $limit = 5): Collection
13    {
14        return KnowledgeBaseArticle::query()
15            ->whereNotNull('embedding')
16            ->whereVectorSimilarTo('embedding', $query) // pass the string, SDK handles the rest
17            ->limit($limit)
18            ->get();
19    }
20}

copied!

whereVectorSimilarTo returns results ordered by semantic similarity - most relevant first. No keyword matching, no LIKE queries. Just meaning-based retrieval.

Search for "what happens if my order doesn't arrive?" and it will find content about "delivery failure procedures" even if those exact words appear nowhere in the query.

You can also add a minimum similarity threshold to filter out weak matches:

1<?php
2 
3->whereVectorSimilarTo('embedding', $query, minSimilarity: 0.4)

copied!

Step 4: A Knowledge Base Agent.

The cleanest way to give an agent similarity search capability is the SDK's built-in SimilaritySearch tool. No custom tool class needed for the common case:

 1<?php
 2 
 3// app/Ai/Agents/SupportAgent.php
 4 
 5namespace App\Ai\Agents;
 6 
 7use App\Models\KnowledgeBaseArticle;
 8use Laravel\Ai\Contracts\Agent;
 9use Laravel\Ai\Contracts\HasTools;
10use Laravel\Ai\Promptable;
11use Laravel\Ai\Tools\SimilaritySearch;
12 
13class SupportAgent implements Agent, HasTools
14{
15    use Promptable;
16 
17    public function instructions(): string
18    {
19        return

copied!

Wire it to a controller:

 1<?php
 2 
 3// app/Http/Controllers/SupportController.php
 4 
 5class SupportController extends Controller
 6{
 7    public function ask(AskSupportRequest $request): JsonResponse
 8    {
 9        $agent = new SupportAgent();
10        $response = $agent->prompt($request->validated('question'));
11 
12        return response()->json([
13            'answer' => (string) $response,
14        ]);
15    }
16}

copied!

If you need more control over what gets retrieved - filtering by team, tenant, published status - use the closure form instead:

 1<?php
 2 
 3public function tools(): iterable
 4{
 5    return [
 6        new SimilaritySearch(using: function (string $query) {
 7            return KnowledgeBaseArticle::query()
 8                ->where('published', true)
 9                ->where('team_id', $this->teamId)
10                ->whereVectorSimilarTo('embedding', $query)
11                ->limit(5)
12                ->get();
13        }),
14    ];
15}

copied!

For cases where you need to build a fully custom tool that does its own retrieval and formats results precisely, you implement the Tool interface directly:

 1<?php
 2 
 3// app/Ai/Tools/SearchKnowledgeBaseTool.php
 4 
 5namespace App\Ai\Tools;
 6 
 7use App\Services\KnowledgeBaseSearchService;
 8use Illuminate\Contracts\JsonSchema\JsonSchema;
 9use Laravel\Ai\Contracts\Tool;
10use Laravel\Ai\Tools\Request;
11use Stringable;
12 
13class SearchKnowledgeBaseTool implements Tool
14{
15    public function description(): Stringable|string
16    {
17        return 'Search the knowledge base for articles relevant to a question or topic. '
18            . 'Always use this tool before answering any question — do not answer from general knowledge. '
19            . 'Returns the most relevant articles with their full content.';
20    }
21 
22    public function handle(Request $request): Stringable|string
23    {
24        $articles = (new KnowledgeBaseSearchService())->search($request['query'], limit: 3);
25 
26        if ($articles->isEmpty()) {
27            return 'No relevant articles found in the knowledge base for this query.';
28        }
29 
30        // Format retrieved content clearly for the AI
31        return $articles
32            ->map(fn($article) => "## {$article->title}\n\n{$article->content}")
33            ->join("\n\n---\n\n");
34    }
35 
36    public function schema(JsonSchema $schema): array
37    {
38        return [
39            'query' => $schema->string()
40                ->description('The question or topic to search for')
41                ->required(),
42        ];
43    }
44}

copied!

The Part Most Tutorials Skip: Chunking.

Here's where the difference between RAG that works in a demo and RAG that works reliably in production lives.

Your vector column stores one embedding per database row. If those rows contain entire documents - 5,000-word articles, full PDF exports, lengthy support threads - the embedding becomes an average across all the meaning in that document, and retrieval quality suffers significantly.

Search for "cancellation within 24 hours" and a 10,000-word terms document might match - but you'll retrieve the whole document, and the AI gets buried in irrelevant context trying to find the relevant section.

The solution is chunking: split your content into smaller, overlapping pieces before embedding. Each chunk gets its own embedding and its own row. Retrieval returns the specific chunk that's relevant, not the whole document.

 1<?php
 2 
 3// app/Services/DocumentChunkerService.php
 4 
 5class DocumentChunkerService
 6{
 7    public function chunk(
 8        string $text,
 9        int $chunkSize = 400,    // words per chunk
10        int $overlap = 50        // words overlapping between chunks
11    ): array {
12        $words = preg_split('/\s+/', trim($text));
13        $chunks = [];
14        $i = 0;
15        $totalWords = count($words);
16 
17        while ($i < $totalWords) {
18            $chunk = implode(' ', array_slice($words, $i, $chunkSize));
19 
20            if (!empty(trim($chunk))) {
21                $chunks[] = $chunk;
22            }
23 
24            $i += ($chunkSize - $overlap);
25        }
26 
27        return $chunks;
28    }
29}

copied!

Store chunks in their own table, referenced back to the source document:

 1<?php
 2 
 3// In your migration
 4Schema::ensureVectorExtensionExists();
 5 
 6Schema::create('document_chunks', function (Blueprint $table) {
 7    $table->id();
 8    $table->foreignId('document_id')->constrained()->cascadeOnDelete();
 9    $table->text('content');
10    $table->integer('chunk_index'); // Position in the original document
11    $table->vector('embedding', dimensions: 1536)->nullable()->index();
12    $table->timestamps();
13});

copied!

 1<?php
 2 
 3// app/Jobs/IndexDocumentJob.php
 4 
 5class IndexDocumentJob implements ShouldQueue
 6{
 7    public function __construct(protected Document $document) {}
 8 
 9    public function handle(DocumentChunkerService $chunker): void
10    {
11        // Remove any existing chunks
12        $this->document->chunks()->delete();
13 
14        $chunks = $chunker->chunk($this->document->content);
15 
16        foreach ($chunks as $index => $chunkText) {
17            $chunk = $this->document->chunks()->create([
18                'content'     => $chunkText,
19                'chunk_index' => $index,
20            ]);
21 
22            // Queue each embedding separately to spread API load
23            GenerateChunkEmbeddingJob::dispatch($chunk);
24        }
25    }
26}

copied!

Your SimilaritySearch tool then points at DocumentChunk instead of the parent document, and retrieval returns precisely relevant snippets rather than overwhelming the AI with whole documents.

For most support documentation: 300–500 words per chunk with 50-word overlap works well. For technical documentation with code examples: chunk by section or heading boundary rather than word count - code blocks shouldn't be split mid-example. For structured data (FAQs, product specs): chunk by record - each FAQ item or product attribute set should be its own chunk.

Structured Output.

For features that need to consume the AI's response programmatically rather than just displaying it, implement HasStructuredOutput. The schema is defined using the same fluent JSON schema builder used across the SDK:

 1<?php
 2 
 3// app/Ai/Agents/KnowledgeBaseAnswerAgent.php
 4 
 5use Illuminate\Contracts\JsonSchema\JsonSchema;
 6use Laravel\Ai\Contracts\Agent;
 7use Laravel\Ai\Contracts\HasStructuredOutput;
 8use Laravel\Ai\Contracts\HasTools;
 9use Laravel\Ai\Promptable;
10 
11class KnowledgeBaseAnswerAgent implements Agent, HasTools, HasStructuredOutput
12{
13    use Promptable;
14 
15    public function instructions(): string
16    {
17        return 'Search the knowledge base and answer the question based only on what you find.';
18    }
19 
20    public function tools(): iterable
21    {
22        return [
23            SimilaritySearch::usingModel(KnowledgeBaseArticle::class, 'embedding'),
24        ];
25    }
26 
27    public function schema(JsonSchema $schema): array
28    {
29        return [
30            'answer' => $schema->string()->required(),
31            'confidence' => $schema->string()->enum(['high', 'medium', 'low'])->required(),
32            'sources' => $schema->array()->items($schema->string()),
33        ];
34    }
35}

copied!

Structured responses are accessed via array syntax - not a method call:

1<?php
2 
3$response = $agent->prompt($question);
4 
5$answer     = $response['answer'];     // 'The return window is 30 days.'
6$confidence = $response['confidence']; // 'high'
7$sources    = $response['sources'];    // ['Return Policy', 'Shipping Guide']

copied!

This is particularly useful for displaying source citations alongside answers or routing low-confidence responses to human review automatically.

Our Honest Take.

RAG is the most immediately practical AI concept for most Laravel applications. If a client wants to search their documentation, answer questions about their products, or build an assistant that knows about their business specifically - RAG is the answer, and it's not as complicated as the surrounding jargon suggests.

Where it gets genuinely difficult is retrieval quality. Getting something working is achievable in a couple of days. Getting it to return the right chunks reliably, across a range of real user queries, requires iteration with real data. Chunk size, overlap strategy, how many results to retrieve, whether to rerank by relevance after initial retrieval - these are tuning decisions that need real queries to calibrate against.

The other thing to be clear-eyed about: RAG reduces the surface area for hallucination, it doesn't eliminate it. An AI that receives relevant context is much less likely to make things up, but it can still make reasoning errors. Strong instructions ("base your answer only on the retrieved content; if it's not there, say so") are not optional - they're load-bearing.

Build with RAG, tune with real data, and keep the guardrails in place.

Got questions about RAG implementation specifics - vector store choice, reranking strategies, handling multilingual content? We're happy to go deeper in a follow-up.

Need RAG built into your Laravel application? We've shipped this in production and know where the gotchas are. Get in touch.

RAG in Laravel: Finally, an AI That Actually Knows Your Business

What Problem Are We Actually Solving?

The Four Components You Need.

Setup.

Step 1: Prepare Your Database.

Step 2: Generate and Store Embeddings.

Step 3: Semantic Search at Query Time.

Step 4: A Knowledge Base Agent.

The Part Most Tutorials Skip: Chunking.

Structured Output.

Our Honest Take.

More from the AI series.

Using Claude Code Is Easy. Using It Well Takes More Than That.

RAG, MCP and Agents: The Map Every Developer Needs Before Building AI Features

What Problem Are We Actually Solving?

The Four Components You Need.

Setup.

Step 1: Prepare Your Database.

Step 2: Generate and Store Embeddings.

Step 3: Semantic Search at Query Time.

Step 4: A Knowledge Base Agent.

The Part Most Tutorials Skip: Chunking.

Structured Output.

Our Honest Take.

More from the AI series.

Using Claude Code Is Easy. Using It Well Takes More Than That.

RAG, MCP and Agents: The Map Every Developer Needs Before Building AI Features.

RAG, MCP and Agents: The Map Every Developer Needs Before Building AI Features