03 / Delivery system

Website-to-demo delivery pipeline

Role
Sole engineer and technical lead. Self-directed build through Fridai, for my own outreach.
Scope
Seven staged skills: intake, site inventory, design directions, build, grounded chatbot, deploy, priced proposal.
Stack
Astro · Cloudflare Pages and Functions · MiniSearch + Gemini embeddings, RRF-fused · Firecrawl · Playwright
Status
Two live prospect demonstrations deployed from one template repository and seven agent skills.
6 days → 1 day
first hand-built run, then the reusable template
29 + 33 cases
independent golden-QA suites, one per demo
7 human gates
stage approvals before the pipeline advances
2 prospect demos
live URLs, not paying-client deployments

An agency selling an AI-assisted website rebuild is selling something the prospect cannot picture. The pitch that closes is the prospect's own site, already rebuilt, with a chatbot answering real questions from their real content. Hand-building one of those costs days per prospect, most of whom never reply, so what limits how many businesses an agency can pitch is the cost of the demo rather than the difficulty of the work.

1. The problem

This is the pipeline I built to remove that cap. A prospect's URL goes in; a rebuilt site of around ten pages, a chatbot grounded only in that business's own content, and a priced proposal come out, with every stage stopping at a human gate before the next one starts. It runs against education-sector businesses because that is where my outreach points. The two instances below are prospect demos, not paying-client deployments.

2. Two runs, one reusable core

Productising the pipeline took two real builds and one extraction step between them:

First build
Hand-built: six days took one education business's live site to a deployed replacement with a working grounded chatbot and its own 29-case golden-QA suite. The artefact sold well. The delivery model did not: a week of work per prospect, spent before anyone has replied, is not a sales motion an agency can run at volume.
Reusable core
So I extracted the machinery from it. On the day that build closed, the repeatable parts came out into a template repo and seven skills, one per stage, with the prospect-specific material isolated behind a config file, a facts file and a token sheet. The demo stayed a live pitch asset; everything reusable moved out from under it.
Second build
The second instance is the proof. Two days later, a different education business went from intake to live deployment in one day, with its own 33-case suite and the priced proposal out the following morning. Same architecture, entirely separate content, brand and chatbot corpus. The six-days-to-one-day gap is the whole return on productising it.

3. Seven gates

Each stage ends at a human gate. Nothing advances until I sign off on the previous stage's output. The gates sit where an autonomous run would otherwise commit to something expensive to undo: scraping a live site, picking a visual direction, shipping a chatbot, sending an offer. On the second build the inventory diff kept 158 of 418 discovered URLs, and the build shipped thirteen pages. At the build gate, generic templates carry the routine pages while the one or two pages that do the selling are built bespoke, against screenshots of the real site rather than from memory of it.

At the design gate the brand signals become a small set of concrete directions, each rendered as a real homepage using the business's actual copy and imagery. Choosing from rendered pages rather than descriptions is what keeps the direction argument to one round.

Intake

Input
Prospect URL.
Human approval
Fit and stack approval.
Artifact
Project scaffold.

Inventory

Input
Three-way URL discovery.
Human approval
Coverage-diff approval.
Artifact
Scoped map.

Design

Input
Brand signals and real content.
Human approval
Rendered direction approval.
Artifact
Selected design.

Build

Input
Selected direction and scoped pages.
Human approval
Key-page review.
Artifact
Static site.

Grounded chatbot

Input
Curated sourced corpus.
Human approval
Complete golden-suite approval.
Artifact
Chatbot index.

Deploy

Input
Green build.
Human approval
Preview and live-URL QA.
Artifact
Cloudflare Pages deployment.

Proposal

Input
Scoped demonstration.
Human approval
Offer/terms approval.
Artifact
Branded proposal and outreach kit.

4. Evaluation as the ship gate

The suite passes in full or nothing ships. Each instance has its own golden question-and-answer set: 29 cases on the first, 33 on the second, asserted with must and must-not patterns against the reassembled streamed answer. A shared guard appended to every case catches leaked chain-of-thought, model or provider names, and "as an AI". Both suites reserve a block for adversarial input: prompt injection, identity probes, invented offerings, price and off-topic bait, and citing a page that was deliberately cut from the build.

Then the same suite runs against production. A chatbot that passed against a local build is a different system once it is serving from another origin with separately configured secrets, so the pre-deploy green is treated as necessary and not sufficient. Both instances passed their full suite against the deployed worker, not just in development.

The evals earned that status by catching a real bug. An adversarial question about the business's location came back with an address the rendered site no longer showed. The cause was structural, not a prompt problem: the index builder chunked the raw scrape directly and bypassed the editorial layer that had already corrected roughly 89 FAQ answers, so the chatbot could resurface exactly the content the site had been fixed to stop showing. Correcting it at the source cut the index from 1,315 chunks to 753, most of the loss being navigation and footer noise.

One failure was the eval's own: a price-bait case flagged an honest clarifying question as wrong, so I relaxed the assertion rather than steer the model into satisfying it, because a suite tuned to keep itself green stops measuring anything.

5. Retrieval that degrades instead of failing

Keyword and vector search, fused. MiniSearch over the chunked corpus and 768-dimension embeddings are retrieved separately and combined by reciprocal rank fusion, which keeps the answers that only exact wording finds alongside the ones that only semantics finds.

The vector store self-guards. Each build stamps the corpus with a hash; if the vector file is missing or its hash no longer matches the index, retrieval falls back to keyword-only instead of answering from a stale embedding or failing outright. A keyword-only answer is worse than a fused one and much better than a confident answer built on an index that no longer matches the site.

6. Evidence