Bart Lind

Stoa — sovereign consultation analysis

A paid co-development engagement with a New Zealand qualitative-research firm that analyses public consultations for New Zealand and Australian government bodies. Stoa turns tens of thousands of public submissions into reviewed, auditable theme analysis.

1. Problem

The firm runs public consultations on behalf of government agencies — the kind of exercise where a single contentious bill can draw in over 300,000 individual submissions. Coding that volume of open-text responses into themes by hand doesn't scale, and it wasn't going to get easier: submission counts were only trending upward. At the same time, the agencies it serves operate under data-sovereignty rules that rule out sending citizen submissions to commodity cloud SaaS products. The firm needed an AI-assisted analysis pipeline it could run on infrastructure it controlled.

2. Engagement

I'm co-developing Stoa with the client as a paid engagement through Fridai, my consultancy — an ongoing relationship with an agreed roadmap rather than a single fixed-scope delivery. The system is currently in pilot: it's being tested against real consultation data as the analysis pipeline and review workflow are hardened for production use.

3. Architecture

Stoa runs a 9-phase configurable analysis pipeline that turns raw submissions into a hierarchical theme tree — broad categories, down through groupings, down to leaf-level themes — so reviewers can navigate from a high-level summary to the specific submissions behind any given theme. The application is a Django backend with an Astro/Svelte frontend, packaged and deployed with Docker.

Heritage, stated plainly: Stoa began as a fork of UK i.AI's open-source Consult project. The upstream Django core, Terraform, and CI are i.AI's — I forked and extended them. The analysis pipeline extensions, the Bedrock/RAG/GPU work, the review frontend, and the AWS migration are mine.

4. Sovereign port

The data-sovereignty requirement drove a self-hosted architecture: no dependency on a commodity cloud AI API for processing citizen submissions. That meant evaluating open-weight, Llama-class models as a substitute for proprietary hosted LLMs, including self-hosting Llama 3 70B on GPU — first via Ollama, then load-tested on RunPod rigs — to establish whether an open-weight model could hold up against the proprietary baseline on this task. In parallel, I assessed on-shore New Zealand compute options, including AWS Bedrock, as a sovereignty-compliant middle ground between fully self-hosted inference and a general-purpose cloud API. Retrieval for legislative context uses a targeted, hierarchical RAG approach built for the way legislation is structured, backed by server-side hybrid search — Postgres full-text search combined with vector embeddings — rather than a naive single-index vector store.

5. Human review

No theme mapping ships without a human able to see it. The review frontend I built presents AI-proposed theme mappings side by side with the source submission, lets a reviewer filter and correct in bulk, and keeps a timestamped audit trail of every change. Each mapping's provenance — whether it was AI-proposed or human-corrected — stays attached to the record, so the audit trail reflects who actually decided what.

6. Procurement

Alongside the platform build, I co-authored a two-envelope RFP response for a New Zealand government agency's procurement — technical and commercial envelopes submitted separately, against pre-conditions drawn from NZISM and Protective Security requirements, with data-residency guarantees written into the response. The bid wasn't selected, but it left me with first-hand experience of how government AI procurement is actually structured and evaluated — knowledge that doesn't come from reading the tender documents alone.

7. Evidence