Skip to content

Evaluating and Developing Locally

The README gets you a running single-organization demo in one block. This page carries everything after that: choosing a real model boundary, what the lifecycle caches and leaves behind, running the federation lab, the cheapest loop for each kind of change, and how to bootstrap a coding agent.

Lifecycle, caching, and teardown receipts

The final output is the Element URL, @alice:fgentic.localhost, its generated password, and the seeded #lobby:fgentic.localhost room. Every mapped ghost is already a member and has replied to its own seeded transport probe. The command does not mutate the checkout, commit, push, or need a GitHub account; its random credentials live only in the fgentic-demo cluster. Set FGENTIC_DEMO_CACHE_DIR to a persistent directory to reuse BuildKit layers across repeated installs. Remove only that evaluation cluster with mise run demo:down. If teardown is interrupted, rerun the same command: it resumes only the exact identities recorded before deletion, while demo:status reports the pending recovery and demo:up refuses reuse. The content-free receipt lives under XDG_STATE_HOME/fgentic/cluster-teardown/ (or ~/.local/state/fgentic/cluster-teardown/ when XDG state is unset); FGENTIC_DEMO_STATE_DIR overrides the state root. For the persistent full fgentic cluster, mise exec -- k3d cluster stop fgentic/mise exec -- k3d cluster start fgentic preserve its state; mise run cluster:down deletes it.

Choosing the model boundary

Choose the model boundary before using non-demo data:

Choice Sovereignty and cost boundary
demo (evaluation default) Deterministic cluster-only stub; no model credential, prompt egress, or token charge; not a real language model
vertex (local default) Vertex AI google/gemini-2.5-flash; the credential stays at agentgateway, while prompts leave the cluster and the selected GCP project is billed
vllm (GCP default) Real self-hosted model; serving-time prompts stay in-cluster; GCP selects the documented 7B/24 GiB GPU policy profile
mistral EU-hosted API path; prompts leave the cluster and the selected account is billed
anthropic/openai Hyperscaler API path; residency and billing depend on the selected account/profile
azure-openai Azure deployment boundary; region/data-zone selection and billing remain account controls

For example, FGENTIC_LLM_PROVIDER=vllm mise run demo:up selects the real credential-free self-hosted profile. Vertex defaults to google/gemini-2.5-flash and uses ADC locally; Terraform grants the exact GKE agentgateway proxy Workload Identity direct Vertex access without a Google service account or key. Live GKE proof remains spend-gated in #59. Mistral, Anthropic, OpenAI, and Azure OpenAI require the matching key and an explicit FGENTIC_LLM_MODEL. Every paid provider requires FGENTIC_ALLOW_PAID_PROVIDER=yes in the disposable demo lifecycle. The profiles that plainly keep serving-time prompts inside the EU are vllm deployments whose cluster is in the EU, including the tracked GCP default when deployed in its configured EU region; mistral is EU-hosted but remains an external prompt boundary. See the complete residency matrix and provider contract. Do not use evaluation credentials or the deterministic canned-reply stub—it is not a language model—in production.

Running the federation lab

To exercise the federation thesis without an external model or provider account, use the canonical profile:

mise run fed:up

It creates a separate fgentic-fed cluster with participating Synapse homeservers at org-a.fgentic.localhost and org-b.fgentic.localhost, plus org-c.fgentic.localhost as a denied control. The delegation proof verifies the ES256/JCS card at GET https://a2a.org-a.fgentic.localhost/api/a2a/kagent/docs-qa/.well-known/agent-card.json, obtains a short-lived org-B JWT, and calls only the matching POST route. Wrong credentials, audience, client, method, path, or budget fail; one 3,000-token reservation succeeds through docs-qa and the deterministic model, and the second exceeds the 5,000-token azp window. The successful terminal A2A Task metadata carries one seller-signed receipt bound to that canonical request and task identity. Org B verifies the independent JWK, rejects a byte-tampered receipt, and proves denied requests do not append to the content-free single-writer archive. tokensConsumed remains null until per-consumer actuals exist. The separate aggregate model metric must increase, but neither it nor the reservation is presented as per-consumer consumption.

Before a real partner pilot, run the separate mise run fed:split:up drill. It keeps canonical fed:up unchanged and places A/C plus the exported Agent plane in fgentic-fed-a, and B plus its Keycloak issuer in fgentic-fed-b. The drill uses independent cluster APIs, ingress, Docker networks, CA private keys, trust stores, and exact cross-network TLS routes; bidirectional Matrix proves both relay directions, while host-driven probes verify C denial at both ingress planes plus the Signed AgentCard, JWT, quota, and receipt contracts. Canonical fgentic-fed must first be fully removed with fed:down, because it shares A's 127.0.0.2 host binding; the split lifecycle never deletes it implicitly. Use fed:split:status, fed:split:stop, and fed:split:down for the independently owned drill. Both clusters still share one workstation, Docker daemon, kernel, physical network, and clock, so this proves neither WAN nor real-DNS behavior, and it does not prove independent failure domains. See §8.5.3.

On a memory-constrained laptop, explicitly opt into mise run fed:up:constrained. It keeps the same three Synapse servers, shared Postgres, Keycloak, Traefik, Flux, agentgateway, docs-qa/kagent runtime, and acceptance proofs while applying lab-sized workload/controller tuning, a 1 GiB soft target for the disposable k3s server, pausing the optional metrics-server, and serializing the expensive first installs. kubectl top is therefore unavailable while the constrained profile is running. The k3s target is creation-time state, so switch between canonical and constrained capacity with mise run fed:down; same-mode fed:stop/fed:up still reuses the exact cluster and image volume. The constrained wait fails after 20 minutes without a new immutable Flux convergence milestone or after 60 minutes total, whichever happens first. Set FGENTIC_FED_TRACE=yes on either fed:up command to write an allowlisted, content-free resource trace under .agents/tmp/federation-resources/; the exact tuning, measurement method, and laptop budget are defined in §8.5.2.

The Matrix proof requires room-v12 policy, participant-only server ACLs, bidirectional messages between A and B, rejected join plus signed-federation-send attempts from C, and a Synapse callback dropping a disallowed event before it reaches A. mise run fed:policy-reload additionally proves a git policy change takes effect through Flux without restarting either Synapse pod and restores the canonical deny policy. The cluster stays running for inspection: mise run fed:status reports its capacity mode, state, and retained bytes, while mise run fed:stop releases CPU/RAM and preserves the exact owned cluster and image volume for the next same-mode fed:up. mise run fed:down removes all lab-owned containers, network, image volume, and locally built images. An interrupted teardown is retryable through its exact-identity receipt; pending fed:status is inspect-only and fed:up fails closed until fed:down completes. The constrained path creates no additional persistent cache automatically; an explicitly configured FGENTIC_DEMO_CACHE_DIR remains caller-owned. See the federation lab topology and trust boundary; use the separate partner onboarding runbook before enabling a real organization.

The smallest sufficient development loop

The repository owns its development cluster; no global Kubernetes setup, default kubeconfig context, GitHub account, SOPS key, or paid model is required. Use the cheapest proof that reaches the boundary you changed:

  1. Authoring an agent: mise run agent:new <name> scaffolds it, then mise run agent:test <name> runs that one agent's golden tasks against the deterministic zero-spend model — offline, no cluster — the same code path as CI's test:agents-golden. The full scaffold → edit → test → promote → roll-back loop is the agent authoring runbook.
  2. Go behavior: run the focused package tests, then the bridge suite before commit.
mise --cd apps/matrix-a2a-bridge exec -- go test ./internal/a2aclient/ ./internal/bridge/
mise run test:app
  1. Matrix ↔ A2A wire behavior: mise run test:integration creates and removes its own isolated kind fixture; it does not need the platform cluster.
  2. Interactive bridge work: create the lightweight cluster once with mise run dev:up, then use mise run dev:reload after a code change or mise run watch for automatic reloads. Reuse is bridge-only: it does not rebuild the local Git source, reinstall Flux, reconcile the platform, or reseed the room.
  3. Manifest/profile or final end-to-end proof: run mise run demo:up; this intentionally reconciles the current checkout and repeats admission plus seeded Matrix → bridge → agentgateway → kagent acceptance.
  4. Full platform-only features: use mise run cluster:up and the production-shaped bootstrap only for Keycloak SSO, observability/tracing, Trivy, SOPS, or full Flux behavior omitted from the demo.

mise run dev:status reports the lightweight cluster, mise run dev:stop releases its active CPU/RAM while preserving state and images, and mise run dev:down deletes only that owned cluster. All dev:* commands use a temporary kubeconfig and reject a same-named cluster without the demo ownership label. Docker Desktop on macOS and Docker Engine on Linux are both supported through the repo-pinned mise toolchain; ports 80/443 on 127.0.0.1 must be free while the demo is running.

Bootstrapping a coding agent

A fresh clone uses one non-mutating bootstrap:

mise run agent:setup

It installs the pinned contributor and application toolchains, downloads the Go modules, and syncs the locked Python environment. It does not install Git hooks, rewrite dependency manifests, create a cluster, load credentials, or select a paid provider. Run mise trust once on the root and each apps/*/mise.toml, otherwise the aggregate gates abort on an untrusted config.

Run one local agent session against one checkout. Local parallelism across git worktrees or extra clones is not supported: worktrees isolate source and Git state but share the Docker daemon, k3d cluster names, image tags, and host ports, so two sessions running gates at once contend for exactly the resources that matter and can exhaust a constrained host. The .claude/ and .codex/ bootstrap files remain in the repository so a fresh clone or a provider-managed checkout configures itself; they are not an invitation to run several at once.

For Codex Cloud or Claude Code on the web, configure this repository setup script in the provider environment:

set -eu
curl https://mise.run | sh
export PATH="$HOME/.local/bin:$PATH"
printf '\nexport PATH="$HOME/.local/bin:$PATH"\n' >> "$HOME/.bashrc"
mise trust --all --yes
mise run agent:setup

Keep provider credentials, ADC, SOPS keys, kubeconfigs, and local platform overrides out of hosted environments. The repository contains the shared AGENTS.md, CLAUDE.md, project skills, attribution-safe Claude settings, and deterministic checks needed by fresh hosted clones. See Codex Cloud environments and Claude Code on the web.

Cloud agents are where parallelism belongs: run code, documentation, unit-test, and PR work there when it should continue without your laptop and cost it nothing. Keep local agents for the tasks that genuinely need Docker/k3d, ignored local configuration, or machine-specific acceptance — and keep them to a single session, since image builds/imports and the dev:*, demo:*, fed:*, cluster:*, kind, and runtime-test lifecycles all share one Docker daemon, one set of cluster names, one set of image tags, and the same host ports.