Akili
3a68f64eb6
Add a set of guest-facing and operator-facing features that make the deployment-strategy demos (recreate, rolling, canary) more visual: - Emoji reactions (heart/tada/thumbsup/smile): visitors react without an account, a first-party cookie keeps counts honest, the tally streams live to every tab over SSE. Backed by a new reactions table with an idempotent upsert; whitelisted emoji so the endpoint can't be abused. - Light & dark themes: the wall follows the OS preference, visitors can toggle it (persisted locally), and a moderator can force a theme for everyone from the admin console, pushed live over SSE. - Per-replica request counter: a middleware tallies requests answered by each replica; surfaced in /healthz, /api/info and the admin replica table. Under load-balancing the counts diverge, and a rolling update resets them one container at a time — the rollout made visible. - Rolling-update wave in the Chaos lab: bounces replicas one at a time (via the crash endpoint) while polling the replica table, so a training session can watch each replica reset and rejoin — what a rolling update looks like from the inside. - Admin console: reactions tile, per-replica Requests column, reaction tallies in moderation, and a wall-theme setting. - Favicon + /favicon.ico redirect. Tests cover reactions (idempotency, tallies, whitelist) and the theme setting round-trip. Akili-Task: tsk_xgg6sfuzkzymyc6gstr4 Akili-Agent: goldfish
162 lines
8.9 KiB
Markdown
162 lines
8.9 KiB
Markdown
# Guestbook demo runbook
|
||
|
||
A step-by-step script for showing Miabi with the guestbook. Each act builds on
|
||
the previous one and takes 3–8 minutes. Skip acts freely: everything after
|
||
act 2 only assumes the app is deployed.
|
||
|
||
Keep two windows open: the **wall** (`https://<domain>/`) and the **admin
|
||
console** (`https://<domain>/admin`). A third tab on the Miabi app page is
|
||
handy for logs, events and scaling.
|
||
|
||
| Act | Shows | Miabi features |
|
||
|-----|-------|----------------|
|
||
| 1 | Deploy from Git / image | pipeline, build, domain + TLS |
|
||
| 2 | Managed Postgres | databases, env injection |
|
||
| 3 | Secrets + admin console | secret env vars |
|
||
| 4 | Scale out → broken live updates | replicas |
|
||
| 5 | Attach Redis → fixed | managed Redis, env prefix |
|
||
| 6 | Break it on purpose (+ rolling wave) | restarts, health gating, alerts, logs, analytics |
|
||
| 7 | Canary v1 → v2 | canary weights, promote |
|
||
| 8 | Scheduled cleanup | cron jobs |
|
||
|
||
---
|
||
|
||
## 0. Prepare (before the session)
|
||
|
||
```bash
|
||
docker build --build-arg VERSION=1.0.0 -t <registry>/guestbook:1.0.0 .
|
||
docker build --build-arg VERSION=2.0.0 -t <registry>/guestbook:2.0.0 .
|
||
docker push <registry>/guestbook:1.0.0 && docker push <registry>/guestbook:2.0.0
|
||
```
|
||
|
||
- Pick an admin token: `openssl rand -hex 16`.
|
||
- Create an empty Postgres and a Redis in the workspace ahead of time, since provisioning is slower than talking.
|
||
- Open `scripts/hammer.sh` in a terminal (see act 7).
|
||
|
||
## 1. Deploy
|
||
|
||
1. **New application**, from the Git repo (the `.miabi/pipeline.yaml` builds, scans and deploys) or from image `guestbook:1.0.0`.
|
||
2. Port `8080`. **Health check**: HTTP, path `/healthz`. (The default is *none*, and without it act 6 loses its punch.)
|
||
3. Add a domain. TLS is issued automatically.
|
||
4. Open the wall. It runs on **SQLite**, as the first seeded signature says.
|
||
|
||
> Talking point: zero config. With no database attached the app falls back
|
||
> to a local SQLite file. Mount a volume at `/data` if you want it to survive
|
||
> redeploys.
|
||
|
||
## 2. Attach Postgres
|
||
|
||
1. App → **Databases** → attach the Postgres, **no prefix**.
|
||
2. Miabi injects `DATABASE_URL` (as a secret reference) plus `DB_*` and flags the app for redeploy.
|
||
3. Redeploy. The app migrates the schema and seeds the wall on first boot. The welcome signature now says *PostgreSQL*.
|
||
|
||
## 3. Secrets and the admin console
|
||
|
||
1. App → **Environment** → add `ADMIN_TOKEN` = your token, marked **secret**. Add `DEBUG_ENDPOINTS=true` too (used in act 6).
|
||
2. Redeploy, open `/admin` and sign in with the token.
|
||
3. Tour it:
|
||
- **Overview**: signatures per day, total reactions, replicas (with each one's **request counter**), and *this replica*'s runtime (host, DB, uptime, memory).
|
||
- **Moderation**: each signature shows the signer's IP, forwarded by the gateway, plus its reaction tally. Click the IP to filter by it. Pin a signature. It jumps to the top of every open wall, live. Hide one and it disappears.
|
||
- **Settings**: set a banner ("Welcome to the Miabi training 👋"), pick a **wall theme** (force light or dark — every open tab flips instantly), then toggle **Pause signing**. The wall goes read-only instantly in every tab.
|
||
|
||
> Talking point: the token never appears in the app's config in plain text.
|
||
> Rotating it (edit + redeploy) signs every admin out, because sessions are
|
||
> HMAC-signed with it.
|
||
|
||
Ask the audience to open the wall on their phones and sign it. The
|
||
**online** counter climbs live. Have them tap the ❤️ / 🎉 reactions — the
|
||
tallies move on every screen at once, with no refresh.
|
||
|
||
## 4. Scale out, and watch it break
|
||
|
||
1. App → **Replicas** → scale to **3**.
|
||
2. In the admin **Overview**, the *served by* chip changes as you refresh, but the **Replicas** table only ever lists one host. The hint says *in-process broker*.
|
||
3. Open the wall in two browsers (or phones) until their clock cards show **different hosts**. Sign in one: **the other doesn't update.** Each replica only pushes to its own SSE clients. Reactions are the same: a ❤️ on one host never shows on the other.
|
||
4. Also note the `via replica-x` tag on new signatures: each one records which replica accepted it. And `curl -s https://<domain>/healthz` a few times — the `requests` counter climbs on whichever replica answered, so you can watch the load-balancer spread traffic even though live updates stay broken.
|
||
|
||
> Talking point: this is the classic "works on one box" bug. Stateless apps
|
||
> need shared state for fan-out. Scaling is a platform feature; being
|
||
> scalable is an app property.
|
||
|
||
## 5. Attach Redis, and it's fixed
|
||
|
||
1. App → **Databases** → attach the Redis with prefix **`REDIS`**. Miabi injects `REDIS_DATABASE_URL`, which the app reads (so does `REDIS_URL`).
|
||
2. Redeploy. Startup logs show `live updates ready broker=redis`.
|
||
3. Repeat the two-browser test: signatures **and reactions** now appear everywhere. The **Replicas** table lists all 3 hosts with their client counts **and request counters**, and the wall header says *3 replicas*.
|
||
4. **Chaos lab → Traffic probe** → *Run probe*: the bars show requests spread across the three hosts. Compare with the per-replica `requests` counters in the table — they tell the same story server-side.
|
||
|
||
## 6. Break it on purpose
|
||
|
||
All buttons are in the admin **Chaos lab**. Each acts on *the replica that
|
||
receives the request*, and the activity log shows which one.
|
||
|
||
| Button | What happens | Where to look in Miabi |
|
||
|--------|--------------|------------------------|
|
||
| 💥 Crash it | process exits 1; the restart policy restarts it; the page reports when it's back | app **Events**, **Notifications** (crash-loop if repeated) |
|
||
| 🩺 Unhealthy | `/healthz` → 503 on that replica | `app_unhealthy` alert; a deploy made now fails the health gate |
|
||
| 🔥 CPU burn | all cores busy for N s | app metrics |
|
||
| 🧠 Hold memory | allocates N MiB. Set a **Memory limit** of 256 MB first, then hold 400 MiB → OOM kill → restart | `app_oom` alert |
|
||
| 📜 Log burst | N error lines | **Logs** tab (Live) |
|
||
| Probe `/api/debug/slow` | 400 ms latency | **Analytics → Performance** |
|
||
| Probe `/api/debug/error` | 5xx responses | **Analytics → HTTP Traffic** |
|
||
|
||
The same endpoints work from a shell:
|
||
|
||
```bash
|
||
T=<admin-token>; D=https://<domain>
|
||
curl -XPOST -H "Authorization: Bearer $T" $D/api/debug/crash
|
||
curl -XPOST -H "Authorization: Bearer $T" -H 'Content-Type: application/json' \
|
||
-d '{"healthy":false}' $D/api/debug/health
|
||
curl -XPOST -H "Authorization: Bearer $T" "$D/api/debug/memory?mb=400&seconds=30"
|
||
```
|
||
|
||
Remember to click **Healthy** afterwards, or redeploy.
|
||
|
||
### The rolling-update wave
|
||
|
||
Still in the Chaos lab, **Rolling-update wave → Bounce each replica, one at a
|
||
time** restarts the fleet exactly like a rolling update does, but driven by the
|
||
crash endpoint. Keep the **Replicas** table in view: each replica disappears,
|
||
comes back, and its **request counter restarts from ~0** while the others keep
|
||
climbing. The wall never goes down — which is the whole point of rolling
|
||
updates, shown from the inside.
|
||
|
||
> Talking point: recreate would drop every replica at once (all counters would
|
||
> reset together, with a gap of downtime). Rolling keeps capacity by replacing
|
||
> one at a time. Same mechanism, two strategies.
|
||
|
||
## 7. Canary v1 → v2
|
||
|
||
1. In a terminal: `scripts/hammer.sh https://<domain>`. It prints a live count and, on Ctrl-C, a tally of `status version host`.
|
||
2. Set **deploy strategy** to *canary* (initial weight 10%), and deploy image `2.0.0`.
|
||
3. On the wall, reload a few times: ~1 in 10 loads shows the **v2 badge** in a different colour **and the new live server-time card**. In the admin **Traffic probe**, the bars split by version; in the **Replicas** table, the two versions sit side by side with their own request counters.
|
||
4. **Promote now.** The probe goes 100% v2, and `hammer.sh` should show `0 failed`.
|
||
|
||
> The v2 build is the same code with a different `VERSION`. Nothing needs to
|
||
> differ for the demo to work — but the clock card (absent in v1) makes the
|
||
> split visible to the audience without a terminal.
|
||
|
||
## 8. Scheduled cleanup
|
||
|
||
The binary has a `cleanup` subcommand that permanently removes deleted
|
||
signatures and optionally expires old ones:
|
||
|
||
```bash
|
||
guestbook cleanup --purge-after 168h # default: purge deletes older than 7 days
|
||
guestbook cleanup --max-age 720h --dry-run # preview expiring >30-day-old, unpinned entries
|
||
```
|
||
|
||
1. **Jobs** → *New cronjob*, image blank (uses the app's image), command `/app/guestbook cleanup --purge-after 1h`, schedule `*/15 * * * *`.
|
||
2. Delete a signature in **Moderation**, then **Run now**. The job log shows `cleanup finished purged=… remaining=…`.
|
||
|
||
`guestbook migrate` works the same way as a one-off **Job**, for teams that
|
||
want migrations as an explicit step rather than on boot.
|
||
|
||
---
|
||
|
||
## Reset between sessions
|
||
|
||
- Admin → Settings: clear the banner, set the theme back to *System*, un-pause.
|
||
- Chaos lab: **Healthy**.
|
||
- To wipe the wall: detach/recreate the database, or `DELETE FROM entries; DELETE FROM reactions;` and restart (the seed runs when the table is empty).
|