how this works

bringing the sumerian hosts SDK back from the dead

aws-samples/amazon-sumerian-hosts was last updated around 2023. the upstream is abandoned — no maintainer, no releases, no response to issues. the SDK shipped animation state machines, viseme curve logic, and character assets, but the runtime was tied to a dead hosting platform and a bundling strategy that aged out of compatibility with modern engines

our fork at chasko-labs/sumerian-hosts is a clean-room reimplementation, not a patch. we extracted values — curve coefficients, viseme mappings, animation state machine logic — from the MIT-0 reference material, then rebuilt the runtime from scratch

the stack:

• pure ESM modules, no CommonJS shims
• bun as the development runtime
• babylon.js 9.3.4 as the first (and currently only) engine adapter
• MIT-0 reference material provided the mathematical foundations: animation state transition logic, blend curve coefficients, phoneme-to-viseme weight mappings. extracted as numerical values, not code copy
• all 9 characters vendored under CC-BY-SA-4.0: cristine, fiona, grace, maya, alien, luke, jay, preston, wes

the characters are GLB meshes with full skeletal rigs, bone-driven animation groups for facial animation, and pre-baked idle clips. they were the best part of the original SDK — high-quality humanoid assets with expressive face rigs — and they deserved a runtime that could actually ship them to a browser in 2026

implementation status

honest accounting of what works as of 2026-08-19. waves 1-3 shipped the full 6-layer animation stack:

the core game is playable with full 3D character animation. hosts speak with bone-driven lipsync, gesture during speech with keyword-mapped animations, emote on quip completion, blink on a random schedule, and exhibit subtle face micro-expressions. the camera zooms to face close-up during speech and pulls out during gestures. the only unimplemented layers are POI gaze tracking and inter-host ambient reactions — documented below for future work

recent additions (v0.020-v0.038)

new in v0.038 — BG preload, persistent hosts, eye gaze, fanfare, rizz

background preload (lazy, non-blocking)

after first paint, host-pool.scheduleBackgroundPreload() runs via requestIdleCallback(timeout 3000) or setTimeout 800 fallback. it warms the HTTP cache for Enlil audio (/audio/enlil/q*.mp3 + json, 10 next via <link rel="prefetch"> + fetch force-cache) and neighbor host GLBs (fixtures/hosts/<id>/<id>.glb + stand_idle.glb for next 3 board-adjacent hosts). subsequent challenger cam opens <400ms from cache vs 5s cold on 5Mbps Pixel. flagged by __ssBgPreload (false disables).

persistent host pool (cap + LRU)

host-pool.ts keeps up to N live Engine/Scene/Canvas hosts in the grid. getHostPoolCap() derives N from navigator.deviceMemory and hardwareConcurrency: 9 on desktop >=8GB/8 cores, 6 on phone 6GB, 2 on 2GB, 4 default. URL ?cap=N overrides. when pool exceeds cap, LRU entry is disposed (stopRenderLoop, scene/engine dispose, canvas removed, poster fallback). target <300MB at 9 alive. flagged by __ssPersistHosts.

lively idle

stand_idle + face_micro loop + generic_a/b every 6-10s (deciding-alive) + hemi breathe ±0.06 jitter at 220ms + POI look_center half-speed. pool hosts stay in deciding-alive even when not challenger, so grid watches alive.

welcome fanfare

on first load, Enlil (Polly Brian welcome.mp3 + welcome.json, fallback Web Speech Google UK English Male → Daniel) + orb sh-orb--speaking says “Welcome to Sumerian Squares, challenger awaits” before prompt. writes sh-arena__fanfare text synchronously. stored ss-welcome-played prevents replay unless ?fanfare.

eye gaze follow

poi-tracker.ts now drives live hosts’ POI groups from mouse. mousemove → yaw ±25° / pitch ±12° clamped, lerped 0.12 at ~30Hz rAF, weights mapped to look_left/right/up/down/center via lerp 0.18. suspended during playGesture/playEmote (additive isolation). flagged by __ssMouseGaze or ?nogaze=1.

tap reactions

pointerdown on Enlil orb or live canvas.sh-cell__canvas--visible triggers non-disruptive reaction: orb scale 1.06 180ms + host generic_a or head nod via playEmote/playGesture, debounced 300ms, preventDefault only on orb side effect, grid game tap still advances if applicable.

namecard unify

static grid and challenger now share .sh-nameplate plate: badge (shape per host, spans 3 rows) + name + archetype + city (hometown from HOST_SCORECARD) in 3-row grid, same accent tint, same radius/shadow, same border. challenger larger, static smaller, both use sh-host-accent tint.

rizz pass

anisotropic 4 on textures after loadHost, lighting hemi 0.85 + rim 0.5 jazz per archetype (existing, reapplied on theme flip), gradients header sweep + stage wash, spacing arena gap 16px, shadows soft shadowmap 1024 + PCF via scene.shadowsEnabled, without regressing FPS (<16ms frame). stage glow sh-stage-glow 12s retained.

recent additions (v0.020-v0.031) — legacy

the horse-face bug — root cause and resolution

the “horse-face” bug manifested as jaw/teeth displacement on the first host loaded in any session. teeth protruded forward of the face during idle, disappearing when lipsync started. the second host loaded in the same session was always clean.

root cause: poi-tracker.ts had a broken targetConverter in its mergeAnimationsTo call. it returned the SOURCE node from the poi.glb container (which was then disposed via container.dispose()), instead of looking up the HOST’s node by name. additionally, only 3 of 27 merged POI animation groups were made additive — the remaining 24 sat on the scene as non-additive groups targeting the jaw bone. BabylonJS’s scene animation evaluator sampled these non-additive groups at frame 0 (which contains the absolute bind-pose position of (0, 0, 0)), pulling the jaw to origin.

why second host was clean: deactivateHost() disposes the entire scene. the second host gets a fresh scene with no stale POI groups.

fix (v0.037): proper name-based targetConverter using the host’s nodeByName map, all 27 POI groups made additive via MakeAnimationAdditive(group, 0), non-active groups explicitly stopped.

diagnostic that found it: a console-injectable script reading jawNode.position revealed (0, 0, 0) instead of the expected rest position (0, 0.8337, 1.4404). the 27 non-additive POI groups with isAdditive=false were the smoking gun — they were evaluating the jaw at bind pose.

loading sequence — what downloads when

phase 1: page load (instant, no 3D)

phase 2: first host click (BabylonJS + GLB assets)

total first-click download: ~18-20MB (sequential GLB loads). subsequent clicks reuse cached BabylonJS + same-type GLBs.

phase 3: host reveal (300ms after render loop starts)

canvas appears with host in settled idle pose. face_micro, blink scheduler, and POI gaze tracking all active. no lipsync groups on scene (deferred to speech start).

phase 4: speech start (after Enlil reads question)

lipsync.glb merges onto scene (~50ms). MakeAnimationAdditive bakes 18 groups. 7,236 animatables created via beginDirectAnimation at peak frame, weight=0. RAF loop drives per-frame weight assignment. entrance gesture + keyword gesture fire concurrently.

why the “black screen” pause exists

the sequential GLB load chain (steps 3-11) totals ~18MB. on typical broadband (10Mbps), this takes 5-10 seconds. the host mesh renders quickly (step 3), but gesture.glb (7MB) dominates the wait. the canvas is hidden until all layers load + 300ms settle. this is the “black screen” period.

future optimization: parallelize GLB downloads (steps 4-11 are independent after step 3). use Promise.all on the fetch layer while maintaining sequential merge order. this would reduce the black screen to the time of the single largest file rather than the sum.

the 7-layer animation stack

each host runs 7 concurrent animation layers. only layer 1 is currently implemented. the correct implementation per the proven fiona-embed.ts (82KB, deployed at clouddelnorte.org):

Layer 1: stand_idle.glb -> mergeAnimationsTo -> start(true, 1.0, from, to)
  NON-additive. Base body pose. Weight reduced to 0.35 during gestures.
  STATUS: WORKING (405 targets, 480 frames, looped)

Layer 2: face_micro.glb -> mergeAnimationsTo -> MakeAnimationAdditive(group, 0) -> start(true)
  Additive. Subtle face motion. Continuous loop.
  STATUS: WORKING (correct additive pattern applied)

Layer 3: blink.glb -> mergeAnimationsTo -> MakeAnimationAdditive each variant -> scheduler
  Additive. 3 variants. Random 3-6s interval. One-shot per blink.
  STATUS: WORKING (scheduler fires 3-6s random interval)

Layer 4: poi.glb -> startPoiTracker(scene, hostHandle, poiUrl, poiConfigUrl)
  Additive. 27 clips. Gaze follows virtual TransformNode target.
  3 sub-layers: LookHead, LookEyes, LookBrows.
  STATUS: NOT IMPLEMENTED (assets on CDN, tracker pattern proven)

Layer 5: gesture.glb + gesture.json -> startGestureQueue
  Additive. In/hold/out phases. Queueable. Promise-based.
  During gesture: standIdleGroup.weight = 0.35
  After: standIdleGroup.weight = 1.0
  14 named gestures: aggressive, big, defense, generic_a/b/c, heart, in, many, movement, one, self, wave, you
  STATUS: WORKING (keyword mapping from quip text drives gesture selection)

Layer 6: emote.glb -> startEmoteQueue
  Additive. Single-clip face expressions. One-shot, queueable.
  STATUS: WORKING (3 emotes: applause, bored, cheer — fired on quip end)

Layer 7: lipsync.glb -> 17 AnimationGroups -> visemeApplicator
  Additive. Frozen at peak frame via beginDirectAnimation(speedRatio=0, isAdditive=true).
  Per-frame animatable.weight drives blend magnitude.
  NOT morph targets. Bone-driven via AnimationGroups.
  STATUS: WORKING (18 groups including container, per-frame weight from Polly viseme marks)

the frozen-at-peak-frame pattern is the key technique: scene.beginDirectAnimation starts a clip but freezes at the peak expression frame via speedRatio=0. per-frame animatable.weight assignment then scales the additive blend from 0 to 1 and back, driving intensity without looping the clip. this gives smooth ramp-in/ramp-out on any additive layer without needing separate fade clips

the viseme pipeline — how lipsync works

the correct approach is implemented. hosts use bone-driven AnimationGroups, not mesh-level morph targets. lipsync.glb contains 17 AnimationGroups, one per phoneme:

sil, p, t, S, T, f, k, i, r, s, u, @, a, e, E, o, O

the correct pipeline (from production fiona-embed.ts):

1. load lipsync.glb via SceneLoader.ImportAnimations
2. bake each group additive: AnimationGroup.MakeAnimationAdditive(group, 0)
3. for each TargetedAnimation in each group, create a frozen Animatable: scene.beginDirectAnimation(ta.target, [ta.animation], group.to, group.to+0.001, true, 0, undefined, undefined, true)
4. speedRatio=0 freezes the timeline at peak expression frame
5. per render frame: set animatable.weight from the viseme weight value (0.0 to 1.0)
6. buildVisemeGroupMap() matches AnimationGroup names to VISEME_ORDER codes

phoneme codes are case-sensitive. the distinction matters for accurate mouth shape:

• S (post-alveolar, “sh”) vs s (alveolar, “see”)
• T (th as in “think”) vs t (top)
• E (bear, open mid-front) vs e (bet, close mid-front)
• O (caught, open-mid back) vs o (code, close-mid back)

Polly viseme marks (what we already have):

• 139 JSON files at /sumerian-squares/audio/<host>/qq<id>.json
• format: [{time: ms, value: phonemeCode}, ...]
• these map 1:1 to the 17 AnimationGroup names via VISEME_MAP
• per render frame: find the current active viseme from elapsed audio time, set that group’s animatable.weight to 1.0, decay previous viseme toward 0

the wasm viseme crate handles the interpolation math — cubic hermite evaluation of weight curves at 60hz. but even without wasm, a simplified lookup against the Polly mark timestamps would produce acceptable results: snap to the nearest mark, set weight=1 on the active phoneme, weight=0 on all others

camera system — measured-geometry framing (reproducible)

the camera framing is self-calibrating. it does NOT use per-host or per-type magic numbers for target height and distance — those failed live QA because two hosts of the same type (cristine and maya are both adult_female) framed completely differently, since the loaded glTF models carry per-model variance in rig height, mesh origin, and head world-Y. one fixed target.y landed on cristine’s eyes but maya’s chin.

the fix, and the approach to reproduce: after a host’s glTF loads, MEASURE the model and derive the framing from the measurement.

how to reproduce the framing

1. after loadHost resolves, measure the world-space bounding box of the whole mesh hierarchy: rootMesh.getHierarchyBoundingVectors(true) → { min, max }
2. take crown = max.y (top of head / mask) and feet = min.y; height = crown - feet
3. derive the humanoid face target from fractions of measured height, applied DOWN from the crown:

4. the face beat centers between the eye and mouth: faceTargetY = (eyeY + mouthY) / 2. the gesture beat targets the collar so head-and-shoulders + hands read
5. radius scales with the MEASURED head span (crown-to-mouth), not an absolute distance, so the head fills the same proportion of frame on every model regardless of its scale:

6. a per-beat camera beta (the ArcRotateCamera polar angle) controls the vertical tilt. beta = PI/2 is dead level with the target; beta < PI/2 shoots from slightly ABOVE looking gently DOWN — the flattering portrait angle. face beat sits a touch below level (LEVEL - 0.08 female, LEVEL - 0.06 male); gesture beat holds a whisper above level. the vertical FOV is BabylonJS’s default 0.8 rad, and the visible world half-height at the target plane is radius * tan(FOV/2) — every framing derivation uses this so the geometry stays honest

the alien (non-humanoid)

the alien rig has no legs (a floating mask/bust ~1.2 units tall), so no humanoid eye/mouth fraction applies. instead of guessing where its “face” sits, frame the WHOLE measured bbox with margin: target the bbox vertical center, and solve the radius so the full measured height fills a fixed fraction of the frame (ALIEN_FACE_FILL = 0.55, ALIEN_GESTURE_FILL = 0.45). the crown never clips because the margin is the whole point. radius solves from 2 * visibleHalfSpan(r) = height / FILL

mobile portrait bias

on phones the question card is a bottom-sheet covering the bottom ~42% of the viewport (PORTRAIT_OCCLUDED_FRACTION = 0.42). the camera biases target.y DOWN by visibleHalfSpan(radius) * 0.42 so the face lands in the center of the visible band above the card. deriving from visibleHalfSpan ties the bias to the real world-per-screen scale at the current radius, so it self-corrects at both the tight face radius and the wide gesture radius (an earlier radius * fraction * gain heuristic doubled the bias when the measured radius widened and shoved the crown off-frame)

fallback

per-host-TYPE constants (CAMERA_FRAMING in camera.ts) remain ONLY as a fallback for when no measurement is available (measurement failed, or a beat fires before load completes). the measured path is primary and drives the normal case; the fallback reuses the per-type beta signatures so the tilt is consistent either way

debugging the framing

set window.__ssDebugFraming = true in the browser console (or flip DEBUG_FRAMING in camera.ts). every host then logs its measured min.y / max.y / height, the derived crown / eyeline / mouth / target, and the FINAL applied beat (post portrait-bias) with radius, targetY, beta, and the resulting visible world span (top and bottom of frame). read those numbers instead of guessing rig heights — that is the line that ended the Aug-30 guessing loop

the exact fraction and radius constants live in assets/sumerian-squares/src/camera.ts (owned by the engine track). the values above are current as of this writing; if they drift, camera.ts is the source of truth

gesture and emote assignment per saying

from Jake Smeester’s Twitch demos and the gesture.json config shipped with each host type:

keyword-to-gesture mapping:

each gesture has three phases defined in gesture.json: in (attack), hold (sustain), out (release). the gestureQueue pattern from fiona-embed manages these phases as promises. during the “in” and “hold” phases, standIdleGroup.weight drops to 0.35 so the gesture dominates. on “out” completion, standIdleGroup.weight returns to 1.0

emote assignment: each quip in the question bank should declare an emote to fire AFTER the speech completes:

• funny quip — emote: “cheer”
• dramatic reveal — emote: “surprise”
• wrong answer caught — emote: “concern”

ambient reactions on non-speaking hosts (from game design spec):

these ambient reactions use the same gesture and poi systems — they just fire on non-active hosts in response to game events rather than speech content

asset URLs on CloudFront

all assets served from d161lxp8cb37vp.cloudfront.net (S3: sumerian-hosts-site, account 946179428633)

host type mapping:

• cristine, fiona, grace, maya = adult_female
• jay, luke, preston, wes = adult_male
• alien = alien

bone naming per type:

• adult*female and adult_male: char:def*{l|r|c}\_{boneName}(e.g.char:def_l_browA)
• alien: char:{l|r|}_{boneName} (e.g. char:l_brow_01, char:jaw)

cross-type animation compatibility:

• cross-humanoid animation reuse works: adult_female anims loaded onto adult_male mesh binds 7236 targets successfully
• cross-rig reuse FAILS silently: humanoid anims loaded onto alien mesh binds 0 targets — the bone names do not match. alien must use its own animation set exclusively

the emit-message pattern for host interaction

from Jake Smeester’s Conversing Hosts Twitch broadcast. this is the pattern for multi-host coordination — NOT IMPLEMENTED yet but the architecture is proven

hosts communicate via named message channels:

1. Host A finishes speech — emits message “start B”
2. Host B listens for “start B” — enters wait state (700ms for natural pause) — B starts speech
3. B finishes — emits “start A”
4. loop continues

messages are case-sensitive. the wait state between listen and speech-start is what makes conversations feel natural. without it, responses are unnervingly instant — the 700ms pause simulates the cognitive gap between hearing and responding

for sumerian-squares: this pattern drives ambient reactions. when the active host finishes their quip, they emit a message that triggers head-nod/look reactions on 2-3 other hosts. the receiving hosts use their poi layer (look toward the speaking host or the claimed square) and a brief gesture layer activation (head nod via a short generic_a clip)

rust and webassembly

numerically intensive paths moved to Rust, compiled to WebAssembly via wasm-bindgen. the wasm binary is approximately 45KB and runs the blend-shape math that would otherwise be expensive JavaScript per-frame work

three crates handle the heavy lifting:

crates/viseme/ — 17-channel viseme weight calculation at 60hz. takes Polly speech mark timing data and evaluates phoneme-to-weight curves, outputting a 17-float weight array every frame. the curve evaluation uses cubic hermite interpolation with coefficients extracted from the original SDK’s animation data

crates/asset-tools/ — GLB animation channel stripping. generates face_micro.glb from face_idle.glb by filtering animation channels against bone-name regex patterns. adult hosts keep 195 of 399 channels (facial bones only). the alien host keeps 123 of 279 (different skeletal topology). this runs at build time, not runtime — the stripped GLBs ship as static assets

crates/tts-handler/ — Polly speech mark parsing and timing signal extraction. converts Polly’s JSON speech mark output into the frame-aligned timing structures the viseme crate consumes

the wasm module initializes once on first host activation and shares its memory across all 9 hosts. no per-host wasm instantiation overhead

pre-recorded audio — zero runtime TTS

all quips are synthesized at build time via AWS Polly. the game ships static mp3 files paired with viseme JSON — zero network calls during gameplay. no Polly API keys in the client, no latency spikes from synthesis requests, no cost per play session

the AudioManager lazy-loads audio per host on first selection. a single AudioContext with StereoPannerNode handles spatial placement:

• left column hosts: pan -0.7
• center column: pan 0.0
• right column: pan +0.7

Polly neural voices assigned per host:

voice selection matches character archetype. lower-register voices for authority figures, higher for the alien’s otherworldly presence. neural voices produce more natural speech marks than standard voices, which translates directly to more accurate viseme timing

browser quirks addressed:

• safari re-lock mitigation: AudioContext.resume() called before each playback. safari suspends the context aggressively after periods of silence
• firefox: AudioContext only unlocks on button click events, not arrow key events. keyboard navigation does not create or resume the AudioContext — only explicit click interactions do. this prevents firefox from blocking audio entirely when the user navigates by keyboard first

babylon.js integration

the rendering architecture uses a single Engine instance with one canvas per active host. single-canvas, not multi-view — we abandoned the engine.registerView() approach

CDN-loaded UMD builds: babylon.js loads from cdn.babylonjs.com as UMD scripts, not bundled ESM imports. this is deliberate. both ESBuild (Hugo’s js.Build) and Bun tree-shake babylon.js internal constructors during bundling, causing X is not a constructor crashes during glTF parsing. the CDN UMD builds self-register all extensions globally — loaders, serializers, materials — so nothing gets tree-shaken away. app code is bundled by Hugo js.Build with externals declared for @babylonjs/core and @babylonjs/loaders

consent-gated loading: zero babylon.js bytes download before the user clicks “activate 3D”. the initial page renders with static poster images. babylon.js loads only on explicit opt-in, respecting bandwidth and user intent

lipsync weight trick: AnimationGroup weight set to -1 for non-speaking hosts. weight=0 still evaluates the blending code path (multiplying everything by zero). weight=-1 tells babylon.js to skip the blend computation entirely — a meaningful difference when 8 of 9 hosts are silent at any given moment

geometry sharing: AssetContainer.instantiateModelsToScene shares geometry buffers and material instances across all 9 hosts. only skeletons and animation state get cloned per instance. this keeps GPU memory roughly constant whether 1 or 9 hosts are active

degradation tiers handle device capability:

• full: all 9 hosts render in their viewports
• viewport: 5 visible hosts render, 4 off-screen hosts suspended
• hero: only the selected host renders, others show poster frames
• poster: static images only, no WebGL context created

build and CI

hugo extended 0.152.2 handles the site build. js.Build compiles TypeScript directly — no webpack, no vite, no separate bundler step. hugo’s asset pipeline handles tree-shaking and minification

linting gates in CodeBuild:

• biome enforces tabs, double quotes, and CSS lint rules
• markdownlint validates all content files

the deploy pipeline: push to main triggers CodePipeline, which runs lint, then hugo build, then a two-pass S3 sync. assets get 1-day cache headers. HTML gets no-cache headers. CloudFront invalidation fires on the changed paths

feature branches do not auto-deploy. the path is: PR review, merge to main, pipeline fires. no preview deploys, no branch-specific URLs

s3-prefix-registry.json governs shared-bucket collision avoidance. bryanchasko.com hosts multiple apps in the same S3 bucket — listen, mom, bench, design, chat, news, and now sumerian-squares. the registry prevents path collisions between them

design evaluation loop

every phase goes through a visual quality gate before deploy:

1. parallel authoring wave — engine code, CSS, and content authored concurrently
2. ghost-scribe-style-enforcer validates text against banned words and tone requirements
3. Playwright captures screenshots at 5 viewport widths
4. Amazon Nova Lite VLM scores the screenshots on hierarchy, typography, spacing, accessibility, and interaction states
5. score below target loops back to authors with specific findings (maximum 3 iterations before escalation)
6. score meets target: PR, CI, merge, deploy
7. re-score the deployed URL to confirm production matches staging

the VLM scoring is not decorative. it catches regressions that unit tests miss — text over low-contrast backgrounds, touch targets that shrunk below 44px, focus rings that disappeared after a refactor. the loop runs automatically as part of the authoring workflow

MCP infrastructure

the haunting agent ecosystem uses MCP (Model Context Protocol) servers to connect AI agents to external services. sumerian-squares development uses several of these during authoring and testing

dockerized containers on rocm-aibox (192.168.4.53)

persistent MCP servers run as docker containers managed by docker-compose at ~/code/heraldstack/heraldstack-infra/docker-compose.yml:

containers bind to 0.0.0.0 so they are network-accessible from the mac mini development machine. each has a bridge launcher at ~/code/heraldstack/heraldstack-mcp/launchers/bridges/<name>-bridge.sh that wraps supergateway to translate streamable HTTP to stdio for kiro-cli consumption

cloud-hosted MCP (AWS account 946179428633)

these services run in AWS, accessed via Bedrock credentials or IAM role assumption:

nova act connects to Bedrock AgentCore which provisions a managed Chromium browser in us-east-1. the agent drives the browser via CDP with natural language instructions. authentication uses the bryanchasko-kiro AWS profile (account 946179428633) which has the DenyThirdPartyBedrockInvoke guard allowing amazon.* models

local MCP (no container, no network)

some servers run as direct processes without docker:

how agents reach MCP servers

the ~/mcp-launchers/run.sh dispatcher auto-detects the host:

• on rocm-aibox: launches the server locally (exec the launcher script)
• on mac mini: SSH tunnels to rocm-aibox for docker-hosted servers, uses local supergateway bridges for HTTP endpoints

agent JSON configs declare MCP servers via the stdio binding pattern:

{
	"command": "bash",
	"args": ["-c", "$HOME/mcp-launchers/run.sh s3vectors-bridge.sh"]
}

deploy.sh translates relative ./mcp-launchers/ paths in the repo to absolute $HOME/mcp-launchers/ at deploy time, so the same agent JSON works across both Linux and macOS

MCP servers used during sumerian-squares development

nova act test infrastructure

test files live at tests/nova-act/:

• sumerian-squares-full-interaction.py — 16-step gameplay validation (grid load, host click, question, buttons, marks, AI, game completion)
• sumerian-squares-3d-check.py — 6-layer animation validation per host (camera zoom, lipsync, gesture, emote, face_micro, blink)

authentication requires NOVA_ACT_API_KEY env var (free tier) or AWS_PROFILE=bryanchasko-kiro with @workflow decorator (AWS service tier). the tests validate what headless Playwright structurally cannot: whether mouth movements are visually synchronized to speech, whether gestures look natural, whether the camera framing is intimate enough

user interactions

the hover-to-look behavior uses the poi animation layer. when the cursor enters a grid cell, that host’s point-of-interest target moves to camera position. the head bone slerps toward it with a 30-degree clamp — enough to feel responsive without breaking the idle pose. cursor leaves, the target resets to forward-facing

staggered wave activation on the “activate 3D” click prevents a frame spike. hosts initialize in groups of 3 with 200ms gaps between waves, spreading the mesh instantiation and texture upload across multiple frames

design tokens (–sh-* namespace)

these tokens extend the bryanchasko.com design system for the sumerian-squares scope:

all tokens scoped under .sumerian-squares — zero global pollution to the rest of bryanchasko.com. prefers-reduced-motion media query kills all animations and sets transitions to instant (0ms). the clamp values on gap and cell-size handle responsive scaling without breakpoints — the grid flows naturally from mobile through ultrawide

the stacked split screen redesign

the old layout was a flex-column stack. everything stacked vertically, the page scrolled, elements drifted out of view on smaller screens. Nova Act could not interact with elements that had scrolled off — the automation framework needs everything visible simultaneously. mobile was unplayable because the grid and arena competed for vertical space in an unbounded column

the redesign locks the viewport to 100dvh with overflow: hidden. zero scrolling at any viewport size. a 3-row CSS grid divides the viewport into nav / arena / grid:

.sh-layout {
	display: grid;
	grid-template-rows: auto auto minmax(0, 1fr);
	height: 100dvh;
	overflow: hidden;
}

the three rows: nav (auto — shrinks to fit status text and score), arena (auto, capped at max-height: 40dvh — holds the 3D host viewport), grid (1fr fills whatever remains). the minmax(0, 1fr) on the grid row is critical — without it, grid items default to min-height: auto which prevents the 1fr track from shrinking below its content’s intrinsic height. this caused cells to overflow the viewport on tablet. the min-height: 0 fix (via minmax) lets 1fr actually mean “fill remaining space, even if content wants more”

the arena is state-driven. its content rotates through 5 states: idle (poster image), question (trivia text), result (correct/wrong feedback), help (rules overlay), intro (Eye of Enlil greeting). each state maps to a child element that toggles visibility — the arena container itself never changes size, only its content swaps

responsive breakpoints

four breakpoints reshape the layout without breaking the one-screen contract:

the phone layout inverts the hierarchy: the grid dominates because touch targets need maximum surface area. the arena becomes a transient overlay anchored to the bottom of the viewport, triggered by cell selection. this avoids the trap of shrinking 9 interactive cells to make room for a 3D viewport that barely fits

the sumerian material texture system

every visual surface references a Mesopotamian material tradition. the cells are not styled — they are textured:

• clay tablet cells — repeating-linear-gradient at -45deg across each .sh-cell mimics cuneiform wedge impressions in wet river clay. the gradient uses --sh-panel and a slightly lighter variant at 4px intervals
• lapis lazuli active glow — the selected cell’s border transitions to --sh-glow-violet with a box-shadow bloom (0 0 20px). references the blue inlay of the Standard of Ur
• gold leaf nameplates — host names render in --sh-glow-amber with a dual text-shadow: warm highlight above (gold catch-light) and dark below (impressed into clay)
• cylinder seal impression — the name label text-shadow pattern (dark 1px below, warm 0.5px above) produces the visual of text rolled into soft clay by a cylindrical seal — the ancient Sumerian method of marking ownership

active-state isolation

when a cell is selected, only that cell’s decorative LEDs twinkle. every other cell dims its LEDs to --sh-led-cool (the residual amber glow of a cooling filament). this is handled by a CSS class toggle:

.sh-cell.active .sh-led {
	animation: sh-led-twinkle 2s infinite;
}
.sh-cell:not(.active) .sh-led {
	opacity: 0.3;
	background: var(--sh-led-cool);
}

the isolation draws the eye to the speaking host without requiring camera tricks on the grid itself. combined with the arena zoom, the active host dominates both the 3D viewport and the grid cell simultaneously

the custom game nav bar

the site header disappears on the game page via body:has(.sumerian-squares) > .header { display: none }. in its place, .sh-game-header replicates the site nav’s glassmorphism (backdrop-filter: blur(12px)) and animated gradient accent line but replaces menu items with game state:

• left: game title + current round indicator
• center: status text (whose turn, what phase)
• right: score display + help button (?)

the gradient accent line uses the same @keyframes as the site nav but shifts the color stops to the sumerian palette — amber through violet instead of the site’s purple-through-lavender. same motion language, different material vocabulary

accessibility

the game implements the ARIA grid pattern:

• role=grid on the 3x3 container
• role=row on each row
• role=gridcell on each cell
• aria-live=polite on the namecard region (announces host selection to screen readers without interrupting)
• descriptive alt text per host: “cristine, queen of heaven”

keyboard interaction follows the roving tabindex pattern: the focused cell has tabindex=0, all others tabindex=-1. arrow keys move focus between cells. this means tab enters the grid, arrows navigate within it, tab leaves — matching user expectations for grid widgets

visual design choices with accessibility intent:

• :focus-visible glow ring (not :focus) avoids showing outlines on mouse clicks while preserving them for keyboard users
• 44px minimum touch targets on all interactive elements — AGREE, DISAGREE, grid cells, activate button
• X and O marks distinguished by shape and color (not color alone) — colorblind users can distinguish game state by geometry
• prefers-reduced-motion: all animations frozen, transitions instant. no motion sickness triggers
• keyboard-only gameplay fully supported — every interaction reachable without a mouse
• AudioContext not created on keyboard navigation events (firefox compliance) — prevents audio system from entering a permanently locked state when the first interaction is a keypress

performance optimizations

the game’s loading strategy prioritizes perceived speed over total transfer size:

• deferred BabylonJS — both babylon.js (1.5MB) and babylonjs.loaders.min.js (120KB) load with defer fetchpriority=low. the game is playable as a 2D grid before the 3D engine loads. first paint is never blocked by the engine download
• Draco mesh compression — host meshes converted from .gltf + .bin to single Draco-compressed .glb files. decoder configured via CDN-hosted wasm (zero install). mesh data compressed 15-33% per host
• audio preload — first host’s audio preloads lazily via requestIdleCallback after page load. eliminates perceptible delay on first cell click
• question bank preload — <link rel="preload" as="fetch"> starts the question bank download before any JavaScript executes
• immutable CDN caching — 3D assets on CloudFront use max-age=31536000, immutable. content-hashed JS bundles enable 24h cache on page assets

lighting

the scene uses two lights:

• hemispheric ambient — HemisphericLight("hemi") at intensity 0.8, illuminates from above
• directional rim — DirectionalLight("rim") at intensity 0.4 with violet diffuse Color3(0.6, 0.5, 0.9), aimed from behind the host (Vector3(0, 0.3, -1)). separates the character from the dark background per VLM visual quality feedback

POI gaze tracking

when a host activates, poi-tracker.ts loads the host type’s poi.glb from CloudFront, retargets the center-look animation group onto the host skeleton as an additive layer. the host subtly tracks toward the camera (player). simplified from fiona-embed’s full 27-clip per-frame system to a single additive group appropriate for the single-host game context

game-over screen

when a match ends, showGameOver() creates a DOM overlay with:

• result messaging (you win / you lose / draw) with color-coded classes (--win green, --lose orange, --draw amber)
• match stats (games won by each player)
• fade-in animation via @keyframes sh-fade-in
• play-again button remains below the overlay

the game-over template lives in the Hugo partial (<template id="sh-game-over-template">) to co-locate markup with its CSS styles

host greeting flow

when a player clicks a cell, the game shows a greeting before the question:

1. host name + rotating greeting text appears in the quip area (“ah, a bold choice!”, “welcome, challenger…”, etc)
2. wave gesture plays during the 1.5s greeting pause (playGesture("wave", 1200))
3. after 1.5s, the actual trivia question replaces the greeting
4. quip audio loads and plays with lipsync

this creates the “host comes alive” feeling that bridges the gap between clicking a static poster and seeing animated speech

the Eye of Enlil

the game master is an orb. no character model, no mesh — pure CSS. the Eye of Enlil drives gameplay: asks trivia questions, introduces the game to first-time players, announces rounds and results

the visual design is concentric rings built from nested divs with border-radius: 50% and absolute positioning:

• outer ring: diorite stone (dark, textured border)
• middle ring: lapis lazuli (deep blue gradient fill)
• inner ring: gold (radial gradient from amber center to transparent edge)
• center: radial-gradient eye (dark pupil, amber iris, white highlight dot)

active state: the rings rotate via CSS keyframes (outer clockwise 12s, middle counter-clockwise 8s), the inner gold ring pulses in opacity (0.7 to 1.0, 2s cycle), and the eye center gains a box-shadow glow in amber

the intro sequence fires on first visit per session. the Eye appears with a fade-in, displays “I am Enlil, keeper of the tablets. Shall we test your wisdom?” in the arena text area, and renders a BEGIN GAME button below. clicking BEGIN dismisses the intro and sets localStorage.setItem('ss-intro-seen', 'true'). subsequent visits within the same session skip the intro and drop directly into the game grid

HTML structure: .sh-enlil container with .sh-enlil__outer, .sh-enlil__middle, .sh-enlil__inner, .sh-enlil__eye nested inside. no canvas, no SVG — the entire orb is box-model geometry with gradients. this means it renders identically on every device without WebGL support

reduced motion: all ring rotations and pulse animations are disabled under prefers-reduced-motion: reduce. the orb renders static — rings visible but frozen, eye visible but not glowing. the intro text and BEGIN button remain interactive

future: TTS voice for the orb reading questions aloud. the eye would pulse in time with speech amplitude, using the same AudioAnalyser approach as the LED system. not implemented yet

sumerian material textures

five material traditions from ancient Mesopotamia map to CSS techniques:

the active state lapis lazuli glow uses layered box-shadow: 0 0 20px rgba(38, 97, 156, 0.6), 0 0 40px rgba(38, 97, 156, 0.3), inset 0 0 10px rgba(38, 97, 156, 0.2). three layers create the depth — tight inner glow, medium spread, wide atmospheric bloom

the gold leaf nameplate on active cells uses color: var(--sh-glow-amber) with text-shadow: 0 0 8px rgba(218, 165, 32, 0.6). the amber glow mimics light catching hammered gold foil

color-material mapping for game state marks:

active-state isolation

the problem: all LEDs on all cells animated simultaneously. 9 cells x 4 LEDs each = 36 blinking lights competing for attention. no focal clarity, no hierarchy, visual noise

the solution uses :has() for ancestor-aware isolation. when any cell gains the .sh-cell--active class, the grid container responds:

.sh-grid:has(.sh-cell--active) .sh-cell:not(.sh-cell--active) .sh-led {
	animation: none;
	opacity: 0.15;
}

.sh-grid:has(.sh-cell--active) .sh-cell:not(.sh-cell--active) .sh-poster {
	filter: brightness(0.5) saturate(0.6);
}

.sh-grid:has(.sh-cell--active) .sh-cell:not(.sh-cell--active) .sh-nameplate {
	opacity: 0.4;
}

the active cell’s LEDs switch to gold (var(--sh-glow-amber)) and blink faster (1.2s cycle instead of the ambient 3s). all attention funnels to the speaking host

opponent turn dimming: when the AI is thinking, .sh-grid--opponent-turn applies to the container. all cell animations reduce further — LEDs dim to 0.1 opacity, posters desaturate. signals “wait, this is not your interaction moment”

LED invitation flash at turn start: when control returns to the player, .sh-cell--available-flash fires a 1-second pulse animation on all unclaimed cells. the LEDs briefly flash amber then return to ambient — a visual tap on the shoulder saying “your move”

responsive design — 4 breakpoints

the phone breakpoint inverts the hierarchy because touch targets need maximum surface area. shrinking 9 interactive cells to fit a 3D viewport above them makes neither usable. the bottom-sheet pattern keeps the grid at full size and presents the arena as a transient overlay

center stage (mobile)

on phone viewports (<480px), when a host is active, the grid collapses to show only that host’s cell at full width. this is center stage mode

.sh-main--center-stage toggles via JavaScript — a matchMedia('(max-width: 480px)') listener watches viewport width and activates the mode when a cell gains .sh-cell--active on a narrow screen

non-active cells: display: none. the grid visually becomes a single card

active cell sizing: width: 100%; height: 100%; max-height: calc(50dvh - 36px). the 36px subtraction leaves room for the arena bottom-sheet when it slides up from below

entry animation: @keyframes sh-center-stage-in — scale from 0.9 to 1, opacity from 0.5 to 1, duration 300ms with ease-out timing. the cell “zooms forward” into focus

exit: when the cell is deactivated (host finishes speaking, answer resolved), center stage dismisses. the grid returns showing all 9 cells with their X/O marks visible

per-host camera framing — superseded by measured geometry

historical note: an earlier build used a CAMERA_FRAMING lookup table keyed by HostType with fixed target.y / radius per type (adult_female, adult_male, alien). that could not frame two hosts of the same type correctly — cristine and maya are both adult_female yet sit at different rig heights, so a single target.y landed on cristine’s eyes and maya’s chin.

the fixed per-type table now survives ONLY as a fallback. the primary path is the measured-geometry framing documented under “camera system — measured-geometry framing” above: measure the model’s bounding box after load and derive target/radius/beta from the measurement. see that section to reproduce the approach; the constants live in camera.ts

steal mechanic

game flow for stealing:

1. player selects a cell and answers the trivia question
2. player answers wrong — the square is not immediately claimed
3. “stealing” phase activates — the opponent (AI) gets a chance to answer correctly
4. if the AI steals successfully: the square is claimed by the AI

AI steal behavior:

• visible 1.5s “thinking” delay (the Eye of Enlil pulses while the AI “considers”)
• 60% correct rate (random, not adaptive — keeps the game accessible)
• “OPPONENT STEALS!” text flashes in the arena with a red highlight animation
• the cell’s O mark animates in with the standard violet glow

player steal (reverse scenario):

• AI answers wrong on its turn
• question re-shown to the player with “YOUR TURN TO STEAL” prompt text
• agree/disagree buttons reappear
• player answers — if correct, steals the square with an X mark

if the steal attempt fails (neither player answers correctly): the square stays unclaimed. the cell returns to its neutral state. play advances to the next player’s turn

state machine flow: select → answering → deciding → stealing → (award or unclaimed) → select

audio speed and lipsync sync

all host audio plays at 1.2x speed via AudioBufferSourceNode.playbackRate.value = 1.2. this keeps gameplay snappy — quips that would feel sluggish at 1.0x feel conversational at 1.2x without sounding chipmunk-fast

lipsync timing scales proportionally. the viseme applicator’s step function multiplies elapsed time by PLAYBACK_RATE when looking up the current viseme mark from the Polly JSON timing data. if the audio is at 1.2x, the viseme lookup advances 1.2x faster through the mark array

the wasm visemeWeights output is always computed at 1.0x timing — the raw curve evaluation does not know about playback rate. scaling happens in the render-loop driver (lipsync-driver.ts), not inside the wasm crate. this separation means playback rate is adjustable without recompiling wasm

PLAYBACK_RATE is exported as a constant from audio-manager.ts. single source of truth — changing it in one place adjusts both the audio playback and the lipsync timing proportionally

Nova Act validation infrastructure

heraldstack-nova-mcp runs at port 8170 on rocm-aibox (streamable HTTP transport). it exposes 5 tools to the haunting agent ecosystem:

the dual-verification pattern separates concerns into three layers:

• Layer 1 (programmatic) — BabylonJS engine state checks. zero AI involvement. asserts: scene.meshes.length > 0, animationGroups playing, camera.radius within expected range. these are deterministic pass/fail gates
• Layer 2 (AI visual observation) — Nova Act navigates the page, observes the rendered output, describes what it sees. “I see a 3D character with mouth movements synchronized to audio” vs “the character appears frozen.” qualitative, not quantitative
• Layer 3 (VLM scoring) — Amazon Nova Lite Vision Language Model scores screenshots against criteria: animation quality, visual hierarchy, text readability, contrast. returns a numeric score per criterion

the gameplay play-through: 3 complete games played end-to-end by Nova Act. 38 screenshots captured at key interaction points (cell click, question display, answer reveal, mark placement, game over). evidence published at dev.clouddelnorte.org/_previews/sumerian-squares/

integration with CodeBuild: the post-deploy phase runs python3 tests/nova-act/sumerian-squares-dual-verify.py. non-zero exit code fails the deploy. evidence artifacts stored at /tmp/nova-dual-verify/ during the build, then pushed to s3vectors for historical tracking

auto-versioning

the VERSION file at repo root is the single source of truth for the game’s version number. the scheme is a single monotonic counter 0.XXXX (not semver) — one running release number, bumped by exactly 1 on every deploy that changes what a player sees or does

Hugo reads it at build time via {{ readFile "VERSION" | strings.TrimSpace }} and injects the value into the game template. the badge renders v0.XXXX (<git-sha>) bottom-right of the game viewport via .sh-version — 0.55rem font size, 30% opacity. the human number is for conversation, the short SHA (auto-stamped at deploy into VERSION-BUILD) pins the exact commit. visible enough to identify which build you are looking at, invisible enough to never compete with gameplay

each deploy bumps the version manually. enables actionable communication between dev and QA: “I see a bug on v0.1013” maps to a specific commit range. future: buildspec reads VERSION, increments the counter, commits back — fully automated version tracking per deploy