Moodle Course Builder: CI Pipeline, Nova Act Regression, and Kiro Orchestration

update (2026-09-06): the bryanchasko.com site has since retired CodeBuild entirely for a one-minute local deploy. the next chapter is in local-first deploy: retiring CodeBuild.

The moodle course builder is a Chrome MV3 extension that takes a CSV spreadsheet and builds a complete Moodle course — sections, activities, and questions — by driving Moodle’s actual DOM. No API wrappers, no mock forms. The extension opens the real quiz editor, fills real fields, clicks real buttons, and waits for real page loads across 6 question types: multichoice, truefalse, numerical, essay, shortanswer, and matching.

The CI pipeline that gates every push to this extension runs Nova Act regression tests against a live Moodle instance, validates OTEL instrumentation via Jaeger span gates, and was stood up in under an hour by a Kiro CLI agent swarm. Here is the full topology.

the ci migration

Woodpecker CI served 35 repos on a single AMD workstation for months. When it died, CodeBuild replaced it in under an hour — not through manual console clicking, but through Kiro CLI agent orchestration. One session. One agent persona dispatching ghosts for IAM, ECR, webhooks, SSM parameters, and the CodeBuild project itself. The entire infrastructure materialized as a single coordinated deployment.

The self-hosted era carried its own value (zero per-minute billing, full hardware control), but CodeBuild brings managed scaling, native AWS credential chains, and no single-machine failure mode. The migration was less a decision than an obvious next step once the agent tooling existed to execute it cleanly.

pipeline topology

Five pipelines cover the extension’s quality surface:

lint-test-build

Fires on every push and PR. Runs biome (formatting + lint), TypeScript type checking, jest unit tests, and webpack production build. If the dist output is malformed or tests regress, the push is rejected before any human reviews it.

# codebuild/buildspec-tier1.yml (simplified)
phases:
  build:
    commands:
      - npm ci
      - npx biome check .
      - npx tsc --noEmit
      - npx jest --ci --coverage
      - npm run build
      - node scripts/verify-dist.js

nova-act-regression

The real validation surface. Amazon Nova Act v1 drives a live Moodle instance through the extension’s complete flow — CSV upload, plan generation, build execution, and verification that all 6 question types land correctly in Moodle’s database. Fires on PRs touching src/ or tests/, and on every push to main.

nightly-regression

Runs the full integration suite plus a Jaeger span gate. The extension ships built-in OTEL instrumentation — every step dispatch, every DOM operation, every webservice call emits a span. The nightly pipeline queries Jaeger for the expected span tree and fails if instrumentation has regressed. Coverage without an E2E run is a half-truth; this validates the observability layer independently.

screenshot-freshness

A pre-push git hook, not a pipeline. Regenerates product page screenshots via headless Chrome + CDP, then diffs against committed versions. Stale screenshots fail the push. The product page always shows what the extension actually looks like today.

image-integrity

Deploy-time check. Every screenshot referenced in the product page HTML must exist in the build output. Missing images — from a renamed file, a deleted screenshot, a broken path — fail the deploy before CloudFront sees it.

the custom ci image

CodeBuild needs Chrome, Xvfb, and Node 22 in the same container to run extension tests. The standard AWS images do not carry Chrome for Testing. So a custom image lives in ECR:

• Chrome for Testing (stable channel, matching chromedriver)
• Xvfb on display :99
• Node 22 LTS
• npm, webpack, jest, biome — all baked in
• Total size: 1.43 GB

# build and push
docker build -t chrome-extension-ci:latest -f codebuild/Dockerfile .
aws ecr get-login-password | docker login --username AWS --password-stdin $ECR_URI
docker tag chrome-extension-ci:latest $ECR_URI:latest
docker push $ECR_URI:latest

The image carries the three Chrome flags that prevent background tab throttling under Xvfb:

--disable-background-timer-throttling
--disable-backgrounding-occluded-windows
--disable-renderer-backgrounding

Without these, Chrome throttles the very tabs the extension creates — the service worker dispatches to a new tab, and Chrome starves it of execution time because no real display exists.

nova act testing

This is where it gets interesting. Amazon Nova Act v1 is a foundation model that drives browsers through natural language instructions, backed by Playwright for low-level page interaction. The extension’s regression suite uses it via Bedrock @workflow to validate the full user flow end-to-end.

what it tests

The suite uploads a structured CSV, confirms the generated build plan, executes the build, then verifies the output in Moodle:

• CSV upload and parse (column mapping, question type classification)
• Plan generation (sections, activities, question counts per type)
• Build execution (38 build steps for a typical course)
• Question type verification — each type has its own DOM path through Moodle’s quiz editor:
  • multichoice: 4 option fields, one marked correct
  • truefalse: binary toggle
  • numerical: numeric input with tolerance
  • essay: TinyMCE iframe initialization (known timeout-sensitive under Xvfb)
  • shortanswer: single text field with case sensitivity flag
  • matching: pipe-separated term/definition pairs

how it runs

Nova Act connects to a managed Chromium instance via Bedrock AgentCore Browser Tool. The test script loads the extension, navigates to the local Moodle instance at 192.168.4.53:8765, and issues natural-language instructions that Nova Act translates into browser actions:

async with nova_act.ActSession(
    starting_page=f"{MOODLE_URL}/course/view.php?id={course_id}",
    ignore_https_errors=True
) as session:
    # upload CSV via the sidepanel
    result = await session.act(
        "Click the extension icon to open the side panel, "
        "then upload the file at /tmp/test-course.csv"
    )

    # confirm the build plan
    result = await session.act(
        "Review the build plan showing 4 sections and 24 questions, "
        "then click Start Building"
    )

    # wait for completion
    result = await session.act(
        "Wait for the build summary to show all steps completed"
    )

The full suite runs with 4 parallel workers. A healthcare course with 24 questions across all 6 types validates end-to-end in 479 seconds — under 8 minutes for complete confidence that the extension works against real Moodle.

why nova act over puppeteer

Puppeteer scripts break when the DOM changes. A renamed CSS class, a restructured form, a Moodle theme update — any of these invalidate hardcoded selectors. Nova Act operates on visual and semantic understanding of the page. When Moodle updates its quiz editor layout, Nova Act adapts without script changes. The test describes intent (“fill in the correct answer field”), not implementation (document.querySelector('#id_answer_0')).

The tradeoff is speed and cost. Nova Act is slower per action than a direct Puppeteer click. But the maintenance cost of brittle selectors across Moodle version bumps is higher than the per-invocation cost of a model that reads the page like a human does.

kiro cli orchestration

The CI infrastructure did not materialize through a CloudFormation template or a Terraform module. It was built by Kiro CLI agents in a single session. The orchestration layer:

poltergeist-harald-moodle-ext-dev

The sole-persona super-agent for this extension. Product owner, sprint coordinator, and dispatch authority. Does not write source code — decomposes work, dispatches ghosts, synthesizes results, and decides what ships. Operates under enforcement gates that structurally prevent it from touching src/, tests/, or scripts/ directly.

ghost-orin-ci-cd

GitHub operations specialist. Creates branches, commits, pushes, opens PRs, merges after review passes. Owns the buildspec.yml files and CodeBuild project configuration. When harald decides the CI needs a new pipeline, orin builds and ships it.

ghost-ellow-e2e-tester

CDP screenshot capture and headless browser validation. Drives Chrome for Testing under Xvfb, captures screenshots at defined breakpoints, and writes artifact files that other agents consume. Does not interpret results — captures evidence.

the design scorecard

After ellow captures screenshots, a vision-model review scores each capture against 7 dimensions: layout integrity, color contrast, text readability, interactive element visibility, responsive behavior, brand consistency, and accessibility compliance. Scores below threshold block the PR. The extension’s UI is continuously validated against design standards without a human reviewer looking at every pixel.

single-session infrastructure

The full CI stack — CodeBuild project, ECR repository, GitHub webhooks, SSM parameter paths for credentials, IAM roles with least-privilege policies — was dispatched and verified in one agent session. Not one day. One session. The agent swarm parallelized IAM policy authoring with ECR creation, sequenced webhook registration after the CodeBuild project existed to receive events, and validated the full chain by triggering a test build before reporting completion.

the numbers

what this means

A browser extension that automates a complex LMS workflow now has a CI pipeline where the regression tests operate at the same level of abstraction as the users — driving real browsers through real pages, verifying real outcomes. The test suite does not mock Moodle. It does not stub the extension’s message passing. It loads the extension into Chrome, uploads a CSV, and checks that the course exists in Moodle when it is done.

The orchestration layer that built this infrastructure is the same one that builds features, reviews PRs, and manages releases. The agents that wrote the CodeBuild configuration are the agents that will update it when requirements change. There is no gap between “who built the CI” and “who maintains the CI” — it is the same system, with the same context, operating continuously.