Terraform from my phone

Over the last two days, mostly from my phone, I’ve directed Claude through standing up a new AWS organisation, a sandboxed member account, a permissions-bounded identity for itself, and then a working AI demo on top: a court-interpretation quality tool with a streaming speech pipeline, an LLM judge running in-region on Bedrock, and a public status site. My hands touched a keyboard for maybe twenty minutes of that, almost all of it pasting pre-written blocks into CloudShell as root.

I’ve got form working this way – the anthem-scoring method was built the same way, from a train in Japan – but that was open-web analysis, where the worst case was a wrong chart. This time the agent held real cloud credentials and could spend real money, and the reason I stayed comfortable with that is everything we put in place before it wrote a line of HCL. So this post is mostly about guardrails.

The last time I did this properly

My last serious infrastructure work was at FCDO, adding Apache Superset to an existing stack. I liked Terraform a lot, and it still cost me the best part of a week to get my head around it. The loop was the tax: plan, apply, cryptic error, forum archaeology, try again. Almost all of that week went on syntax and state rather than the questions that mattered, like who can touch what, and what this all costs when nobody’s looking.

This time the week simply didn’t happen, because I wasn’t the one holding the Terraform. I held the constraints instead, which I’ve come to think is the better job. Two days of reviewing an agent’s guardrails taught me more about IAM boundary design than years of sitting near cloud teams in government did.

The division of labour

The split between me and the agent mostly forced itself, via an AWS fact I’ve become quite fond of: the root user cannot assume IAM roles. At all, ever. The most privileged identity in the estate is the one least able to delegate, so anything root-only – creating the organisation, changing the agent’s own permissions, accepting Bedrock model agreements – has to be a deliberate human act, whether I’m feeling disciplined or not. AWS polices my lane for me.

flowchart LR
    subgraph human["My lane · a phone and one CloudShell tab"]
        R[Me, directing by chat]
        CS[CloudShell as root<br/>pasted blocks only: bootstrap ·<br/>boundary changes · model agreements]
    end
    subgraph agent["Claude's lane · everything else"]
        CC[Claude Code session<br/>orchestrator plus cheaper subagents]
    end
    subgraph mgmt["Management account"]
        ORG[Organisation · payer-side budget<br/>alerts at 50 / 90 / 100%]
    end
    subgraph member["Member account · locked to eu-west-2 by a permissions boundary"]
        IAM[claude-dev keys, whose only power<br/>is assuming claude-orchestrator:<br/>admin ∩ boundary]
        TF[Terraform: VPC · Fargate · ALB · ECR<br/>CodeBuild · Bedrock · the status site]
        CT[CloudTrail on from minute one]
    end
    R --> CC --> IAM --> TF
    R --> CS --> ORG
    CS -.->|the only path to the leash| member

    style R fill:#2d8cff,stroke:#fff,stroke-width:2px,color:#fff
    style CS fill:#f59e42,stroke:#fff,stroke-width:2px,color:#fff
    style CC fill:#10b981,stroke:#fff,stroke-width:2px,color:#fff
    style IAM fill:#1e293b,stroke:#fff,stroke-width:1px,color:#fff
    style TF fill:#1e293b,stroke:#fff,stroke-width:1px,color:#fff
    style CT fill:#1e293b,stroke:#fff,stroke-width:1px,color:#fff
    style ORG fill:#6366f1,stroke:#fff,stroke-width:2px,color:#fff

The workflow settled quickly: Claude writes an idempotent bootstrap script and a paste-block for anything root-only, I read it on my phone, paste it into CloudShell, and paste the output back. Everything downstream of that runs through an identity Claude designed for itself – and, importantly, designed to constrain itself.

The guardrails

This is the part I’d urge anyone trying the same to get right first, because how much autonomy you can sensibly grant depends on how real your guardrails are. Ours were all proposed by the agent and approved by me:

• A separate member account. Blast radius becomes a property of the account. The worst case is “close the account”, and billing shows the project as its own clean line.
• A permissions boundary as the ceiling. The agent’s role is admin intersected with a boundary policy: nearly everything works in eu-west-2 (London), and almost nothing works anywhere else. No Marketplace subscriptions, no reserved instances, and an EC2 instance-type allowlist so a mistake can’t launch a £25-an-hour GPU.
• The boundary is out of the agent’s reach. It denies the agent any IAM action on its own identities or on the boundary policy itself, and any role it creates must carry the same boundary or creation fails. We tested this adversarially – role creation without the boundary, region hops, attempts to strip the boundary off – and the account refused each one.
• A payer-side budget with alerts at 50, 90 and 100 percent. (Learned the hard way: member accounts under consolidated billing can’t own budgets at all.)
• Torn down as the default state. The expensive pieces – load balancer, compute – are off unless a variable says otherwise, so every session ends at roughly zero pounds per day. Standing the demo up for a visitor is one command, and so is putting it away.

None of this is exotic. It’s bog-standard AWS, which is rather the point: the primitives were always strong enough to hold a hard boundary. Maintaining them just no longer costs a week of my life.

What it actually built

The demo deserves its own post, but briefly: it’s a glass-box workbench exploring AI quality assurance for court interpreting, a domain where roughly 1% of interpreter bookings are ever quality-checked. A synthetic ten-minute mock hearing with 18 deliberately seeded interpretation errors runs through a real pipeline: two speech-to-text engines scored against ground truth, machine translation, and an LLM judge on Bedrock that never leaves London. The judge currently catches 94% of the seeded errors at 74% precision, and the false positives are printed on the report rather than hidden – a QA tool that cries wolf is worse than none. There’s also a live mode for interpreters, with a measured 0.4 second latency against the 3 second ceiling the interpreting literature allows. A full QA run costs about 50p.

My favourite small mechanism is the status page. The public site needs to say whether the demo is up, and rather than run a health-check service, terraform apply itself writes a status.json into the site’s bucket on every run. Applies are the only way the infrastructure changes, so the page can’t quietly drift out of date – and the visitor’s browser double-checks anyway, by pinging the demo’s health endpoint itself.

flowchart LR
    A[terraform apply<br/>the only way demo state changes] --> S[status.json in the site bucket]
    S --> W[Static site on CloudFront]
    W --> V[Visitor's browser<br/>verifies /healthz itself]

    style A fill:#2d8cff,stroke:#fff,stroke-width:2px,color:#fff
    style S fill:#1e293b,stroke:#fff,stroke-width:1px,color:#fff
    style W fill:#1e293b,stroke:#fff,stroke-width:1px,color:#fff
    style V fill:#10b981,stroke:#fff,stroke-width:2px,color:#fff

The site is live at dzpq41hjedxw9.cloudfront.net if you want to see whether the demo happens to be up as you read this.

The potholes were the fun part

A few things went wrong along the way, all of the sort that would once have eaten half a day each. This time each one was a message or two:

• Root can’t assume roles, so the bootstrap needed an ephemeral bridge user that creates itself, does one job, and deletes itself.
• Bedrock model access silently flipped to “agreement required” mid-afternoon, hours after working invocations. The fix was root-side (Marketplace actions are deliberately denied to the agent) – one paste.
• Amazon Transcribe’s automatic language identification collapsed on code-switched courtroom audio: 44.5% word error rate. One fixed-language pass per language – the language pair is always known from the booking – took that to 8.5%, and the failure itself became demo material.
• Something in the middle of my network path strips WebSocket upgrade headers, which prompted the correct paranoid response: a polling fallback, because government networks will do the same.

Each was caught by a guardrail or a measurement rather than by luck, which is what let me keep directing from a phone instead of hovering over a terminal.

Learning to love it

I came back to AWS expecting the old feeling: powerful, hostile, a tax on attention. Most of the hostility turned out to live in the interface, and the interface is now negotiable. The primitives underneath – accounts as blast radii, budgets as tripwires – are exactly what you want when your junior engineer is a tireless language model. AWS gives you the material for a cage worth trusting, and the agent does the caged work better than I would.

Total cloud spend for the whole build so far is about two dollars, which is pleasing, though the week of my life it didn’t cost is worth rather more.