AWS released AWS Emulator on November 11. It is a local copy of more than 200 AWS services. Autonomous agents wrote every line of its code. Senior principal engineer Matthew Goldberg said his team neither wrote nor reviewed any of it.
The emulator serves AWS APIs from your laptop. The CLI, SDKs and coding agents connect to a local HTTPS endpoint. They create queues, tables and buckets that live in memory. No AWS account is needed.
Three agent teams and a merge gate
Three teams of agents built it. The first writes fidelity tests from API models and verifies each one against real AWS. The second replays those tests against the emulator and files issues when behavior diverges. The third team does root-cause analysis and writes fixes.
Every change passes a merge gate. The gate blocks code that fails to compile, violates coding standards or fails security checks. It also blocks changes that regress fidelity. Once merged, the code flows through a release pipeline on AWS Developer Tools.
The agents started from an empty web server that handled requests for 438 AWS API models. It did nothing with them. The first working emulator took about 15 minutes. An agent built an in-memory Amazon SQS implementation that passed its own tests.
The fidelity score is public
AWS publishes a fidelity score with each release. At launch, the emulator passed 24,243 of 42,937 tests, or 56.5 percent. Of the 68,007 individual steps in those tests, 73.2 percent pass. Many scenarios fail on one step out of several.
Service scores vary widely. Among the 342 services with at least ten tests, the median pass rate is 61.3 percent. Forty services score 90 percent or more. Another 124 score below 50 percent. EventBridge sits at 71.5 percent. CloudFormation is at 18.9 percent.
AWS says the emulator does not evaluate IAM policies the way AWS does. It also skips cross-region behavior. The evaluation suite is public on GitHub with full commit history.
What I would check
In Atelier, the agent shows me its plan before it sends, posts or drafts email. I read the plan, then I approve or refuse. The agent writes code I never review, but it cannot ship without showing its hand first.
The Punchlist turns a bottleneck into a scoped fix. Its AI advisor writes automation, but a human decides whether to run it. I would check whether AI Trader could log what each model said before the trade.
AWS Emulator puts the trust on a different layer. The agents write code no human sees. The gate checks whether the code compiles, meets standards and passes security scans. It also checks fidelity against real AWS. The score tells you where the gaps are.
The engineers set the rules and read the scorecard.
I would check the fidelity score for each service I need. Then I would decide whether to trust it in CI. A 56.5 percent pass rate means half the scenarios still fail.