~/
kellerstein
writing
about
rss
Writing
1 post · tagged evaluation · oldest Mar 2025
all
agents
26
architecture
19
ai
16
infra
10
kubernetes
6
debugging
5
testing
5
azure
4
python
4
product
3
tooling
3
docker
2
git
2
observability
2
qemu
2
refactoring
2
security
2
windows
2
automation
1
claude-code
1
evaluation
1
fastapi
1
helm
1
mcp
1
mitmproxy
1
models
1
retrospective
1
reverse-engineering
1
terraform
1
traefik
1
OpenOperator
1 post · 2025
github
watch demo
2025·03·26
"It looked like it worked": grading a non-deterministic agent
The least trustworthy signal in the system is the agent reporting success. Here is the evaluator we built instead — and the four-word commit that forced a whole second repository into existence.
agents
testing
evaluation
7 min