Building Endpoints That Test Themselves

7th May 2026

We started building a local API test-building agent.

The big picture is an agent that can be dropped into any API repo, inspect the available endpoints, analyze the existing tests, find coverage gaps, and eventually generate missing API tests in the style of that repo.

So far, the agent can scan app.py and discover FastAPI endpoints like POST /user/, GET /user/{user_id}, and DELETE /user/{user_id}.

It can also scan the existing pytest files and detect which endpoints already have test coverage.

The important part is the generic API test stencil. This stencil defines common API test expectations, such as happy-path tests, not-found tests, invalid path parameter tests, missing payload tests, wrong data type tests, and malformed JSON tests.

The agent compares each endpoint against that stencil and reports what already exists and what is missing. For example, it can say that GET /user/{user_id} has a success test but is missing a not-found test and an invalid-ID test.

We also added a learning file where the agent can summarize the existing project's test style. That will matter later when we ask it to generate tests that look like the tests already in the repo.

Finally, we added unit tests for the agent itself so we can keep changing it safely.

The next step is to teach the agent to turn missing stencil items into actual pytest files.

The end goal is agentic API testing: analyze the repo, find the gaps, generate the tests, and keep coverage aligned as endpoints change.