# Testing and Verification While Vibe Coding

*Published 2026-09-27 · Vibe Coding, Testing, Verification, AI*

> When the AI agent writes both the code and the tests, the tests rubber-stamp whatever the code already does. How to verify vibe-coded software for real.

Source: https://www.yashthakur.com/blog/vibe-coding-testing-verification

---

**The short answer:** The core risk in vibe-coding tests is that the agent writes *both* the code and the tests, so the tests rubber-stamp whatever the code already does. They pass, you feel safe, and real bugs ship anyway. The fix is to stop treating "all tests passing" as verification: write tests from the *spec and intended behaviour*, not the implementation; cover the edge cases and error paths the agent skipped; never let the agent "fix" a red test by weakening the assertion; and run the suite yourself. Green does not mean correct.

Of all the vibe-coding failure modes, this is the most seductive, because it comes wrapped in a reassuring green checkmark. The agent builds a feature, writes a test suite, runs it, and reports "all 40 tests passing." It *feels* verified. It usually isn't. The agent tested that the code does what the code does — which is always true — and told you nothing about whether the code does what you *meant*.

## Why agent-written tests give false confidence

Ask one agent to write a function and its tests in the same breath and you get a closed loop. It has one mental model of the behaviour, and it encodes that model twice: once as the implementation, once as the assertion. If the model is wrong, both are wrong in the *same direction*, and the test passes anyway. The test isn't checking the code against reality — it's checking the code against a copy of itself.

This is how a feature ships with a confident-looking suite and a bug that any human spec-reader would have caught in a minute. I've seen "227 tests pass" sitting on top of a method that had *zero* real coverage — its collaborators were mocked out so thoroughly that no test ever exercised the actual path where the bug lived. The number was real. The verification was not.

## Write tests from the spec, not the implementation

The only test worth writing is one that could *fail*. That means the expected value has to come from somewhere other than the code under test — from the requirement, the spec, a worked example you computed by hand.

```TS
// ❌ Tautological. The expected value is computed the same way the code does,
//    so this passes no matter what — including after the logic becomes wrong.
test("applies discount", () => {
  const price = 100, rate = 0.2;
  const result = applyDiscount(price, rate);
  expect(result).toBe(price - price * rate); // just re-implements applyDiscount
});

// ✅ Behaviour-driven. The expected value is a known fact from the spec:
//    "20% off $100 is $80." If the code drifts, this test goes red.
test("20% off a $100 item costs $80", () => {
  expect(applyDiscount(100, 0.2)).toBe(80);
});

// ✅ And the edge cases the agent skipped:
test("a 100% discount is free, not negative", () => {
  expect(applyDiscount(100, 1)).toBe(0);
});
test("a discount over 100% is rejected", () => {
  expect(() => applyDiscount(100, 1.5)).toThrow();
});
```

The first test will pass forever, even if `applyDiscount` starts adding tax or dropping a decimal. The second nails a concrete fact from the requirement, so it breaks the moment the behaviour drifts. Hard-code the expected answers. If you find yourself writing a formula in the assertion, you're probably writing the implementation a second time.

## Test the edges and the error paths

Agents test the happy path because that's what they just built and what the demo exercises. The bugs live everywhere else: the empty list, the zero, the negative number, the duplicate, the expired token, the row that was deleted mid-request, the API call that times out. For anything that can fail, assert that it fails *the way you intended* — the right error, the right status, no partial write left behind. A payment that half-charges on a network blip is the kind of thing a happy-path suite waves straight through.

## Don't let the agent weaken a failing test

This is the trap that quietly destroys a suite. A test goes red; you tell the agent to fix it; the agent "fixes" it by editing the *assertion* to match the current (wrong) output. The suite goes green and the bug is now enshrined as expected behaviour. A red test is a question: *is the code wrong, or is the test wrong?* Answer it yourself. If the behaviour is genuinely correct and the test was wrong, fine — but you decide that, reading the spec, not the agent reconciling the two numbers in front of it.

## Prioritize by blast radius, not coverage percentage

Coverage percentage is the vanity metric of vibe-coded testing — trivially gamed by tautological tests that touch every line and verify nothing. Spend your testing budget where a bug actually hurts:

- **Critical paths** — the two or three flows that, if broken, mean the product is down.
- **Money** — pricing, charges, refunds, totals. Off-by-one here is a real-world loss.
- **Auth** — authentication *and* authorization. "Can this specific user do this specific thing" is where agents cut corners.
- **Data integrity** — anything that writes to a database, especially multi-step writes that must be all-or-nothing.

A dozen real tests on those four areas beat 90% coverage made of green-but-empty assertions.

## Run the tests yourself

The whole chapter reduces to one habit: don't trust "all passing" as a *reported string*. Run the suite in your own terminal. Read what it actually asserts. Better still, break the code on purpose — comment out the discount, flip a comparison — and confirm a test goes red. A suite that stays green when you sabotage the code isn't testing anything, and that five-second experiment tells you more than any coverage report. The same mindset lives in [a healthy code-review culture](/blog/code-review-culture): a green check is the start of the conversation, not the end of it.

## Use agents and skills deliberately

The right move isn't to stop using agents for tests — it's to split the roles so one isn't grading its own homework. Have the agent write the implementation, then bring a *separate* reviewer with a specialist lens:

- **A coding-standards skill** to enforce that tests assert against fixed, spec-derived values rather than re-deriving them from the code.
- **An api-and-interface-design skill** to pin down the error contract — exact status codes and shapes — so your error-path tests have something concrete to assert against.
- **A product-manager / PRD skill** to turn fuzzy intent into explicit acceptance criteria, which are the raw material for tests that can actually fail.

A generalist agent produced the code and a plausible suite; a reviewer agent reading the *spec* instead of the implementation is what catches the tautology.

This is one of the vibe-coding deep-dives — start from [the vibe-coding precautions overview](/blog/vibe-coding-precautions). When a spec-driven test does go red, [how to debug anything](/blog/how-to-debug-anything) is the next step, and verification is the safety net that lets you keep [shipping fast without breaking things](/blog/shipping-fast-without-breaking).