ChatGPT unit tests: make the assertions prove the function's contract

A generated test passes after the function starts changing its input. The expected return is right, but nobody asserted that the caller's data must stay unchanged. Give ChatGPT the full contract before asking for tests, including side effects the function must avoid. Then run the tests and check that a deliberately broken version fails for the intended reason.

This is a small Node.js example. You can paste the function and contract into ChatGPT; no repository connection is required. OpenAI's prompting guidance supports defining the result and relevant constraints. The test design below is an original worked example, rather than a recorded AI response.

Separate the expected behavior from the implementation

The function receives an array of strings and returns a new array containing only strings whose length is at least a supplied minimum. It must preserve order and leave the input unchanged. The minimum in this example is a supplied nonnegative integer; validation of other types is outside this contract.

export function longEnough(words, minimum) {
  return words.filter(word => word.length >= minimum);
}

With ['oak', 'pine', 'elm', 'cedar'] and minimum 4, the expected return is ['pine', 'cedar']. The word pine checks the exact boundary. oak checks below it; cedar checks above it.

Compute those expectations from the stated rule. Do not ask the implementation to calculate the test's expected result using the same logic: the same defect could then appear in both places.

Cover the boundary, empty result, and unchanged source

Save the function as long-enough.mjs and the tests as long-enough.test.mjs:

import test from 'node:test';
import assert from 'node:assert/strict';
import { longEnough } from './long-enough.mjs';

test('keeps the exact boundary and original order', () => {
  const input = ['oak', 'pine', 'elm', 'cedar'];
  const result = longEnough(input, 4);
  assert.deepEqual(result, ['pine', 'cedar']);
  assert.deepEqual(input, ['oak', 'pine', 'elm', 'cedar']);
  assert.notStrictEqual(result, input);
});

test('handles empty input and no qualifying strings', () => {
  assert.deepEqual(longEnough([], 4), []);
  assert.deepEqual(longEnough(['a', 'bb'], 3), []);
});

test('zero includes empty strings', () => {
  assert.deepEqual(longEnough(['', 'a'], 0), ['', 'a']);
});

Run node --test long-enough.test.mjs in the folder containing both files. Node's test-runner documentation describes the built-in runner, and its assertion documentation explains the comparisons. Use a maintained Node version with these built-in modules available.

The unchanged-input assertion checks the array's content. The reference assertion checks that the returned array is a different object. Both are needed for this contract: equal contents alone would allow the function to return the same array.

Check whether a plausible defect is caught

Temporarily changing >= to > should make the boundary test fail because pine disappears. Restore the correct code afterward. If the test still passes, inspect whether it really calls the edited function and whether its assertions include the boundary value.

This small mutation check answers a specific question about one defect. It does not prove that every possible bug is covered. A percentage labeled coverage also cannot tell you whether the assertions express the right requirement.

Copy a contract-to-tests prompt

Draft unit tests for the function below.
Runtime, module format, and test framework: [ENTER]
Function and relevant dependencies: [PASTE]
Contract, including boundary rules and forbidden side effects: [PASTE]
Inputs outside the agreed contract: [PASTE]

Derive explicit expected values from the contract. Cover meaningful
boundaries, empty cases, output order, and required source preservation.
Do not duplicate the production algorithm to compute expectations.
Identify unresolved behavior instead of inventing a requirement.
Provide runnable tests and the exact command for this environment.
Suggest one small defect the tests should detect and explain which
assertion would fail. Do not claim any test passed without execution.

The broader requirements-to-test-cases guide handles state transitions and specification questions before automation. Here, finish by running the function-level tests, checking a meaningful failure, restoring the intended implementation, and running the suite again. Keep the resulting tests beside the code they protect.

Keep reading