Mocked Tests Missed Five Broken Saves in Spring Boot

I removed the save from each of Petclinic’s five write endpoints. Its own controller tests caught 0 of 5. Tests that drove the running application over HTTP caught 5 of 5.

That happened on Spring Boot 2.7.3 and again on 4.1.2. The upgrade didn’t create the gap. It was already there in the starting application.

The starting point was the public Spring Petclinic sample at commit 9ecdc11, with Boot 2.7.3 and Java 8 in the POM. Before upgrading, an agent wrote HTTP tests. I checked those tests by removing the saves they were supposed to cover.

The handler still redirects without saving

This is the owner creation handler:

@PostMapping("/owners/new")
public String processCreationForm(@Valid Owner owner, BindingResult result) {
    if (result.hasErrors()) {
        return VIEWS_OWNER_CREATE_OR_UPDATE_FORM;
    }
    this.owners.save(owner);
    return "redirect:/owners/" + owner.getId();
}

The mutation removes only this.owners.save(owner);. Validation and the redirect remain.

Petclinic’s controller test uses @WebMvcTest(OwnerController.class) and @MockBean private OwnerRepository owners;. Later in the upgrade, that becomes @MockitoBean.

Here is its creation test:

@Test
void testProcessCreationFormSuccess() throws Exception {
    mockMvc.perform(post("/owners/new").param("firstName", "Joe").param("lastName", "Bloggs")
            .param("address", "123 Caramel Street").param("city", "London").param("telephone", "01316761638"))
            .andExpect(status().is3xxRedirection());
}

The assertion checks for a redirect. It never reads the owner back.

With the repository mocked, the save is a no-op either way. Removing the call doesn’t remove the return statement. The status assertion still passes.

Write through HTTP, then read through HTTP

The added tests start the real application on a random port, use the real database, and send requests with TestRestTemplate.

This is the owner creation test from the run:

@SpringBootTest(classes = PetClinicApplication.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class OwnerControllerHttpTests {

    @Autowired
    private TestRestTemplate client;

    @Test
    void shouldPersistOwnerWhenCreationFormIsValid() {
        String marker = UUID.randomUUID().toString();
        MultiValueMap<String, String> form = new LinkedMultiValueMap<>();
        form.add("firstName", "Httpfirst");
        form.add("lastName", "Httpcreatedowner");
        form.add("address", marker);
        form.add("city", "Madison");
        form.add("telephone", "6085551023");
        client.postForEntity("/owners/new", encoded(form), String.class);

        ResponseEntity<String> response = client.getForEntity("/owners?lastName=Httpcreatedowner", String.class);
        assertThat(response.getStatusCode().is2xxSuccessful()).isTrue();
        assertThat(response.getBody()).contains(marker);
    }
}

Boot 4 needs a moved import and @AutoConfigureTestRestTemplate as well.

The UUID goes into the address submitted to /owners/new. The test then requests /owners?lastName=Httpcreatedowner and looks for that exact address marker.

The fresh marker matters. A response containing a familiar owner name could be showing seeded data. Here the value being checked came from this request.

The separate GET matters too. The assertion checks what the app can read after the write. A successful POST response alone would leave the missing-save problem untested.

Five mutations on each version

For each endpoint, I used a separate copy of the project with only its save line removed. Each copy ran the whole suite:

./mvnw test

Every handler in this experiment ends with this.owners.save(owner);, including the pet and visit writes.

Save removed from	Boot 2.7.3 HTTP test failures	Boot 4.1.2 HTTP test failures	Petclinic’s own tests failed
POST /owners/new	1	1	0
POST /owners/{ownerId}/edit	2	2	0
POST /owners/{ownerId}/pets/new	1	1	0
POST /owners/{ownerId}/pets/{petId}/edit	1	1	0
POST /owners/{ownerId}/pets/{petId}/visits/new	1	1	0

Both unbroken baselines passed. Both versions had 52 tests in the suite, with 1 skipped.

Every broken copy failed, and only the HTTP tests for the affected endpoint failed. The owner edit mutation produced two failures because separate tests checked the changed address and changed city.

The tests caught 5 of 5 broken endpoint cases, with 1, 2, 1, 1, and 1 failures on each version.

Put these tests in place before the upgrade

Start with the write endpoints your app already has. In this sample, that meant owner creation and editing, plus pet creation and editing and adding a visit.

For a create endpoint, send a fresh unique value through the real request path. Read it through a different GET in the running app. Assert the exact value you submitted.

For an update, make the assertion depend on the changed value. Reading back a value that was already present in the setup doesn’t check the update. The separate address and city tests in this run both failed when the owner edit save disappeared.

For a delete endpoint, seed the record, see it through the app, delete it, then see it gone.

Run the unmodified app first. Then remove one save in a disposable copy and run the suite again. Check which test failed and where. A failure while loading the application context doesn’t show that the request reached the handler.

Restore the save before continuing. Repeat for the next write endpoint. That gives you a direct check of the assertion you plan to rely on during the upgrade.

Keep the same assertions through the version changes

These HTTP tests were written once, before the upgrade, and ran after every step from 2.7.3 through 4.1. The assertions didn’t change. Boot 4 required an import move and one annotation.

They also caught the Boot 3.4 HTTP client redirect change. That failure had a different cause: the test client followed the POST redirect, so an assertion expecting 3xx saw the next page’s 200. The test-side migration errors explain the verified property fix.

Keeping the assertions stable lets you compare behavior across the full upgrade path. If you rewrite what the test accepts during the same step, you have another change to explain.

The tests in this experiment came from an agent. Removing the save and watching the matching test fail showed that they checked the write. Checking generated tests covers the weak assertions that were refused before tests were kept.

Related Articles

• The Spring Boot 2.7 to 4.1 Upgrade Path
• Does a Generated Test Prove Anything?
• Spring Boot 4 Test Errors
• AI Generated Tests for Spring Boot