How I Cut 200 Test Cases Down to a Single-Page Smoke Test

Last month, while organizing my desk drawer, I found a printout of a test plan from a project three years ago. Twelve sheets of A4 paper. Double-sided. 9-point font. I held it up, sighed, and thought: did we really run this every single time?

Honestly, back then I believed that more test cases meant more safety. High coverage numbers put my mind at ease. But the reality was different. The night before a release, I'd blaze through 200 cases, stamping "pass" on half of them with glazed-over eyes. Was that quality assurance, or just a ritual?

It was the same with my apartment in Incheon. Even living in a studio, unused stuff took up half the floor. Then one day, after hauling out six garbage bags, I could suddenly see exactly where everything was. Testing works on the same principle. You have to cut things down before you can see clearly.

A quick word on what smoke testing is

A smoke test is the bare minimum check you run when a build comes out to ask, "Does this thing basically work?" It's not a comprehensive inspection. It's more like a fire alarm. It tells you when there's smoke. Analyzing the cause of the fire comes after that.

My goal was a smoke test suite that fits on the front side of a single sheet of A4. Physically one page. Something you can print out and stick next to your monitor.

How I trimmed it down

First, I laid out all 200 cases in a spreadsheet and wrote next to each one: "Has this case actually caught a bug in the last six months?" The results were grim. Over 70% had never caught a single bug. Cases whose only reason for existing was "they were already there."

Next, I grouped the remaining ones by functional area. Login, payments, core data retrieval, notifications. I filtered each area with a single question: "If this breaks, can the user not use the app at all?" If the answer was "It's inconvenient, but they can still use it," I removed it from the smoke suite. Inconvenience is the job of regression testing.

What I ended up with was 14 cases. One sheet of A4, 11-point font, with generous margins. Each case was written in a single line. Something like: "Logging in with a valid account on the login screen brings up the main screen." The key is fitting the expected result within that one sentence. If you create a separate "Expected Result" column, it just starts growing again.

Rules for keeping it to one page

Trimming is a one-time effort, but the real challenge is maintenance. Let your guard down and it balloons again. A PM says "Add this to the smoke suite too," a new feature comes along and you tack on one more line, and before you know it you're back to three pages.

The rule I set is simple. Add one, remove one. If a new case is important enough to add, then a relatively less critical existing case gets demoted to the regression suite. I fixed the hard cap at 15 cases. Physical constraints change behaviour. It's like how having a fixed closet size keeps you from buying clothes on impulse.

Once a quarter, I print that single page and actually look at it. On a screen, you get the scrolling illusion (it feels like there must be more if you scroll down) but paper doesn't have that. The sense that what's written here is everything matters.

What actually changed

After making this switch, the biggest change wasn't execution time. Of course, going from 200 to 14 cases cut the time too, but the real difference was focus. When you run 14 cases, you pay attention to each one. There's no wondering "Why are we even doing this?" You know every single one genuinely matters, so your eyes don't glaze over.

The team's reaction was interesting too. At first they asked, "Is it really okay to have this few?" But once they saw that the pre-release smoke test took only 15 minutes, they started warming up to it. Before, getting assigned to smoke testing used to make people groan.

Cutting things out is scary, I'll admit. It's only natural to feel anxious when you think, "Can I really remove this?" But even after removing 186 out of 200, there were zero critical issues that blew up in production. The bugs those removed cases used to catch were mostly already being caught by automated tests or code reviews. They were redundant.

It was the same when I cleaned out my Incheon apartment. Out of everything I threw away, the only thing I ended up buying again was a single clip that cost 3,000 won. For everything else, nothing happened at all.

If you want to slim down your smoke test suite, open a spreadsheet right now and add just one column: "Number of bugs caught in the last six months." Once you see the cases where that number is 0, you'll feel a lot better about letting them go.