In the last year, I’ve heard the phrase “Smoke Testing,” used by many teams that I’ve worked with. It is used to cover a nebulous collection of activities, which dip into operations, quality, development, and test.
This must stop!
I asked the gentleman in the photo above, if what he was doing was a smoke test, and he said yes. He did not, go to the fuse box, and turn the electric off. He did not nip outside for a cigarette. He did not ask his mate, what he thought of the smoke alarm.
He used the appropriate tool, and smoke tested the alarm.
I’ll keep it brief, but it is so frustrating to hear the phrase, “Smoke Test” with regard to development. What on earth are we doing as professionals, making things that aren’t comprehensively tested in the most appropriate way? From other posts, you’ll see that I am not a fan of testing pedants. There is testing that tells you something useful about your product, and there is testing that is a complete waste of time.
The reason I round on smoke testing in development, is that it is an attempt to catch unknown defects, at a stage that is too late in the process. The cost has already become too great, because the effort in triaging any defect found in smoke testing will be too much. Even if the fix is simple, the administration overhead, and development frustration that will be felt, will be bad news. Consider the context shift of a developer who has moved to a new task, only to find that there is some issue found during smoke testing.
There are enough other testing techniques, and operational practices, to assure that the operational behaviour of software is consistent with expectations. Testing for leaks, late in the day, is not going to improve cost of quality to the same extent that testing at system boundaries during development will.
I realise that for some teams, it seems like smoke testing is sensible. For teams that can back up a step though, please try and avoid getting into this situation. Test units, test behaviour, test integration as required (but try to mitigate expense), but don’t try to cover a lack of quality, with hastily pulled together tests, that don’t add value.