Summary

You are trying to develop a new application and want to ensure high developer output while installing quality control.  Before installing build servers, setting up testing scripts, etc., make sure that you have proven your market first.  The easiest and cheapest way to create quality in your application is via the developer run smoke test. A lean smoke test is a basic test of the system to ensure that 80% of the application works.  The goal of smoke tests are to ensure a high-quality routine deliverable at very low cost.  This is not a substitute for more complex testing which requires more infrastructure.

Setup Time

5 minutes to set up a template at the start of a project. 5 minutes to update at the end of every sprint with either new tests or to reduce tests. If the tests take more than 10 minutes, then they are no longer smoke tests, they have moved into the realm of regression testing.

Who

  • Software Developers  You do not need nor should you have a tester or PM run your smoke tests.  The dev team needs to be in the repeatable habit of ensuring quality without the laborious burden of detailed regression testing.
  • Scrum Master /PM Ensure that the Smoke tests are accessible via the web to every team.

Actions

For example, if the team is building a ticket sales system, a simple action or a set of simple actions like:

  1. User Looking For Events
  2. User buys tickets
  3. User prints tickets

Ideally, if you have more than one person, you will assign them a different platform, so they are routinely testing it against different operating environments to find problems before they arise.  For web development, this could be IE or Firefox.  For mobile, you could have them alternate between iOS and Android.

Frequency

Like all things, it depends.  Ideally, any time a team conducts development, a smoke test would be completed.  This, however, isn’t always a good idea and introduces waste, especially if you are building your MVP.  A more realistic scenario would be time/effort based.  As a rule of thumb, every 80 hours of development and deployment cycle time should be followed by a smoke test.  A team may elect however to run smoke tests more often if they are trying to debug a set of problems.  Additionally, if you enter production, the product owners may require a zero bug atmosphere.  If this is the case, you need to have the cost vs. benefit conversation for smoke testing.

Time Ratio

80 hours x 60 minutes per hour = 4800 minutes = 10/4800 =  0.2% of time for devs to test

 

Client Discussion Template

Client:  I tried to do XYZ and found a bug.  Your team screwed up!  I AM NOT PAYING YOU FOR BUGS.  And if you can’t tell, I am angry.

You:  We take quality very seriously. The developers, like you, want to do a great job and make you happy.  There is a balance however between having bugs and building software.

Client: You mean you are delivering shoddy work and I just have to deal with it?

You:  No, that isn’t the case.  We are building functionality into your product to make it more attractive to users and provide value to them.  We can spend more money to reduce errors.  Those efforts include building improved test features, conducting software practices like Test Driven Development, build suites of automated UI, backend testing and would most likely require us to bring another person on the team to achieve those results.   That work requires additional money, time, and management.

Client: That is crazy.  I don’t want all of that.  I just want bug free software.  You should be building software that doesn’t have any bugs.

You:  That is fine, we can do that, but it is going to cost you more money.  You control how we spend our time and what is important to you.  Using our experience, we select software libraries and techniques that will reduce bugs as much as possible.

Client:  Hmmm, well, I guess we can live with minor bugs.  What about the major ones?

You:  Agree.  We cannot have major bugs.  We will spend as little as possible to make sure that major bugs don’t happen and will start conducting small tests every sprint called smoke tests.  That should resolve most of the issues and cost very little on your part.

Client:  Ok, as long as it doesn’t cost me more money.

You:  It is a very lean option.  If we build more complexity, then we will need to invest in more testing capabilities.

Client:  If our product is flying off the shelf, then the investment will make more sense.

You:  Now you are thinking lean!

Join the discussion One Comment