continuous integration

Practical Approaches To Testing

Testing is a lot like going to the gym. We all know we should do it. But when the alarm clock sounds after a long night of coding, most of us hit the snooze button. 

Here’s the thing: tests should work for you, not against you. Testing may be hard, but it doesn’t have to be a nightmare. 

Mike Cohn, of Mountain Goat Software, created this testing pyramid.

He emphasizes abundant unit tests to ensure code works at a granular level, a healthy amount of integration tests to verify cross-functional behavior and a thin layer of end-to-end tests to test the user interface.

Mike’s testing pyramid matches up well with Google’s approach, which recommends your test suite be made up of 70% unit tests, 20% integration tests and 10% end-to-end tests.

Justin Searls, of Test Double, takes an entirely different approach. He recommends creating two types of tests: high-level end-to-end tests and low-level unit tests. (He also has a great talk on testing, given at RubyConf 2015.)

 

See all that grey space where integration tests would normally go? You can skip those. Searls believes your high-level tests will cover base expectations for user experience and ensure the app works. 

This design is sometimes referred to an hourglass pattern of testing where a test suite is made up of verbose end-to-end tests and a generous amount of unit tests with few, if any, mid-level integration tests. 

So why do some people emphasize end-to-end tests and others minimize them?

Well, there’s a bit of a debate. 

End-To-End Testing

There are a number of advantages to end-to-end tests. Managers love them because it eliminates the change of them receiving hateful twitter messages when the app breaks in production. 

And some developers — not you of course — like them because it reduces the pressure to write thorough unit tests.

But there are major challenges too. 

End-to-end tests are slow. To truly verify customer behavior, you have to spin up a browser for every test. The more code you write, the more expensive it is to run your test suite. 

Some end-to-end tests may mask issues that should have been caught in a unit test. On top of all that, some end-to-end tests are brittle to code changes. 

What’s the right answer? It depends. Small apps tend to benefit from thorough end-to-end testing while larger apps often require more dedication to unit testing. 

There’s no single “right” methodology. Your approach will depend heavily on your team and your unique circumstances. 

We’re all unicorns and snowflakes, aren’t we? 

https://assets.squarespace.com/universal/scripts-compressed/common-5fbd4cec44d42fa41cebb-min.en-US.js