Time to stir up some controversy. It’s been a while since I’ve posted, so why not come back with a bang. Disclaimer, though: these are only the opinions of me, Adam McCrea, and not of EdgeCase.
TATFT
There’s a meme going around for a while that’s gained a lot of popularity, and I for one would like to say that I do not, nor do I have any desire to Test All The Time. Being religious about testing is a recipe for brittle tests, wasted effort, and a false sense of security. I’d rather shift from religion to pragmatism. This means thinking hard about what and when we test.
In a few recent projects, I’ve begun heavy integration testing. No, I’m not breaking any new ground, but the fact is prior to these recent projects, I had no integration tests. I was a bad, bad little programmer. That aside, I have to say that it’s been incredibly eye-opening.
I like to test as far up the stack as I can, getting as close as possible to the end user. Right now that means using Webrat to mimic actual clicks and form interaction to navigate the app. Sticking to this approach, I can change the internals of the app however I want as long as the generated HTML for links and form fields remains untouched. If you had told me a few months back that I could substantially refactor my code and not change one test, I would have thought you were crazy. But integration tests make that possible.
Less testing
But how does that reduce waste? For starters, I no longer write controller (or view) tests. And neither should you. Seriously, just stop. You’re not only wasting your time, but also every future developer who touches that code and has to maintain your useless tests. This assumes two things: that you’re thoroughly integration testing your app, and that your controllers are as skinny as possible. If you still think you need controller tests, you’ve probably got crap in your controllers that shouldn’t be there. Get it out, and stop writing those tests.
I’ll proudly admit that I’m also testing my models a lot less. Seriously, what’s the point of testing things like attribute readers/writers and associations if I’ve got a full suite of integration tests? If my comments don’t belong to posts, I’ll get yelled at, and it’s not going to take long to pinpoint the problem. So, I’m really only unit testing substantial model and helper methods. Everything else gets hit at the integration level only.
What about coverage?
I still think coverage is important, but only at the integration level. Whether or not a piece of code is unit tested is purely subjective, but every line of code must be tested from the point of view that matters most – the user’s. 100% integration test coverage is no guarantee that you’ll catch every possible bug, of course, but anything less would indicate that you wrote some code without thinking about how it will be used.
So is Cucumber the answer?
Honestly, it just doesn’t matter. Just test your code from as far up the stack as possible using whatever tool you prefer. Cucumber, Rails integration tests, Selenium, well-trained monkeys, whatever. Go for it. I do like Cucumber, though, so it’s certainly worth a try if you’re new to full-stack testing.
Ideally, JavaScript would be included in these tests, I just haven’t found a solution yet that makes it practical. It would require a masochist to get 100% coverage with Selenium, but Culerity is one promising option that has popped up. I haven’t had a chance to check it out, though.
This approach certainly isn’t for everyone, but the current wave of TATFT is in desperate need of a counterpoint. If you like to use unit tests as a way to design your code, you probably think I’m way off base. That’s fine. We don’t all have to play by the same rules. Hell, even I still TDD, just not at the unit level. The point here is to make sure you’re very conscious of what, when, and why you test, and that you’re getting enough value from them to justify the time spent.