Mutation testing with Mutant
When working on DataMapper and its libraries we put a lot of effort into testing. Our libraries must have 100% code coverage and even that is not enough. What we want to achieve eventually is full mutation coverage. What is that? If you’ve ever heard or used Heckle then you’re probably familiar with the concept and you can skip the first part of this post and read about mutant. Code Coverage vs Mutation Coverage If your library has 100% code coverage and you think you did a great job then I have some bad news for you. You did a decent job but there’s a big risk you missed a lot while writing your tests and there are bugs that sooner or later users of your library will discover. Sometimes it’s not a big tragedy, you’ll get a bug report, you’ll fix the bug and everybody’s happy. On the other hand there’s a risk your code base is big enough that bugs that are found too late might be really difficult to fix. This can even require a bigger refactoring just to fix something. That’s one of the reasons why mutation testing can you help you in catching bugs early enough and making sure your code is going in a good direction. ...