Sunday, March 13, 2016

Review Board RBTools Example

I was experimenting with Review Board in an effort to extend RBTools. I was surprised by the lack of examples using the RBTools API.

I can't say that I'm pleased with my implementation but it works.

Sample output:

https://reviews.reviewboard.org/r/7763/
Diff: 14884
rbtools/commands/post.py
rbtools/commands/publish.py
Diff: 14887
rbtools/commands/post.py
rbtools/commands/publish.py

Produced by this code:

On GitHub: bminard/experimental/reviewboard/review_requests

Monday, March 7, 2016

Successful Tests Find Bugs

The software team I work with has been challenging my notion of testing. Fundamentally, I approach testing from the perspective of the constantly dissatisfied. I am dissatisfied whenever a test doesn't produce a failure. Comments have been made that I should be satisfied with a successful test result. I beg to differ.

In my view, the purpose of testing is to find bugs. Thus a successful test is one that identifies a product deficiency relative to its stated requirements. If a test produces a successful result, great. It may be a valuable component of our regression tests but it hardly means we've been successful.

If we run a series of tests and have a series of successful results we are no longer getting value from that testing. We should re-evaluate the requirement and make a decision on whether we have exceeded the requirement or need to alter our testing.

The contrary position is also important: tests which exceed requirements are not valuable either. Except perhaps to delineate a boundary on the product's operation.