[Fact] vs [Test]

I got into a great discussion with Jeremy this morning on xUnit's use of the attribute [Fact] instead of a more common term like [Test]. I cringed the moment I saw [Fact]; it just felt wrong. And the more I read about the author's justifications for it, the more I cringe.

In most unit test frameworks, the word Test is used to describe or annotate the methods you write to prove your behaviour. For example, MSTest uses the [TestMethod] attribute, while NUnit and MbUnit simply use [Test]. xUnit has decided to go against convention and use [Fact]. Their reasoning seems to stem from an earlier decision they had made, which was to use [Theory] to annotate what are essentially data-driven tests, or more appropriately tests that are run against a set of inputs (think 'for-each thing in this set, do this test').

Here is what James Newkirk had to say in explaining [Fact]:

...someone in the audience said that if you were going to call "for-all" tests "Theory" then you should call a single test "Fact". Since then I have had a number of conversations with people and Fact seems to fit.

That explanation didn't exactly blow me away, so here is another one, this time from James' partner Brad Wilson:

The primary motivation is that it changes the expectations that what you do with this framework is capital-T Testing (aka, quality assurance). In reality, Test Driven Development is not really about testing at all. It's an example-driven design methodology which uses code to express the intentions of the class that is under design. The fact that it increases quality is a secondary benefit, and should not be considered a replacement for the work done by capital-T Testers. In Brian Marick's four quadrants of exploration through example, TDD-style code really only represents one of those quadrants (the technology-facing programmer support quadrant).

Additionally, as a word, [Fact] has very good symmetry with [Theory]. The two kinds of tests are fundamentally different; a [Fact] is an invariant statement which is always true, and a [Theory] is a statement which is true for all the given input values.

Now here is where my feathers start to ruffle. Brad writes about the xUnit framwork that "it's an example-driven design methodology which uses code to express the intentions of the class that is under design." Say what? Isn't that a fancy description of a unit test framework? Reading that is like going into a restaurant and having the waiter describe the ravioli as "a splendid blend of finely aged cheeses, high-quality bovine, and succulent savoy spinach, hand-sealed between two layers of circular pasta dough before being boiled in freshly salted mineral waters".

The argument that TDD is not really about testing at all does not fly with me. Of course it is about testing. Are there ways of doing TDD that provide more benefits than others? Sure, but they ALL provide the benefit of added quality (unless you are writing completely useless tests I suppose).

The entire usage of "capital-T Testing" irks me. Capital-T Testing is not the only kind of testing in a project, nor should it be. Look at it this way: we have a development environment, a QA environment, and a production environment. They are all still environments. Similarly we have developer testing and quality assurance testing. They are both forms of testing, and I've never heard anyone argue that developer (or unit testing, which is just a more structured and accurate approach than old-school test harnesses) should take the place of quality assurance testing.

Next, Brad almost spells out the exact reason why [Fact] is a poor choice: "a [Fact] is an invariant statement which is always true". Ok, except when that "example-driven code expression" fails during a build, in which case your [Fact] is no longer a fact, it is something else.

Finally, lets just see how this would work in practice:

"Hey Joe, your facts are failing in the current build".

Here is the bottom-line: all of this smells like people changing things just to be different. I can be convinced to get behind [Theory] for the 'for-all' tests, but until I hear some better rationale, the decision to switch to [Fact] seems arbitrary, and, if I may be so bold, wrong.