November 30th, 2008
I’m encouraged by a couple of recent conversations to go public with this possible wacky idea. It has to do with code and testing.
I’ll start with the idea, and then say something about why I’m thinking along these lines.
The idea is for a programming system designed in such a way that the code and its tests are physically together, in one file. Furthermore, that file is not executable. You have to run it through a dedicated filter utility to generate the actual code file(s) from it.
So it’s a bit like, and indeed inspired by, Knuth’s Literate Programming, where the code and its documentation are fused together in a single file which contains both but is, itself, neither. You can’t execute that file; you have to generate the real code files from it.
Adapting the master-file idea to testing, as I envision it, would also entail the following constraint: that the system would refuse to generate the code files unless the code involved already had tests, and those tests passed. In other words, the whole system would militate against using untested code in production, by physically obstructing the creation of executable code files for untested stretches of code.
It seems to me that this would make for a much more sensible and efficient flow of energy than what we’ve got now. What we’ve got now are separate files, and therefore the possibility of running untested code. As long as that possibility exists, people will run untested code. Reordering things so that the creation of the executable code comes after the successful test run would, potentially, realign the energy of the whole process in a very productive way.
As things stand now, the energy is flowing in a wrong and wasteful way. The evidence for this is sociological, at least as much as it is technical. Thorough testing involves keeping the code and the tests in contact with each other through willpower and force, like holding like ends of two magnets together. Therefore, people who test consistently end up with bragging rights, which they often exercise. I hasten to add that I’m not talking about the really accomplished, masterful engineers of the great testing frameworks we’ve got available to us. Those people are above bragging. But there’s a sub-population that isn’t.
I’m really tired of seeing the test police needling people about not having written tests. It’s not that people shouldn’t write tests. Like I said, it’s about the energy flowing the wrong way. The whole culture of test machismo is, start to finish, a waste of energy and, above all, doesn’t work. You can’t get the whole world to write tests by trying to shame people into it, one person at a time. As long as the technical conditions allow for untested code, untested code there will be.
So we’ve got untested code, alongside a culture of testier-than-thou assertiveness. Neither is good.
And then there’s the programming should be fun thing. Programming should be fun. Testing should be a big part of programming. Therefore, testing should be fun. However, it’s acquired a sort of “do it because it’s good for you” aura, like using a treadmill or eating your vegetables. Again, this take on testing is wasteful and irrelevant—but it arises directly from the physical possibility of running untested code, and will not go away as long as that possibility exists.
I’ve made some very sketchy, preliminary attempts to see what a Probative Programming file might look like, for a Ruby program. It’s a daunting task, and one I may or may not ever succeed at. But I’m convinced that something along these lines is both possible and desirable.
Finally, if there are existing systems that do what I’m describing, or anything substantially similar to it, I’d be interested in hearing about them.