Some interesting discussions here. Since I raised this question, I thought I’d share a little bit about one of the things that I’m working on.
It’s actually something that came out of my discussions about creation and evolution, and the research that I did for my analysis of Answers in Genesis’s “ten best evidences for a young earth.” One question that I’ve been considering right from the outset has been what standards scientific evidence has to meet. Accurate and honest weights and measures are one example; peer review and reproducibility are another. But there are also all sorts of interesting disciplines and habits that scientists develop that are well worth discussing.
One such example is keeping laboratory notes. This is something that leapt out at me when I was reading Kevin Henke’s review on Talk Origins of the RATE project’s claims about helium diffusion in zircons. He said this:
If Dr. Humphreys and R. V. Gentry did not have R. V. Gentry’s original calculations or laboratory notes, how do they know after more than 20 years that typographic errors had been made in Gentry et al. (1982a)? Was R. V. Gentry simply admitting to the possibility of “typographic errors” to help his friend, Dr. Humphreys, and the RATE project? Also, why were the Q values affected by the “typographic errors”, but not the associated Q/Q0 values? How is this mathematically possible? Correcting errors in previous manuscripts is certainly honorable. However, authors should not agree to any “corrections” unless they can first review their original laboratory notes and confirm that copying, analytical or other errors were indeed made. In other words, scientists should not admit to making mistakes before seeing the evidence.
…
Finally, the lack of documentation from Dr. Humphreys to justify changes in the published data of Gentry et al. (1982a) would never be tolerated in authentic scientific journals. Any editor or peer-reviewer of a legitimate scientific journal would demand a thorough and complete explanation of why these changes are justified before any revisions would be allowed to appear in their journals. Competent editors and reviewers would also insist that if the original laboratory notes had been lost that the results be discarded and the analyses redone.
(Emphasis mine.)
When I read these two paragraphs, my first thought was, why am I not doing that in my own job? Keeping lab notes is a discipline that is second nature to every professional scientist, but that is rare in the software development world. Many programmers view the Agile Manifesto almost as a religion, and the Agile Manifesto tells us that we should favour “working software over comprehensive documentation.” But there have been numerous times throughout my career when I’ve looked at some code and wondered why it was written the way it was, only to find that the person who wrote it has long since left the company, and the only thing to explain it is a comment in source control saying something meaningless such as “A bit more refactoring.” Sometimes, the person making such an idle comment has been none other than myself.
Ever since then I’ve been keeping pretty comprehensive lab notes about the code that I write. When I first started, I wrote a couple of blog posts about the practice:
When I first started up this discipline, I used Microsoft Word for it, but for the past two years or so I’ve been using Confluence. More recently, I’ve been asking the question: what would an electronic lab notebook look like if it were designed by software engineers, for software engineers? About three weeks ago, I started up a side project in my spare time to explore this idea and to see what I could come up with. It’s still very much “me-ware” and only in the very early stages, so I don’t have much to show for it yet, but I do have a few interesting ideas knocking around in the back of my mind, and it will be interesting to see where it ends up.