The way we build and deploy software is deplorable. The success rate of large projects is well under 50%. Even when successful, the capital cost is hideous. In his famous “Mythical Man Month,” Frederick Brooks observed that complexity comes in two flavors: essential (the complexity that comes from the nature of the problem) and accidental (the complexity that we add in the act of attempting to solve the problem). He seemed to be suggesting that the essential portion was unavoidable (true) and the larger of the two. That may have been true in the 60’s but I would suggest that most of what we deal with now is complexity in the solution.
Let’s take a mundane payroll system as a case in point. The basic functionality of a payroll system has barely changed in 50 years. There have always been different categories of employees (exempt and non), different types of time (regular, overtime, hazardous, etc.), different types of deductions (before and after tax), different types and jurisdictions of taxes (federal, state, local), various employer contributions (pension, 401K, etc.), and all kinds of weird formulas for calculating vacation accrual. There have always been dozens of required government reports, and the need to print checks or make electronic deposits. But 50 years ago, with tools that today look as crude a flint axe, we were able in a few dozen man months to build payroll systems that could pay thousands of employees. Nowadays our main options are either to pay a service (hundreds of thousands per year in transaction costs for thousands of employees) or implement a package (typically many millions of dollars and dozens to hundreds of person years to get it implemented).
I’m not a luddite. I’m not pining for the punched cards. But I really do want an answer to the question: what went wrong? Why have we made so little progress over the last 50 years?
One Comment
When we were put in front of computers and certain technological constraints, our ability to model well went the same way those punch-cards went. The Semantic Web community, especially those of us who use these technologies as part of the app stack (and not just as a tool to create one), have the means to bring well-done modeling back from the clutches of technology restraints.
Btw, got here through a reference to your Gist ontology (which looks well-suited to my needs) on the W3C sem-web mailing-list. Thanks, Michael! It seems that you guys are quite aligned with the stuff I do, albeit I’m a Topic Mapper at the core, still using semantic models as part of my app stack. It has always puzzled me why this isn’t more wide-spread, but I do blame a notoriously crummy software development world and shady practices of modeling through it and education. (I’m currently writing a book [self-published, so no great acumen here
] about how I see modeling *specifically* (and the poor state of it) as perhaps the fundamental reason for why software so often is bad and failing.