Monday, 21 July 2008

Software Development Coming of Age

There was a time when computer science was the preserve of academics and big business for the simple reason that computers were expensive and their per-hour cost was high. You wouldn't have an A-Level student being let loose on a system for hours trying to hack out a project for school.
Time have changed however and computers are extremely cheap. Even in countries where the average income is low, many people have at least limited access to a PC which can be used for amongst other things programming.
Like most things, this is both a blessing and a curse. It is a blessing because people who might be very skilled programmers have access to something that they wouldn't have 20 years ago and these people are part of the skill set that businesses use to produce productive software (we hope). However, the curse is that the skill set is polluted with thousands of people with very limited skill and although not necessarily bad in itself, a percentage of these people seem very free with their advice to others who are struggling with something and propose solutions that might be, well rubbish. There is no easy way to work out the value of this advice because Programming is often considered like mathematics where if the solution works, it must be correct. A better analogy would be car mechanics where just because an engine fits and turns over doesn't mean it is the best type of engine or the best way to connect it up - although it might well work. Programming is often a set of balances where speed is offset against readability or where pure theory can be the enemy of pragmatism and just getting something that is 'good enough' rather than 'perfect' in a reasonable timescale. The skill of the programmer is not whether they always produce the fastest code but whether what they produce is appropriate to any given requirement.
Well this can be one problem but it can get worse: people write rubbish sometimes because of poor advice or lack of training but also people often re-invent the wheel. How many people must have written a 3-tier database web application which is 80% the same as every other one in the world? Why can't we share what we have done to more quickly move into the future? Well we sort of share but we can face similar problems to above, we get given eaxmple code by somebody when it might be varying levels of rubbish or we might take some existing code and by not understanding it, we might either modify it and make it rubbish when it was OK before we changed it or otherwise we might apply it to a system where it is not appropriate. For example, a non-secure database application might be fine for a corporate network where hacking is seen as unlikely but it would be inappropriate for a public network where hacking is commonplace. This is compounded by the seemingly high number of people on forums who seem to have little or no programming knowledge asking things like, "how do I generate 3D graphics" or, "how can I write a flight simulator" - can we trust these people to write robust software?
So what do we do? I read a book not too long ago called "Emergent Design: The Evolutionary Nature of Professional Software Development" by Scott Bain and he was talking about more regulation for the profession that is called Software Engineering. A person cannot merely decide that he wants to be a doctor or lawyer and start practising. Even if he is poor or seemingly able, he must attend various courses and take exams to prove his competence. Even for mundane things like driving a car, people have to be a certain age and have to pass a driving test. Why? Because these things carry responsibility. Driving or being a doctor without proven skills is dangerous. Being a lawyer without skill can end up causing somebody to be prosecuted without good reason or cause somebody who is guilty to be released into society when they should be locked up. What about Software Development? Well poor software is often blamed by companies for various corporate problems and who is in a position to deny it? We have all experience poorly written software so we almost expect things to be less than perfect. These bugs can cost us time and money as well as frustration. Although the year 2000 'bug' was not really a bug in one sense, it cost companies millions in proactive and reactive costs over the new millenium eve in case their systems crashed. While we have a totally unregulated industry, we are all in danger. So, imagine we had a regulated system where somebody has to have a certain level of qualification before they can call themselves a "developer" or "software engineer" this would help to solve the general quality of systems being developed - or at least improve them over time since currently many people who teach computer science won't necessairly have a qualification themselves or otherwise they learnt their trade a generation ago when priorities were different.
In order to solve the second problem, i.e. people re-inventing the wheel, I think if the industry became regulated then the industry body could support a single 'red book' which would describe all of the best practices in software where they exist with any caveats to the design that might be appropriate. It would not be a copy and paste because we don't want people to copy-and-paste from one context into another - that causes bugs. What we do want is for a single defining place to say, "if you are designing a new database, you must consider 1) security of database access (link to sub page) 2) Layout of tables and links (link to sub page) etc". A sub page might say "you must implement a security model for stored procedures if you a) have a publically visible server, b) have an application that behaves differently for different users..... but if you secure the procedures you will a) incur additional development time, b) you must produce a comprehensive test case to ensure you have secured them (or create a process that means they will definitely be secured as they are created)...etc"
Hopefully you get the idea. It will never be able to be totally definitive for a specific scenario because the context of software always differs but at least if there is a 1-stop shop for information, people will not forget something and will be able to see the pros and cons of every decision before they make it. Of course, good practice might change with time so the system would need a way of updating users so that they know this has happened but what we end up with is a way of sharing knowledge from bona-fide engineers who know what they are talking about but in a way that does not encourage copy-and-paste with all of its pitfalls.
I suspect such a system exists in various companies and probably a lot of the content is the same but rather than trawling google to find something of dubious value, if this content was all in one place with known reliability then we can all move onwards and upwards.
Post a Comment