Tuesday, 22 January 2013

How to deal with software suppliers

I have no sympathy for suppliers who's software is found to contain SQL injection vulnerabilities. Of course, the people who lose because of the software they buy/use and trust, I do feel sorry for but there is still a whole credibility issue with software and people not being able to tell the difference between good software and bad. I guess it is the same as car repair garages. They all look similar but some are professional and good value, others are poor and expensive but what does this mean for software?
SQL Injection attacks have been known for ages and are so easy to avoid. Setting relevant permissions on the database user (i.e. not using root/sa); using stored procedures only, where possible; using parameterised queries; even basic input sanitisation most of which even by themself will help massively but together make the system air-tight. For any company still selling software with these basic and well known security holes speaks volumes about their ability/credibility and motivation. It is also not acceptable to blame 'legacy' software. If you sell software, it has to be fit-for-purpose and if you have inheritied legacy software you need to be able to extend or modify it to be suitable (or replace it if not). In many companies, it would seem, people are happy to make money on existing software as much as possible rather than investing in replacements/fixes/patches. So no sympathy there.
What are we to do, however, as software buyers if we ourselves are not software savvy? Even if we are savvy, what is to stop the supplier promising lots of things that are either half-truths or even lies? What are we to do when we are talking about potentially 1000s or even millions of pounds of software? Like most trades people we deal with, there are various industry standards including generic ones like ISO27001 (Quality assurance) as well as software specific ones like OWASP (Open Web Application Security Program). If you are spending decent money, you must ask about these. If a supplier has none of these, then ask them why you should trust them - be brutal. Personally, I wouldn't spend more than a few thousand pounds for software from someone with no accreditation. If they have an externally audited system like ISO27001 then you have some recourse if things start to slip but it will be limited and will probably not include any financial backup - the best you could expect is the auditor to pull the accreditation but that won't get your money back. Others like owasp are voluntary and so even if the supplier uses it badly or still allows something that shouldn't theoretically be possible with OWASP then there is still no direct comeback unless these issues are contracted against which can get complicated and expensive (but worth it to some extent for the largest contracts).
Because software in many ways has lots of unknowns, most suppliers will not be happy to provide fixed prices except for simple systems or ones where you can lock down the requirements for 2 years, something which is often unworkable. Others will simply multiply the cost by 5 and use that as their fixed price!
Really, all you have is some kind of relationship that you need to foster. You need to be open to talking about money and that includes things like, "you didn't tell us you needed X and it will cost another 5,000" and be open about asking for cost breakdowns and justifications. For larger systems, it might well pay to employ a dedicated Project Manager who understands software. Although there are skills that transfer from all project management, quite simply, unless the PM understands software, they will not understand whether developing X should really cost Y. When the customer says that a custom protocol for ethernet programmed in C will cost 50,000 - would you know if that was ballpark?
Since security is often where you as a customer can be stung, it is not unreasonable to expect proof that software has been independently penetration tested. Most large bespoke systems should be tested as part of the delivery but if you are buying it off of the shelf, why not visit the supplier web site and find out whether that software has been tested.
Lastly, the other big sting can be ever-increasing costs and for the customer, it can be deadly, for the supplier it might be justified with ever-changing requirements (the curse of software development). There are two things here. Firstly, be open up-front with the likelihood of requirements changing and perhaps what parts might change. In other words, if your system produces a report which is likely to change format - specify it so the software can be designed to easily modify the report later rather than a rebuilding. We assume most software is easy to modify but usually, this is only in certain areas. Secondly, be clear that you want a system that can deliver "the main thing" early on for user testing and feedback. Tell the supplier that you don't want a system that is undeliverable until zero-day when all the money is due because this doesn't allow you to cut your losses if the system is taking too long and costing too much money. Having the bulk of functionality e.g. half-way through the project allows some pressure to be taken off the supplier since this will prove many live issues early on and then allows additional functionality to be added later. Thirdly, be wary of massive customization which is often the part that takes all the time and money. Tweaking web pages is not as easy as most people think (depending on what you are changing) but if an off-the-shelf framework does most of what you want, try and adjust your business processes to suit that rather than spending 1000s trying to modify something into something else - you can imagine this rarely ends up looking pretty and it costs loads but it is so common it is almost unbelievable. Suppliers need to educate customers and customers need to listen to their suppliers. If supplier says X is hard - read expensive and don't dig your heels in for the sake of it.
Personally, I would like to see proper software guilds that individuals or suppliers can join which is like an audit but more closely matched to building trade guilds where non-compliance results in insurance-backed compensation and mediation by independent persons. The result is you get a badge which says, "I follow best practices" and "I get audited regularly". There are so many resistant to such things but I think if it is easy enough to join and not too expensive, then this raises the credibility of companies and gives more peace of mind to customers.
Post a Comment