IT is hard. Software projects often fail big. Companies have shut down because of bad code. For these reasons, it’s usually best to avoid developing your own software and instead buy something off-the-shelf. Eventually, however, you will need to have something developed. It might be something big that will give you a competitive advantage; or it may be something small, like bridging two software systems together.
Developing software is risky, and it’s going to be more risky if you don’t already have an IT provider you trust. What can you do? One thing you can do is use the five times more rule. Besides that, here is some questions you can ask your potential IT providers:
- In the last year, how many lines of main code have you written, and how many lines of automated testing code like unit tests?
(With “main code” I mean all code except for testing code.)
You want these numbers as an order of magnitude, like a thousand or 10 thousand or 100 thousand. Essentially we are interested in comparing lines of main code to lines of testing code. These should be in the same order of magnitude. If the lines of testing code are significantly less than the lines of main code, this is usually an indicator of unreliable code. - What percentage of your code is reviewed?
This does not apply if it’s a solo developer. If it’s a company that has many developers, they should be reviewing one another’s code. If they don’t review one another’s code, it’s an indicator of suboptimal code development processes.
These are the two that come to my mind right now (but I’ll add more as I think about them). Asking these questions is definitely not going to solve the problem. Automated testing and code reviews are not a panacea. In fact, I’ve written lots of code that is unreviewed and has no automated testing and works great and is maintainable. I’m confident that good untested and unreviewed code can be better than mediocre tested and reviewed code. However the answers to these questions may give you some insight into the company culture. Of course, they are experts and you aren’t, and you’ll have a hard time guessing whether they are bullshitting you or bullshitting themselves, but these questions are better than nothing.