How to Choose a Software Vendor

So you’ve got yourself a business problem, and you’re dreaming up an online system that will help you solve it. But wait – you’re not in the software business. Choosing a software vendor doesn’t necessarily mean that you need to know about the latest technologies or the best agile software methods and practices (after all, this isn’t your area of expertise). How can you possibly be expected to evaluate a vendor without being rained on with a tsunami of obtuse acronyms, or being swept away by an aggresive torrent of enterprise Java integrations, object-oriented Ruby-coloured whizbang widgets, Django tangos, Sharp Cs, and other curve-jumping, paradigm-shifting agile-infused platitudes?

The answer is to examine the value system of your prospective vendor, and what this implies relative to your own. You and your vendor will be forming a close-knit team to solve a problem that they don’t yet understand with a process and set of technologies that you don’t yet (or may never) understand. In this ecosystem, shared values and team cohesion are overriding concerns. No adherence to a particular methodology or technology choice will trump them.

The following questions might be useful for you:

What are your values, and their related principles and practices as they apply to software development?

Ok that was an obvious one. But starting with this will allow you to delve more deeply into areas that you’re interested in. The goal here is to determine how much overlap there is relative to what you expect from them.

What are your quality standards and how do you maintain software quality with a continuously evolving product? How will I know if you’re building the system to professional standards?

In software and web development, quality standards are difficult enough to talk about, let alone implement. This is one of the few engineering disciplines that has no standardized certification. Since you can’t rely on certifications, you have more work to do to qualify your vendors’ quality and project management standards practices. Delve into these standards a bit, and ask them how the standards are shared within their company. If there is no reasonable answer to this question, move on!

Do you track your success rates? If so, what criteria are used? What are your project success rates? Can you share with me any raw data that supports this?”

Tracking success rates implies that the vendor has long-term thinking, and is interested in continuous improvement. It takes a lot of effort to do this, and is a good marker of a serious vendor.

Can you characterize your failures or challenged projects? In other words, when your projects go over time and budget, generally how far over time and budget are you, on average? Why is that, from your perspective?

Brace yourself for some wild improvisations here – it’s a difficult question. Project overruns will happen, no matter if your vendor is on time most of the time – but the question is “how controlled are the overruns?”. Note that this should be asked without regard to “who’s fault” it was – remember that your prospective vendor is likely to have just as many problems with you delivering late requirements and going on vacation as they did with their own difficulties with hiring the right people, training them on your domain, and managing and motivating them to do good work. What’s more interesting is whether or not they dealt with challenges and overruns as they arose, and if they did – how did they do it? Ask them if they’ve ever been fired from a project, and why. If you’re feeling plucky, ask if they’ve ever been sued (in case you’re wondering, the answer here should be a flat “no” not a bemused “weeelllll…”).

Can you connect the dots between the problems your method solves, and the specific concerns that we face?

Tell them what those specific concerns are. After they ask you a few questions, they should have a very clear, stepwise idea of what they would do to solve your particular problem, and how it relates to their own general method. This shows that they’ve been around the block a few times, and that they have that all-important and elusive ability to listen and integrate your concerns.

How did you choose the practices within your method, agile or otherwise?

The answer here will give you a clue into whether or not they are thinking about why they chose whatever practices they are using, and not simply parroting the buzzwords of the day. This means that they should be able to answer in terms that you understand.

What are the main problems we’re trying to solve, in your understanding?

This is communications 101 – after you’ve explained the problem to them, get them to explain it back to you. Repeat until satisfied. Actually – don’t repeat this more than 3 times… find another vendor. Bonus points if they provide a little analysis of your problem and feed it back to you in a way you may not have thought about it before, or if they organize your problem in a way that helps clarify your direction.

How often will you report the status of the project to me, and at what level of detail?

Agile methods typically use short delivery cycles (sprints) to feed back progress regularly. But Agile methods have a lot of implications for you the customer that you might not be aware of (have a look at this little Agile primer if you’re new to the world of Agile). Even if they’re not doing agile, you must make sure that your vendor has some well-known project management standards, employing continuous status gathering and recording, planning, re-prioritization, and risk assessment for a series of well-defined tasks in a plan. They should aggregate and report on these regularly. Try to judge how nimble and transparent your vendor’s process is. Do you feel that this will be sufficient to give you the level of visibility you require?

How can I trust you will deliver the system?

They should be able to show past successes. They should be able to relate your problem to other similar things that they’ve done, on some level. They should be able to talk reasonably about where they went wrong in the past and the resulting course corrections they took. You’re looking for confidence grounded in reality, as opposed to sweeping statements about their legendary greatness.

Can you let me talk to a few references?

I’m constantly amazed at how many times this question is skipped. Your vendor should be able to give at least a few references from past projects that will sing their praises at the top of their lungs. What these people tell you is the best you can hope for from them. Whatever they say should be coming from their socks and should sound like sweet music.

Good answers to these questions should be a pretty good indication that your prospective vendor knows both what they are doing, why they are doing it, and that they have the wherewithal to do it successfully. Good luck, and don’t forget to stay involved throughout the project. We need you as much as you need us!

  1. Strangely relevant to what I’m peripherally involved with.

    Thanks. Some things I already knew, but nice angle on others.



    Rob James
    Aug 30th, 2010
  2. Glad you enjoyed it, Rob. I’d be interested to know which points were new to you, and how it’s relevant to what you’re involved with.

    Jeremy Chan
    Aug 31st, 2010

Add a comment

Comment feed
The better to greet you with
No one will ever see this
Your pride and joy
The reason this comment form exists

The crew behind ASOT

We're a team of interactive, software, and business intelligence experts skilled in the design, construction, and management of online enterprise systems.

Visit The Jonah Group site

Get in touch with us