Most software projects inevitably suffer, especially in the humanitarian domain, unless the huge gap between developers and end users is recognized and addressed.
One of the best ways to do this (reduce the gap between these two diametrically opposed groups) could be by actually embedding a developer in your core team on your next mission or assignment.
Why Do Projects Fail?
Over a 15 year career as a software developer I can attest to the fact that poor requirements definition is the chief culprit for failing projects. I have developed mobile, desktop and server applications, worked for several companies ranging from large corporate companies to small start up companies, in a number of diverse sectors, and the single most challenging problem with creating great software is a reoccurring one; requirements definition. End users and developers can’t, won’t and don’t communicate effectively and efficiently leading to the poor definition and implementation of requirements.
As you can see from the above diagram, the main reason projects fail, by a large large margin, is poor requirements definition; finding our what users need and translating this into solutions.
What’s the Big Problem?
In my experience this big problem could largely be solved by bridging the gap between the two groups at either end of the process: those who make the software and those who the software is bring made for.
But it should be easy right? We are all humans, end users are human, software developers are human (mostly).
How could there be much of a problem of communicating ideas from one group to another?
Well, it turns out, there are two big issues: developer apathy and the requirements gathering process itself.
Issue #1 : Lack of Empathy
The first issue is that transposing the ideas from a non-technical user to a technical/techhie developer (who is more comfortable talking to code than people) is a big challenge. A challenge that the software industry has still failed to adequately address and affects software projects again and again and again.
The problem is that there is a huge gulf in experience, skill sets, even the language used for the same things, between a non-technical ‘ordinary’ user and the people who sit behind desks and write code. Software developers lack the empathy to adequately understand the domain and the users.
Ordinary end users talk a completely different domain language and can have completely different experiences than desk-ridden developers.
End users, come in many shapes and sizes. They can range from the very technical to the not quite so technical.
On the other hand, Software Developers are generally desk ridden, highly technical and actually take pleasure from being able to talk a different language from mere mortals.
How many times have you talked your IT department and felt like they were taking pleasure in talking down to you using highly technical terms?
I will let you into a secret, we absolutely do that on purpose!
Do not underestimate the apathy (and even contempt) that developers have for end users. We view users as inferior beings. Even when users tell us directly that they have problems using something we have built, our first instinct is to scoff and assume it is user error and they should be smarter to use it correctly.
This is genuinely the kind of thing we think end users are doing when they ring us with a problem.
Issue #2 : Noise & Barriers
So with issue #1 the problem is that the two groups have often completely different perspectives and find it difficult communicating with one another. Issue #2 is that, in most software projects, there are numerous barriers placed between the developers and end users by the actual requirements gathering process.
The two groups don’t actually get even the chance to miscommunicate directly. There are layers upon layers to make ensure that, even if a clear message is stated by end users on their requirements, by the time this reaches the developer, it has been changed into something different (a bit like a game of Chinese whispers that a lot of us played in the playground as kids).
The Requirements Gathering Process
Take the above simplified example of a requirements gathering process. We just want to get information from end user to developer in a clear efficient way.
At the start we have the user telling a requirements gatherer of their issues.
The gatherer generally translates these findings using their own domain knowledge into a series of documents.
These documents are reviewed by a ‘middle manager’ who filters and translates the raw requirements, again using his/her own judgement.
The middle manager produces a set of faceless requirement documents which are passed to the developer to develop.
At each step some ‘understanding’ of what the end user wanted to express is lost or mistranslated by people and paperwork.
Then, when refinements are needed, the process starts all over again.
The problem?, too many steps and too many intermediaries.
Lets get these two groups talking to, and understanding each other, a bit more. Wouldn’t it be far more effective and efficient if these two groups were allowed to communicate directly with one another?
Of course it would, but because of Issue #1: the fact that they don’t even know how to talk to one another, it is also pretty impractical.
So how do we make it practical?
Turning Apathy Into Empathy
I am not saying it is easy to achieve.
Developers don’t necessarily want to come face to face with users.
Users may feel frustrated with the prospect of having to communicate with (sometimes) abrupt and condescending developers, but, from experience, the most productive and effective projects that I have worked on have involved the end user communicating directly with the developers from day 1.
Not only is it more productive for the two groups to communicate directly but it also increases the chances that the solution produced will be usable (i.e usable by the end user), useful (actually solves a stated problem) and used (not thrown in the bin like so many other projects by users. Users will have actively participated in the creation of the solution, and have a stake in the solution created.
So here my tips, based on experience, of how to help bridge the gap between developer and end user.
- Get developers out of the office (Genchi genbutsu). Get developers out of their comfort zone. Consider deploying them on your next mission to get them to start to understand the domain and gain empathy with users. We need developers to attempt to understand users better, having empathy with end users not only increases the understanding of the domain but also gives developers incentive to make better choices during the development stage. Giving a human face to a requirement, as opposed to a faceless requirements document is hugely influential on the decisions that developers make. This type of understanding and empathy should not be understated in affecting the quality of solution your developers produce!
- Use Visuals. Using visuals, rather than language and documents to share ideas cuts out a lot of the potential miscommunication issues and helps to speed up the software development process. Best to use visuals and prototypes, both developers and uses both speak this language!
- Develop a common domain language. Right from the beginning set out a Common language for communication, try to find a common denominator to terms and phases used by both sides.
- Regular Feedback & Pilot Exercises. Fail Fast. Fail Often. Facilitate a consistent, regular, and always open, channel of communicating. Frequency is key here. For example have a weekly show & tell from developers on progress and the direction they are currently taking, allow feedback from end users that will be allowed to have an impact of the future direction of the solution. Build it, scientifically measure the feedback, and learn from the lessons end users are teaching you.
- Customer Champion. It may be impractical to keep going back to all the end users regularly, but at least make sure that you have a representative user who can be called upon regularly to give feedback on choices and solution direction.
- Hire a Software Developer with domain experience. Find a ready-made developer who already understands the domain. If you are lucky enough to find a developer with domain experience, hire them, your project will thank you for it.
- Consider an Innovation Hub. Bring the development team permanently closer to your organisation by housing them beside your team. See Andrej Verity’s recent blog post for some great advice on how to do this in a humanitarian context.
A Challenge to You
So the challenge I put to you is, “will you consider deploying a developer with your team on your next assignment?”
Are you willing to stand up to your own IT department or IT contractors and demand a more closer working relationship?
Are you willing to drag developers kicking and screaming away from their desks and into the field? Believe me, as a developer myself, you will face resistance with this!
That is the challenge to you, it would not be a challenge if it wasn’t challenging, if it wasn’t difficult!