Electronic Lab Notebooks

I wrote this article by mistake (yeah, I know) but thought it was worth putting up here anyway…

Requirements Gathering – a Broken Process

Project teams have been drawing up lists of requirements since the dawn of time, and since that first list the fate of a project has to a great extent been sealed the moment the requirements have been finalized. As a Civil Engineer my Father was always bemoaning the unrealistic requirements forced upon him by “dreamer” architects, a feeling that I suspect has dominated construction since the pyramids. I recall many a tale of tense ad-hoc negotiation on the construction site, and even the removal of troublesome architects from site entirely!

IT projects share a great many similarities with Construction, although a Civil Engineer has the advantage that she can point out the very obvious real-world difficulties (“You want me to build a roof of glass that large without any supports to spoil the view?”) whereas IT implementors often suffer from the perception that everything is easy. The progress in our field has been so rapid that our customers are used to apparent miracles, and of course there’s always the potential for distrust to arise due to the often large culture gap between the “Geeks” and “Normal people” – and that’s nothing compared to what people often think of Vendors and their dastardly sales people!

It is well known that IT projects can fail to meet expectations, and indeed some of the statistics on IT project success make sobering reading. Plenty of studies have shown the criticality of requirements setting in project success and indeed some practitioners (for example in the “Agile” and “Customer Development” movement) have gone so far as to remove what they perceive to be a very error-prone requirements setting step from their project entirely.

Unfortunately most large companies have purchasing processes which require the business to decide what they need, and then go out to the market to get the “best” solution from what they hope is a selection of competing solutions via a Request for Proposal (RFP) or other formal process. The need to maintain fairness to all concerned means the RFP process if often very rigid and leaves little scope for modification of the requirements in the light of the reality of implementation.

Therefore we are faced with a situation where a potential user’s desire to get the best possible solution forces them into a situation where they have to place almost total reliance on their up-front requirements setting process, an “Aim, and hope” approach which would be abhorred in another sphere of corporate activity. Who would design a process which explicitly prevented feedback from influencing the initial conditions?

And yet, we are here – to paraphrase Winston Churchill, the RFP process and it’s reliance on Requirements Specification maybe the worst way of purchasing systems but in today’s corporate environment is better than anything else available. In the spirit of making the best of the circumstances, this article will take the opportunity to make some suggestions and point out common pitfalls which so many implementation teams fall into.

The problems of requirements and the resulting issues in the RFP process are all the more critical for ELN projects because:

• The ELN replaces an existing paper process, which through the mists of time is often badly understood in itself.
• The term “ELN” can cover such a wide range of functionality and domains that in itself it is a foundation for confusion.
• Most of the current ELN offerings are still only “first generation” solutions which come with their own set of problems.

Thus an issue which bedevils all IT projects is often the founding cause of ELN project failure and requires particular attention.

Apparent “Solutions” which often aren’t

The difficultly in requirements capture/generation are apparent to anyone who has participated in such a project. A common approach is to hire a consultant, and when you get the right one they can single-handedly turn the situation around, although it does require the customer to listen! With apologies to the great consultants out there, the presence of a “consultant” leads to groans in the vendor community because (fairly or not) consultants:

• Are sometimes viewed as having an interest in creating long running complex projects, rather than quick productive wins.
• Despite having lots of industry experience, seldom get a long term view of a project. Often their involvement is restricted to the purchasing process, after which the vendor takes over. This gives them a rather one-sided view of the process.
• Need to get paid and that means making sure the customer is happy. Unfortunately sometimes the customer needs to be told some uncomfortable truths which might lead to them being “fired” as a customer – something a vendor with a large customer base can do, but very hard for a solo consultant to do.

Another approach is to “Stick with who/what you have and know” – for example, if you have an established implementation of SAP or a Document Management package, those solutions might be bent or tailored to meet the new requirements. Unfortunately this doesn’t remove the requirements generation pitfall, and leaves you with an expertise gap as domain-specific solutions ideally come with a vendor who spends their days working on a particular area, who can bring that expertise to bear both in the solution itself and the implementation process.

Some companies will already have an “ELN” deployed in one area, and there is a temptation to view this as being suitable for all kinds of science. This is a sad outcome of the rather generic “Electronic Lab Notebook” term, and is one of the primary reasons why we prefer to avoid the term in day-to-day use; “science” is by definition a very varied activity and you can’t assume that just because two different departments use the same Paper Lab Notebook, that a single ELN will work well in both places. Often all these groups will have in common is they both call themselves “Scientists” and work for the same company – hardly a basis for a common toolset.

It is important to note that these solutions – consultants, re-use of a horizontal tool, and a common ELN across multiple disciplines – aren’t in themselves inherently flawed and can indeed lead to a successful project. There is however a risk of viewing them as the solution to what is at heart a very tricky problem, and project teams who think they’ve somehow reassured themselves of success are often painfully brought back to reality. As the Financial Crisis has taught us, risk doesn’t go away by magic and sometimes the very approaches we take to remove it in fact just increases it, more dangerously so because we’ve stopped being sensitive to it.

Perhaps as Andy Grove says, “only the paranoid survive” and the ultimate key to project success is the recognition that any solution to risk reduction has the potential for problems in itself, often in ways you least expect it.

A Modest, Sadly Unrealistic, Proposal

This article has presented a bleak assessment of how most teams are forced by circumstances to approach their ELN projects, as well as pointing out some common pitfalls that requirements gathering processes fall into. Whilst the problems implicit in these approaches can’t be removed, I hope I have provided at least the opportunity for some reflection. In closing perhaps I might offer some suggestions which I know are unrealistic, but might one day mitigate the issues I’ve described.

One of the problems with an RFP process is the lack of feedback from the implementors; I am sure I am not alone in looking at some requirements and thinking “This project is doomed”. I for one would welcome the opportunity to answer some additional questions, such as:

• “If you could remove 5 of our requirements what would they be and why?”
• “What are the most expensive/troublesome requirements listed above?”
• “Which of these requirements do you think we don’t really need, based on your experience of similar projects?”
• “What are we missing?”
• “If you were us, what are the three things you would be most worried about going forward?”
• “Please rate our chance of success if we go with you, and if we go with another vendor, with reasons”

These questions would afford thoughtful vendors the opportunity to reflect and contribute their experience – after all, for all the conflict of interest that you might perceive in a vendor/customer relationship, a vendor only ultimately succeeds when their customer succeeds. Any vendor team is easily going to see ten times the number of ELN projects that any customer or indeed consultant will see in a year.

Sadly whilst all these questions are interesting I don’t know what project teams would do with this information! In so many cultures project managers are rewarded for following a process and thus any failure is blameless, any reconsideration a failure.

Other Approaches

One very interesting approach we’ve just experienced was where the prospective customer held an RFI (non-binding Request For Information) process which was rather like an RFP but held outside a commercial purchasing process. Crucially the RFI submission and scoring was then followed up with a 1 hour feedback meeting between the customer and the vendor team which allowed for a lot of constructive discussion which no doubt benefited both sides.

We have had good results from projects which have used a Six Sigma methodology with plenty of contributions from all parties – end users, management, IT departments, and outside vendors. This approach tends to be too “heavy” for smaller companies but has delivered great results in larger companies where Six Sigma is part of the culture.

This illustrates the final and most important point: a successful solution is the product of a partnership between everyone involved, and even the largest most process-driven companies reinforce this in their process. Perhaps the greatest danger lies in taking a very formal approach in the purchasing process without counterbalancing that with an up-front listening process – a trap that growing companies often fall into as they formalize their purchasing process without having developed the experience and resources to learn from themselves and the rest of the industry.