Three Simple Truths of Failure

The simplicity/complexity tradeoff is one that many ELN teams struggle with and is the underlying reason for most ELN project failures.

In an interesting post from Jack Vinson (a fellow CENSA alumni) lays out to three simple truths of failure from a post on the IT Failures Blog which in turn was inspired by Dilbert:

  • Complicated plans don’t work.
  • “Spraying energy into the vortex of failure” doesn’t work.
  • Your boss really doesn’t care.

The first is the most important in my opinion; and it is directly relevant to ELN projects. In my experience the more time a product takes to implement, and the greater the change in working practices required, the less chance there is of the project achieving the hoped-for Return on Investment. Which in my mind is a failure, even if you do manage to roll the application out and torture the users with it (aren’t we here to make life better for the end users?).

I’d broaden the last point to no one really cares about the ELN project – they care about its potential to impact their lives in a positive or negative way. If they really cared they’d be on the Project Team! This is a broader varient of my Toaster Analogy which when it comes up in Workshops often gets that kind of embarrassed “Yeah, that’s very true but it makes us comfortable so let’s move on” laugh.

There’s enough material around to show that complexity causes massive problems. There’s enough public literature around to show that IT projects often fail to meet their goals (and you only have to wonder what really goes on in private because who likes washing dirty laundry in public?).

Which is why I am mystified to see complexity still worshipped, with project teams brandishing their laundry list of features (all mandatory) and vendors explaining with great pride their multi-week implementation process. When questioned, people seem to honestly believe it has to be this hard!

Why can’t we find the quickest, simplest way to achieve the outcome we seek? Why are project managers so rarely rewarded for paring down the list of requirements to the essentials, shortening the project timescales, reducing risk, and slashing the budget? Why is it that products that “Just work” are somehow “Not powerful enough”?

I suspect the underlying reason (once you get beyond organisational politics and vendor sports) it is because geeks see value in intrinsic complexity. Which is fair enough – until that complexity meets the real world of people, budgets, and other projects.

Amphora are quite happy worshipping at the temple of “Less is more” because we believe that’s the best way to serve our customers. Seems that this view isn’t shared by everyone but that’s fine – the trend is in our favour. In this economy we all need to renew our focus on adding the most value for the least cost.

Oh, here’s Dilbert:

ELNs and the post-PC era

Almost all “Electronic Laboratory Notebook” vendors assume you are deploying onto reasonably-recent Windows PCs, which might be the case if you are focusing on Big Pharma (which most vendors were) but isn’t true when you start working with Academic Labs and Biotechs.

As a general rule Apple MacOS X, Linux are second class citizens in the ELN world and it is all the salesperson can do to stifle a laugh when you mention those “other platforms”. The iPad and Android equivalents don’t even get a look in!

I’ve felt this situation is increasingly unsustainable – not only is Apple’s Macintosh experiencing a resurgence, but we’re quite possibly on the cusp of a tablet-drive revolution.

An interesting blog post from the CTO at the UK’s Department for Work and Pensions wonders if their current Windows desktop refresh might not be their last.

Personally, I think it likely this is the last version of Windows anyone ever widely deploys, though.

The reason? I think they’ll be fewer workloads that actually require a heavy deskop stack. Today, of course, we have all this legacy that’s coupled to the desktop, but in a decade, I really doubt that will be the case. Most stuff will arrive via the browser.

Talking with our larger Enterprise customers, it appears their Windows Desktop infrastructure is increasingly cumbersome and it is very hard to innovate in such a complex environment. In the smaller Biotechs there’s a real push to avoid cumbersome IT generally and there’s ready adoption of web and Cloud technologies, as well as additional platforms such as Macs and iPads.

The article makes a good long term point which ELN project teams should urgently consider:

From a strategic point of view, if you’re designing the future technology estate of a large organisation, that last thing it makes sense to do in this kind of context is build stuff that depends on a desktop stack. Furthermore, decoupling legacy from the desktop stack also has to be on the agenda, because you just can’t count on that stack being relevent in 10 years time.

Most ELN products on the market are tightly linked into the Windows ecosystem, even to the extent that one vendor just trumpeted the re-launch of their ELN which is now completely based on SharePoint!

My feeling is that organisations looking for an ELN which is going to last for more than 2 years should consider a situation where there are more than just Windows Desktop PCs in their IT infrastructure – not an unreasonable consideration, but one that needs thinking about up front rather than purchasing a product that locks you in to a dying ecosystem. The Windows PC isn’t going to be replaced but it won’t be the only way you’ll want to access your ELN, and whatever you select needs to be able to work with whatever you might adopt. That such lightweight “thin” solutions are easier to deploy than a thick client just icing on the cake.

(update: this story has been picked up in The Register)

Usability starts with the project team

I’m on a bit of a usability kick at the moment, and I am increasingly convinced that:

  • Usability is the key to ELN project success.
  • Usability almost always completely ignored, sometimes because project teams don’t realise it is important.
  • However it is more the case that the project management process incentivises teams to follow a process and build a system rather than focus on the users.

I was intrigued by this post on User Experience in Startups. If you replace “Startup” for project in the first couple of paragraphs this surely rings a bell.

Startups Project teams start up with a single idea: a solution they believe the world their company has never seen. And while no one can argue that they’re the source of much of the industry’s innovation, they’re almost all lacking in purpose.

Most people believe that User Experience is just about finding the best solution for your users — but it’s not. UX is about defining the problem that needs to be solved (the why), defining the types of people who need it to be solved (the who), and defining the way in which it should be solved to be relevant to those people (the how). Yet as a rule, startups projects are being built on the what.

She then goes on to explore the reasons why, and again if you replace the words…

The challenge lies in the lifeblood of startups projects: the venture capitalists budget holder. Most VCs budget holders put their money in whats — not whys or whos or hows.


Electronic Lab Notebook Requirements – possible pitfalls

I wrote this article by mistake (yeah, I know) but thought it was worth putting up here anyway…

Requirements Gathering – a Broken Process

Project teams have been drawing up lists of requirements since the dawn of time, and since that first list the fate of a project has to a great extent been sealed the moment the requirements have been finalized. As a Civil Engineer my Father was always bemoaning the unrealistic requirements forced upon him by “dreamer” architects, a feeling that I suspect has dominated construction since the pyramids. I recall many a tale of tense ad-hoc negotiation on the construction site, and even the removal of troublesome architects from site entirely!

IT projects share a great many similarities with Construction, although a Civil Engineer has the advantage that she can point out the very obvious real-world difficulties (“You want me to build a roof of glass that large without any supports to spoil the view?”) whereas IT implementors often suffer from the perception that everything is easy. The progress in our field has been so rapid that our customers are used to apparent miracles, and of course there’s always the potential for distrust to arise due to the often large culture gap between the “Geeks” and “Normal people” – and that’s nothing compared to what people often think of Vendors and their dastardly sales people!

It is well known that IT projects can fail to meet expectations, and indeed some of the statistics on IT project success make sobering reading. Plenty of studies have shown the criticality of requirements setting in project success and indeed some practitioners (for example in the “Agile” and “Customer Development” movement) have gone so far as to remove what they perceive to be a very error-prone requirements setting step from their project entirely.

Unfortunately most large companies have purchasing processes which require the business to decide what they need, and then go out to the market to get the “best” solution from what they hope is a selection of competing solutions via a Request for Proposal (RFP) or other formal process. The need to maintain fairness to all concerned means the RFP process if often very rigid and leaves little scope for modification of the requirements in the light of the reality of implementation.

Therefore we are faced with a situation where a potential user’s desire to get the best possible solution forces them into a situation where they have to place almost total reliance on their up-front requirements setting process, an “Aim, and hope” approach which would be abhorred in another sphere of corporate activity. Who would design a process which explicitly prevented feedback from influencing the initial conditions?

And yet, we are here – to paraphrase Winston Churchill, the RFP process and it’s reliance on Requirements Specification maybe the worst way of purchasing systems but in today’s corporate environment is better than anything else available. In the spirit of making the best of the circumstances, this article will take the opportunity to make some suggestions and point out common pitfalls which so many implementation teams fall into.

The problems of requirements and the resulting issues in the RFP process are all the more critical for ELN projects because:

  • The ELN replaces an existing paper process, which through the mists of time is often badly understood in itself.
  • The term “ELN” can cover such a wide range of functionality and domains that in itself it is a foundation for confusion.
  • Most of the current ELN offerings are still only “first generation” solutions which come with their own set of problems.

Thus an issue which bedevils all IT projects is often the founding cause of ELN project failure and requires particular attention.

Apparent “Solutions” which often aren’t

The difficultly in requirements capture/generation are apparent to anyone who has participated in such a project. A common approach is to hire a consultant, and when you get the right one they can single-handedly turn the situation around, although it does require the customer to listen! With apologies to the great consultants out there, the presence of a “consultant” leads to groans in the vendor community because (fairly or not) consultants:

  • Are sometimes viewed as having an interest in creating long running complex projects, rather than quick productive wins.
  • Despite having lots of industry experience, seldom get a long term view of a project. Often their involvement is restricted to the purchasing process, after which the vendor takes over. This gives them a rather one-sided view of the process.
  • Need to get paid and that means making sure the customer is happy. Unfortunately sometimes the customer needs to be told some uncomfortable truths which might lead to them being “fired” as a customer – something a vendor with a large customer base can do, but very hard for a solo consultant to do.

Another approach is to “Stick with who/what you have and know” – for example, if you have an established implementation of SAP or a Document Management package, those solutions might be bent or tailored to meet the new requirements. Unfortunately this doesn’t remove the requirements generation pitfall, and leaves you with an expertise gap as domain-specific solutions ideally come with a vendor who spends their days working on a particular area, who can bring that expertise to bear both in the solution itself and the implementation process.

Some companies will already have an “ELN” deployed in one area, and there is a temptation to view this as being suitable for all kinds of science. This is a sad outcome of the rather generic “Electronic Lab Notebook” term, and is one of the primary reasons why we prefer to avoid the term in day-to-day use; “science” is by definition a very varied activity and you can’t assume that just because two different departments use the same Paper Lab Notebook, that a single ELN will work well in both places. Often all these groups will have in common is they both call themselves “Scientists” and work for the same company – hardly a basis for a common toolset.

It is important to note that these solutions – consultants, re-use of a horizontal tool, and a common ELN across multiple disciplines – aren’t in themselves inherently flawed and can indeed lead to a successful project. There is however a risk of viewing them as the solution to what is at heart a very tricky problem, and project teams who think they’ve somehow reassured themselves of success are often painfully brought back to reality. As the Financial Crisis has taught us, risk doesn’t go away by magic and sometimes the very approaches we take to remove it in fact just increases it, more dangerously so because we’ve stopped being sensitive to it.

Perhaps as Andy Grove says, “only the paranoid survive” and the ultimate key to project success is the recognition that any solution to risk reduction has the potential for problems in itself, often in ways you least expect it.

A Modest, Sadly Unrealistic, Proposal

This article has presented a bleak assessment of how most teams are forced by circumstances to approach their ELN projects, as well as pointing out some common pitfalls that requirements gathering processes fall into. Whilst the problems implicit in these approaches can’t be removed, I hope I have provided at least the opportunity for some reflection. In closing perhaps I might offer some suggestions which I know are unrealistic, but might one day mitigate the issues I’ve described.

One of the problems with an RFP process is the lack of feedback from the implementors; I am sure I am not alone in looking at some requirements and thinking “This project is doomed”. I for one would welcome the opportunity to answer some additional questions, such as:

  • “If you could remove 5 of our requirements what would they be and why?”
  • “What are the most expensive/troublesome requirements listed above?”
  • “Which of these requirements do you think we don’t really need, based on your experience of similar projects?”
  • “What are we missing?”
  • “If you were us, what are the three things you would be most worried about going forward?”
  • “Please rate our chance of success if we go with you, and if we go with another vendor, with reasons”

These questions would afford thoughtful vendors the opportunity to reflect and contribute their experience – after all, for all the conflict of interest that you might perceive in a vendor/customer relationship, a vendor only ultimately succeeds when their customer succeeds. Any vendor team is easily going to see ten times the number of ELN projects that any customer or indeed consultant will see in a year.

Sadly whilst all these questions are interesting I don’t know what project teams would do with this information! In so many cultures project managers are rewarded for following a process and thus any failure is blameless, any reconsideration a failure.

Other Approaches

One very interesting approach we’ve just experienced was where the prospective customer held an RFI (non-binding Request For Information) process which was rather like an RFP but held outside a commercial purchasing process. Crucially the RFI submission and scoring was then followed up with a 1 hour feedback meeting between the customer and the vendor team which allowed for a lot of constructive discussion which no doubt benefited both sides.

We have had good results from projects which have used a Six Sigma methodology with plenty of contributions from all parties – end users, management, IT departments, and outside vendors. This approach tends to be too “heavy” for smaller companies but has delivered great results in larger companies where Six Sigma is part of the culture.

This illustrates the final and most important point: a successful solution is the product of a partnership between everyone involved, and even the largest most process-driven companies reinforce this in their process. Perhaps the greatest danger lies in taking a very formal approach in the purchasing process without counterbalancing that with an up-front listening process – a trap that growing companies often fall into as they formalize their purchasing process without having developed the experience and resources to learn from themselves and the rest of the industry.

Presentation: Survey of the ELN Landscape

Here’s my presentation on “Surveying the ELN Landscape” from the SMI ELN Conference in London today. Bullet points:

  • Business drivers
  • Comparing the different sectors and disciplines
  • Build or buy?
  • An overview of the solution space
  • Patterns of success

There’s a few concepts in here which deserve their own posts (presentations are so useful for stimulating the creative juices!) which hopefully I can do over the coming weeks.