That means they have to be looking inside your box to get the data they’re going to aggregate, to get to that astronomical valuation. That’s why they didn’t just go with the enterprise-y user agreements that Microsoft and Amazon use. They don’t want your money. They want the advertisers’ money.

What’s inside your Dropbox says a lot about you. And that, of course, is what Dropbox users (like me) are afraid of.

If that’s the case, you’d have to be very brave to use DropBox for Science that wasn’t already in the public domain… best stick with solutions focused on solving the ELN problem, which have the appropriate technical and business architecture! We’d love to talk to you

I bumped into an analysis of the Terms of Service of various Cloud service providers on Neowin. It isn’t encouraging reading.

I can empathise with the providers; they are providing a generic service to a large number of users, for free or a very low price. The only way they can execute their business at that scale is to tell people “We get to see your data too, and we can re-use it or give it to other people for whatever reasons we decide”.

Unfortunately, that’s not pretty from an IP perspective. I’m no lawyer but I can’t see how some of these terms and conditions are compatible with securing a company’s IP via Patents or even Trade Secrets (let alone personal privacy).

Caveat emptor!

Interestingly Amphora have found ourselves increasingly providing Cloud-like and SaaS-centric services to our customers. We started providing PatentSafe as SaaS but then we’ve moved into providing offsite backup (using a private CrashPlan service) and other services.

In meeting this customer need, we’ve had to do it with our normal IP-centric Terms of Service – which basically means your data is private to you, and we’re only going to disclose it when you ask us (or, in the extreme, when we get a court order). That’s been hard – it has caused us to shy away from some “Cloudy” infrastructure that I know some ELN vendors have gone for, e.g. the Amazon EC2 and S3 products to name just two. Ultimately that means our costs are higher, but to do otherwise would be irresponsible.

I’d like to say this is a matter of “You get what you pay for” but it isn’t as simple as that – these commodity services are just focused on a different market. So before you get the Cloud bug in the Lab, read the Terms of Service and consider if that’s appropriate for your circumstances. When you’ve done that, check with your provider – do they run the services themselves, or do they use another platform – if they’ve got it all on Google or Amazon infrastructure (excellent technical choices! legally trickier) it is worth taking the time to understand who your contract is with and what is happening to your data.

This interview of SAPs CIO makes interesting reading.

Oliver Bussmann told me that SAP’s chief scientist had done an analysis of computers in business, and where they will be going. In their scientist’s view, the mobile and desktop models are converging. That is, instead of rolling up to a desk every day to power up a machine, and sift through screens of information to arrive at a simple dashboard, users will come to expect a smaller device to focus on the data. This smaller form factor and more task-focused paradigm will allow you to call up information almost instantly, with laser focus on specific processes, rather than one large machine that does a dozen things. It’s an evolution of the species, if you will.

We’re seeing this ourselves, both in our internal use of the iPad and also by our ELN clients. The Tablet form factor and the very task-centric paradigm really does create a compelling additional device from which to interact with your data – and we’re pleased that PatentSafe continues to keep up with the innovations in the tablet space.

A large number of our customers have iPad trials ongoing; there are few who are refusing to entertain the iPad at all on the basis that it is a “toy”. With endorsements like this from SAP I can’t help but think they will be reconsidering!

The resulting lack of competition on top of vendor consolidation has led to Chemistry-focused ELNs tending to lag in terms of ease of use, and openness, whilst of course being pretty expensive. As those vendors seek to expand into other scientific disciplines, they bring with them the same costs which are then unnecessarily imposed onto other areas.

One major reason for this is that the Open Source Cheminformatics world has historically been under-developed. My theory is that’s because Cheminformatics started in earnest before Open Source took off as a concept (in comparison to Bioinformatics) but I have no real evidence for this.

Open Source is an important part of todays’ software ecosystem:

• It provides a set of building blocks, and I would imagine almost every software product (commercial or otherwise) has some Open Source components. By sharing the basic foundations, the cost of entry is reduced and this results in more entrants and lower costs for everyone.
• Open Source drives innovation by allowing people to re-mix things to “scratch their own itch” and produce new approaches as needed. Even if those solutions remain in-house they still inspire others, and perhaps allow the engineers inside the commercial vendors to successfully propose new approaches.
• The threat of “free” competition as well as more players in the market generally keeps vendors on their toes. Without a complete lock on particular functionality, vendors must instead compete on value and functionality.

Amphora are not in the Chemistry ELN market (and have no intention of being in that market), but I look at what’s out there and compare with what I see happening in other areas and it is clear there’s a lot that could be done which would benefit the wider ELN world as well. Frankly what’s going on Chemistry is giving the wider ELN community a bad name – especially as marketers keep positioning their products as the only “proper” approach for any kind of science, chemistry or otherwise. You really don’t need to spend thousands of dollars a seat and days/weeks of implementation time to deploy an ELN!

So I’ve waiting for a decent Open Source approach to Chemistry-based searching because if nothing else it will inject some innovation where it has been sorely lacking.

So I was delighted to read this post on how to Enable Exact Structure Search and Substructure Search for Your Chemical Database. I don’t think there’s a great breakthrough here, but it is a straightforward set of instructions on how you can do it which demystifies Cheminformatics a lot.

This could get pretty interesting in the next few years…

• HTML5 and other web technologies are surely at the stage where we don’t need a “thick client” deployed onto a desktop anymore – can’t we do it all in the browser?
• What about all the tablets (like the iPad), can we make them full clients?
• Can we finally have true cross platform chemistry ELNs?
• Can we easily embed chemistry into a variety of other applications, rather than having to buy a complete implementation of someone else’s idea of an ELN?

Amphora’s focus will remain on our particular slice of the ELN problem, which is providing the secure recordkeeping back end, discipline-neutral collaboration etc. Once you’ve done all that work the lawyers generally want to make sure you get the credit for all that Intellectual Property you’ve created even if they don’t explicitly apply for a Patent – even in Academic environments this is becoming more important as the journals and funding agencies raise their expectations in terms of record keeping etc. Amphora’s job is to help our customers focus on the science, and we’ll look after the Intellectual Property and Records considerations.

Even though we don’t plan to directly participate, I’m really looking forward to this. It is great fun working with our customers’ in-house Bioinformatics solutions, and I’d love to see that level of innovation in Cheminformatics.

I don’t know the details of the specific case but this recent legal case where an employee stole trade secrets before leaving for a competitor brings up another way.

Specifically, if they had been using PatentSafe:

• Every read of a document would have been logged
• The Custodian’s Console would have alerted management to unusual activity
• Users would know they were accountable for the company’s secrets

The last of those is most important. If people know they are accountable this would never have happened at all – the scientist wouldn’t have been tempted, the company wouldn’t have had to prosecute, lots of money would have been saved and a prison sentence avoided.

Flexibility is the ability to change how software works; power is the ability to do more with less effort

Historically we have been sucked into believing that “Flexibility = Power” and the complexity of most ELNs shows the result. As Brent mentions, iOS and the like are causing us to understand that the benefits come from Power – that’s how we deliver the return on investment. Flexibility can distract from that.

I suspect in the future we’ll see a lot more focused, powerful systems which allow us to concentrate on the job in hand rather than having to figure out how to configure all the flexibility we thought we wanted.

When you add a new App to your iPad/iPhone you are generally up and running within seconds. Imagine if that was the user experience for ELNs…

When conference organizers count people, the number they care about most is registered paying attendees; they track that every day as the conference approaches. Suppose you are an outsider, considering attending or sponsoring or exhibiting at the conference, and you inquire as to the likely attendance. You will never be given the real number; instead you will be told a number which is at least twice that, and usually higher. This is justified by including trade-show exhibitors’ staff, the conference organizer’s own people, PR folk and journalists, the food service crew, and is basically pulled out of a monkey’s butt. Just thought you’d like to know.

Yup.

When a delegate ticket is the price of a small PatentSafe installation, and vendors pay over $10k to even turn up and give a “talk”…

There has to be a better way for the industry to talk to each other.

So at this time the data portability situation in the ELN world is pretty awful. Which is a shame, and at some point people are going to start noticing – and perhaps the next round of ELN purchases will have open file formats as a purchasing consideration.

I came across the Data Portability project in this article on Tech Crunch which seems to be a really nice way of at least making the Data Portability issues obvious to consumers. They are starting off in the online web app area but clearly it is very relevant to any IT system, either cloud-based or on premises.

For the record, Amphora’s systems are completely open – our view is that it is your data and you should be able to take it where you want, when you want, without even having to involve us.

In addition, our focus on IP means we need to be able to reassure our customers that they can take a record out of our ELN and defend their IP long after their relationship with Amphora has come to a close – with a 50 or 100 year retention timescale, requiring the vendor to be around just isn’t acceptable (which is a big concern with services that claim to outsource IP protection, something I’ll blog on in due course).

We take this a step further in our Hosted/SaaS offerings, where customers can take a copy of their data (via rsync or similar) onto another server controlled by them every night. We also work with those customers to make sure they can spin up their own server as needed. This means that even where we’re Hosting them, they can tell us our services aren’t required and still have complete access to their data without any cooperation for us.

We believe that open data, neutral file formats, powerful APIs and above all a respectful policy to our customer’s IP are the cornerstone of any ELN vendor’s offering.

Our next web site refresh will contain our Data Portability policy. In the meantime I can only hope that as various advocacy groups get more vocal about the need for Facebook, Twitter and others to unlock your data, that will cause Data Portability to be given the consideration it deserves in the ELN world.

We’ve always had a very straightforward model; because our Electronic Lab Notebook needs to know about every user to do its job (every user needs to be uniquely to sign and witness their entries), it is easy to just charge for the number of users which are enabled on the system. Fairly straightforward.

We do have a slight complexity in that we split PatentSafe functionality into modules, so you can have cheaper licenses for people who just want to read, or people who just want to submit and sign stuff, etc. We have assigned “points” values to these functions, and customers buy a certain number for their system which gives them a lot of flexibility.

Sometimes we’re asked about concurrent pricing but that request generally comes from an IT dept who are (quite reasonably) looking for ease of administration and don’t realise every user is going to be setup (automatically or otherwise) with a PatentSafe account anyway. Concurrent licensing wouldn’t help anyone in our use case.

We do have a mix of perpetual and rental options in our licensing structure; this accommodates customers who have capital and want the reassurance of owning something (generally larger more established customers) as well as making enterprise-grade solutions attainable to companies who might either be short of capital (being VC funded and at the end of a round) or unsure of their growth curve.

We don’t do anything silly in terms of copy protection; it just adds to pain for the users, and ultimately support pain for us! We don’t even lock a system to a particular number of users – PatentSafe just points out how many users you have and we trust our customers to have that many licenses. We also don’t charge for test servers, although we do make a small additional support charge if you want production-level support for an additional server.

I guess we are unusual in that we’re a records system which is intended to go into court at some point, so we can trust our users to do the right thing in terms of having the right number of licenses. This level of trust makes everything easier, and interestingly means users are more trustworthy back – in all the years I don’t think I’ve ever heard of a reason to be worried that a customer may be exceeding their entitlement.

I know some software companies view having a complex licensing structure as a sales tool but I’ve never found it all that attractive. We’re providing and supporting a tool which will benefit a customer’s organisation, and the question is how to fairly measure the value we provide and hence should be compensated for. We’ve found the easiest and most reliable way to do that is count up how many scientists we’ve freed from the drudgery of the Bound Paper Lab Notebook, so that’s how we price the system. Simple, transparent, predictable – and fair.

Interestingly treating our customers like adults means they act like adults; and the relief expressed in the sales cycle when the realise we’re straightforward to deal with is quite gratifying! I continue to remain befuddled as to why more companies can’t have understandable pricing schemes – I can’t see how complexity helps the vendors or the customers.

In a move that I take as more of a sign of weakness, Adobe have placed an advert in the press basically saying they love Apple, but Apple are being unreasonable.

I can’t improve on TechCrunch’s thoughts this whole thing. Adobe have painted themselves into a corner and I suspect Flash is a dead end technology. They’d do better to admit that and move on, rather than flailing around like so many other failing businesses – the Flash franchise wasn’t as strong as they tried to make out (being dependant on a plugin being installed), they overplayed their hand, and Steve Jobs called their bluff.

Requirements Gathering – a Broken Process

Project teams have been drawing up lists of requirements since the dawn of time, and since that first list the fate of a project has to a great extent been sealed the moment the requirements have been finalized. As a Civil Engineer my Father was always bemoaning the unrealistic requirements forced upon him by “dreamer” architects, a feeling that I suspect has dominated construction since the pyramids. I recall many a tale of tense ad-hoc negotiation on the construction site, and even the removal of troublesome architects from site entirely!

IT projects share a great many similarities with Construction, although a Civil Engineer has the advantage that she can point out the very obvious real-world difficulties (“You want me to build a roof of glass that large without any supports to spoil the view?”) whereas IT implementors often suffer from the perception that everything is easy. The progress in our field has been so rapid that our customers are used to apparent miracles, and of course there’s always the potential for distrust to arise due to the often large culture gap between the “Geeks” and “Normal people” – and that’s nothing compared to what people often think of Vendors and their dastardly sales people!

It is well known that IT projects can fail to meet expectations, and indeed some of the statistics on IT project success make sobering reading. Plenty of studies have shown the criticality of requirements setting in project success and indeed some practitioners (for example in the “Agile” and “Customer Development” movement) have gone so far as to remove what they perceive to be a very error-prone requirements setting step from their project entirely.

Unfortunately most large companies have purchasing processes which require the business to decide what they need, and then go out to the market to get the “best” solution from what they hope is a selection of competing solutions via a Request for Proposal (RFP) or other formal process. The need to maintain fairness to all concerned means the RFP process if often very rigid and leaves little scope for modification of the requirements in the light of the reality of implementation.

Therefore we are faced with a situation where a potential user’s desire to get the best possible solution forces them into a situation where they have to place almost total reliance on their up-front requirements setting process, an “Aim, and hope” approach which would be abhorred in another sphere of corporate activity. Who would design a process which explicitly prevented feedback from influencing the initial conditions?

And yet, we are here – to paraphrase Winston Churchill, the RFP process and it’s reliance on Requirements Specification maybe the worst way of purchasing systems but in today’s corporate environment is better than anything else available. In the spirit of making the best of the circumstances, this article will take the opportunity to make some suggestions and point out common pitfalls which so many implementation teams fall into.

The problems of requirements and the resulting issues in the RFP process are all the more critical for ELN projects because:

• The ELN replaces an existing paper process, which through the mists of time is often badly understood in itself.
• The term “ELN” can cover such a wide range of functionality and domains that in itself it is a foundation for confusion.
• Most of the current ELN offerings are still only “first generation” solutions which come with their own set of problems.

Thus an issue which bedevils all IT projects is often the founding cause of ELN project failure and requires particular attention.

Apparent “Solutions” which often aren’t

The difficultly in requirements capture/generation are apparent to anyone who has participated in such a project. A common approach is to hire a consultant, and when you get the right one they can single-handedly turn the situation around, although it does require the customer to listen! With apologies to the great consultants out there, the presence of a “consultant” leads to groans in the vendor community because (fairly or not) consultants:

• Are sometimes viewed as having an interest in creating long running complex projects, rather than quick productive wins.
• Despite having lots of industry experience, seldom get a long term view of a project. Often their involvement is restricted to the purchasing process, after which the vendor takes over. This gives them a rather one-sided view of the process.
• Need to get paid and that means making sure the customer is happy. Unfortunately sometimes the customer needs to be told some uncomfortable truths which might lead to them being “fired” as a customer – something a vendor with a large customer base can do, but very hard for a solo consultant to do.

Another approach is to “Stick with who/what you have and know” – for example, if you have an established implementation of SAP or a Document Management package, those solutions might be bent or tailored to meet the new requirements. Unfortunately this doesn’t remove the requirements generation pitfall, and leaves you with an expertise gap as domain-specific solutions ideally come with a vendor who spends their days working on a particular area, who can bring that expertise to bear both in the solution itself and the implementation process.

Some companies will already have an “ELN” deployed in one area, and there is a temptation to view this as being suitable for all kinds of science. This is a sad outcome of the rather generic “Electronic Lab Notebook” term, and is one of the primary reasons why we prefer to avoid the term in day-to-day use; “science” is by definition a very varied activity and you can’t assume that just because two different departments use the same Paper Lab Notebook, that a single ELN will work well in both places. Often all these groups will have in common is they both call themselves “Scientists” and work for the same company – hardly a basis for a common toolset.

It is important to note that these solutions – consultants, re-use of a horizontal tool, and a common ELN across multiple disciplines – aren’t in themselves inherently flawed and can indeed lead to a successful project. There is however a risk of viewing them as the solution to what is at heart a very tricky problem, and project teams who think they’ve somehow reassured themselves of success are often painfully brought back to reality. As the Financial Crisis has taught us, risk doesn’t go away by magic and sometimes the very approaches we take to remove it in fact just increases it, more dangerously so because we’ve stopped being sensitive to it.

Perhaps as Andy Grove says, “only the paranoid survive” and the ultimate key to project success is the recognition that any solution to risk reduction has the potential for problems in itself, often in ways you least expect it.

A Modest, Sadly Unrealistic, Proposal

This article has presented a bleak assessment of how most teams are forced by circumstances to approach their ELN projects, as well as pointing out some common pitfalls that requirements gathering processes fall into. Whilst the problems implicit in these approaches can’t be removed, I hope I have provided at least the opportunity for some reflection. In closing perhaps I might offer some suggestions which I know are unrealistic, but might one day mitigate the issues I’ve described.

One of the problems with an RFP process is the lack of feedback from the implementors; I am sure I am not alone in looking at some requirements and thinking “This project is doomed”. I for one would welcome the opportunity to answer some additional questions, such as:

• “If you could remove 5 of our requirements what would they be and why?”
• “What are the most expensive/troublesome requirements listed above?”
• “Which of these requirements do you think we don’t really need, based on your experience of similar projects?”
• “What are we missing?”
• “If you were us, what are the three things you would be most worried about going forward?”
• “Please rate our chance of success if we go with you, and if we go with another vendor, with reasons”

These questions would afford thoughtful vendors the opportunity to reflect and contribute their experience – after all, for all the conflict of interest that you might perceive in a vendor/customer relationship, a vendor only ultimately succeeds when their customer succeeds. Any vendor team is easily going to see ten times the number of ELN projects that any customer or indeed consultant will see in a year.

Sadly whilst all these questions are interesting I don’t know what project teams would do with this information! In so many cultures project managers are rewarded for following a process and thus any failure is blameless, any reconsideration a failure.

Other Approaches

One very interesting approach we’ve just experienced was where the prospective customer held an RFI (non-binding Request For Information) process which was rather like an RFP but held outside a commercial purchasing process. Crucially the RFI submission and scoring was then followed up with a 1 hour feedback meeting between the customer and the vendor team which allowed for a lot of constructive discussion which no doubt benefited both sides.

We have had good results from projects which have used a Six Sigma methodology with plenty of contributions from all parties – end users, management, IT departments, and outside vendors. This approach tends to be too “heavy” for smaller companies but has delivered great results in larger companies where Six Sigma is part of the culture.

This illustrates the final and most important point: a successful solution is the product of a partnership between everyone involved, and even the largest most process-driven companies reinforce this in their process. Perhaps the greatest danger lies in taking a very formal approach in the purchasing process without counterbalancing that with an up-front listening process – a trap that growing companies often fall into as they formalize their purchasing process without having developed the experience and resources to learn from themselves and the rest of the industry.

Human progress happens at 4% per year, averaged across many fields, ranging from 2% to 7%

How interesting! I wonder what the state of play is for ELNs…. I suspect we’d like to think it’s something like 10% or 20%, but I fear that if we took a good look at what’s really going on, it would be somewhat lower! One of the problems we currently face as an industry is there’s too much marketing spin in our communications and not enough real understanding of what’s going on.

I’d like to think at Amphora we’re on the realistic side – replacing the Paper Lab Notebook is a hard problem and we’re only going to get better if we pay attention to the whole problem and are clear-eyed about what’s really happening. Geeks at heart, we run a lot of stats about the performance of the business and customer experience and it produces really thought-provoking insights.

It isn’t just about the product, there’s lots more involved to the customer experience – from the sales process, pilot & training, commercial packaging, infrastructure requirements etc. all have a significant impact. Interestingly from a product perspective it is often more about what functions you leave out rather than making a complex mess, which does take a lot of discipline. For example we were recently eliminated from an RFP because we didn’t meet a “mandatory” requirement but we’d prefer to lose a deal than bend PatentSafe in weird ways which would lead to a more complex product which was harder for everyone to use. Purity of design is a good thing (yes, we’re Apple fans!).

With regard to “bed behaviour” particularly by Sales & Marketing personnel, I’m afraid I’m one of those naive people that believe open conversation spurs creativity, and if people act badly they’ll just look like idiots to the very people they are trying to “impress”. As long as there’s transparency it tends to self-correct.

However, I realise that only a few of us are willing or indeed able to Blog, so forums such as LinkedIn groups definitely have a role. There are a couple of ELN groups on LinkedIn, one’s pretty quiet and the other is somewhat stifled by rather outmoded policies.

So I’m delighted that a more open ELN group has been setup on LinkedIn – you can join it here and I would strongly recommend you do.

Here’s the group description:

This group was set up to provide a place where all can discuss content free from restrictions.

By joining this group you are agreeing to allow other group members to make comment on your discussions free from any risk of litigation or lawsuit. This means other members are free to make reference to trademarks and express their opinion positive or negative. The only recourse is to answer their discussion. Abusive replies will be removed.

Content from this discussion should not be referenced elsewhere.

Innovation requires uncensored discussion.
So feel free to discuss anything related to this topic in here.

I really like this approach, the only quibble I might have is “Content from this discussion should not be referenced elsewhere” which is going to be really tricky in practice – good ideas and discussion spread, that’s the whole point. But I’m sure we’ll have a discussion about it .

FYI the group was setup by Andrew Lemon who runs The Edge, producers of BioRails which is on my list of “interesting things to look at when I have time”. So guess he’s a fellow supplier although I’m not sure we compete that much – we briefly met at a conference once, but apart from that I have no relationship with him. I’m just pleased that someone is doing something to enhance the quality of conversation in the industry because we urgently need to something to improve the success rate and ROI of ELN implementations.

• Business drivers
• Comparing the different sectors and disciplines
• Build or buy?
• An overview of the solution space
• Patterns of success

There’s a few concepts in here which deserve their own posts (presentations are so useful for stimulating the creative juices!) which hopefully I can do over the coming weeks.

Anyway, you have to give a 10 minute presentation talking about wider industry issues and I thought it was appropriate to draw people’s attention to what’s happening in the consumer space and how it might bring us towards the original vision of an ELN.

I’ll blog more on this tomorrow (after the Apple announcement today!) but for the moment here’s the presentation….

Having said that, whilst I believe we’ve done a really good job in PatentSafe replacing the corporate aspects (record keeping, long term records etc.) of the Lab Notebook, scientists still need a place to work. Sometimes that’s a discipline-centric product (sometimes badged as an “ELN”, sometimes something else), sometimes Microsoft Office and other general Knowledge Worker tools.

Looking forward I can’t help but think that tools like Google Wave and WordPress (especially with 2.9′s nifty features) are the long term future. A lot of vendors have “Web based” ELNs which are nothing more than their thick-client products wrapped in a browser – which I’ve always felt is cheating.

But when you look at what people are doing with web-native UIs these days…surely the next generation of Scientific collaboration products are going to come from the blogging or Web 2.0 space, with a little chemistry added to the mix. They’re cheaper, easier to use, easier to deploy, and often more powerful than a typical thick-client “Enterprise” app – and I suspect they’re more capable of dealing with large-scale use than any of the commercial products on offer at the moment (the lack of scalability being the dirty little secret of most ELN deployments right now).

All these tools need – apart from some open mindedness – is a decent record keeping system. Which we would be happy to help with

What an exciting time…

This discussion will start out with identifying the many forms of information we create and use, and the needs of those doing the creating and using. It would then move on to how best to share this information within our organization, and with our customers and partners in a secure manner. Our mental model will be the most well-known computer network – the Internet.

I really quite like this. I think The Internet has a lot to give in terms of sources of inspiration and it’s sad that the Lab Informatics market has been rather knocked off course by an obsession with a paper artefact rather than looking at what’s really going on.

I’ve always felt fortunate that we’re operating in one of the few “Green field” markets for IT systems. I thought my age would condemn me to working on incremental IT projects, as opposed to all the fun my predecessors must have had in the 80′s and 90′s going into manual processes and achieving quite amazing business impact by automating them.

Perhaps what’s happened is our generation have forgotten some of the basic system analysis skills that our Dads used?

Regardless of the cause, I strongly suspect if the Paper Lab Notebook didn’t exist we wouldn’t have come up with the concept of an Electronic one. Which does make you wonder how much more effective we could all be if we focused on the real problems scientists and their companies have?

I’m sure writing them is difficult.

But answering them is deeply frustrating – it’s being forced to have a one-way conversation about something where you really need to have a chat, which is much more my natural style.

I remember once I turned up to a Government research establishment and was told “We can’t tell you what we do, or anything about how we work. In fact, the people in this room can’t tell you anything at all. So just tell us what you do, and show us your stuff”. Which was painful, but at least the audience gave some (non-verbal) feedback.

Guess I’d better get back to it – although I think my car might need washing, and there are some other chores to do. Or I might just poke myself with a sharp stick or something.

It’s quite nice to have a Queen in such circumstances – she’s above it all (so there’s no finger pointing and has so much respect that when she asks people they jump. So when she asked the obvious question – why no one anticipated the Credit Crunch – a bunch of them sat down to figure out why.

The answer is very relevant. It’s why systems fail, Space Shuttles explode, companies fail, and in my particular interest – how a group of well meaning people in a company can fail in their quest to replace a Paper Lab Notebook with an Electronic Lab Notebook. From the article:

“Everyone seemed to be doing their own job properly on its own merit. And according to standard measures of success, they were often doing it well,” they say. “The failure was to see how collectively this added up to a series of interconnected imbalances over which no single authority had jurisdiction.”

Besley stressed that the experts had not been in “finger-wagging mode” and had agreed that the causes of the credit crunch were extremely complex. “There was a very complicated, interconnected set of issues, rather than one particular person or one particular institution.”

….. snip …..
“In summary, Your Majesty,” they conclude, “the failure to foresee the timing, extent and severity of the crisis and to head it off, while it had many causes, was principally a failure of the collective imagination of many bright people, both in this country and internationally, to understand the risks to the system as a whole.”

Lots of bright people. Complex problem, with interlinked components, dimly understood, with diffuse responsibility. Everyone working diligently on their own bit. It’s no ones fault – they all tried their best, etc.

Sounds very much like a lot of ELN projects . ELNs by their very nature are complex, span multiple areas of expertise, and require different parts of the organization to have conversations they aren’t used to having.

This is why we run “Fire Drills” – to get the customer’s organization to see the whole picture. Works very well. This overall perspective is one of the most valuable things we can bring to our customers, the result of an awful lot of interesting experiences in many different ELN deployments, across a whole variety of industries and company sizes.

Sadly we see a lot of ELN projects which are doomed from the very start, because they fail to appreciate and manage the complexity of what they are trying to undertake. I hesitate to say it, but I am inherently skeptical of any ELN project team, or indeed any consultant, who claims to be fully in control – these projects just aren’t that simple. If you think you know what you are doing, you’re probably just unaware of the problems which are going to seriously ruin your day.

For me, a healthy paranoia is much more encouraging as it shows respect for the challenges ahead. The Credit Crunch has “cost” each of us a mind-boggling amount of money and it would be a shame to lose the lesson.

We use a lot of Open Source components in our products, and I know we’re not alone.

There are vendors who are very committed to a specific platform – Windows (and associated libraries, APIs etc.), Oracle, and so on. Those will almost certainly have some Open Source components in them, but not much.

There are other vendors – Amphora (my company) and Rescentris are two that I know of – who have built on top of an Open Source stack. We do have some proprietary components but where there’s an Open Source alternative we use that.

Why?

• The Open Source stuff just works better (and you can fix it if it isn’t).
• Support is better.
• Licensing issues go away (of course you have to abide by the Open Source license, but that’s not a problem as long as you check everything before the developers start using it).
• It is dramatically cheaper for our customers to deploy. No expensive additional Windows Server or Oracle licenses (unless you want to use those of course – we can support them if you prefer).
• We have much more latitude in deployment options. We can bundle our product in a variety of ways and on different platforms, which we wouldn’t be able to do if we were locked to a specific commercial platform. You can get PatentSafe as everything from SaaS, to an embedded device, to a traditional “Install this on your own server” software product.

A few years ago we got raised eyebrows about our platform choices (“We will only consider applications written in .Net”) but that’s not been an issue for a long time. Everyone assumes that the components we’ve assembled into the solution will work and we’re responsible for the overall performance of that – what bits we’ve chosen seldom get discussed.

From what I can see, Open Source starts at the bottom of the stack – the OS, generally – and is gradually moving up (Database, Application Server, some applications). Every commercial vendor needs to keep an eye on what value they are bringing compared to what’s provided by the community.

As an aside, we don’t consider ourselves to be a “Software Vendor”. We solve a business problem and it just so happens we deliver our expertise as some software which implements a “best in class” process. But we don’t consider we’re charging for “software” – we get paid for our expertise and how we deploy that to help customers solve their problem.

What that means is that as the software environment changes (and quite probably Open Source gets further up the stack) that’s not something that threatens our identity. I know some vendors (particularly those locked to proprietary platforms) aren’t so lucky, and I wonder how they will fare as the Open Source community begins to provide more and more of the “ELN” system.