Positive Feedback on eHealth Risk - Opportunity Workshop

The attendees at the first eHealth Risk-Opportunity Report Card Workshop on April 17/18 at the University of Waterloo gave good grades to the Waterloo Institute for Health Informatics Research and workshop instructor Brendan Seaton for the event. Average participant evaluation scores for workshop content was 4.3 out of 5, and for presenter knowledge and presentation 4.5 and 4 out of 5 respectively. Overall organization of the workshop was rated at 4.4 out of 5.

A thank you to the participants who were fully engaged and who offered a great deal of constructive criticism and advice for the next version of the report card.

Comments on eHealth Risk Opportunity Report Card Paper

Rupak Mazumdar, Senior Risk Analyst for the Ontario Smart Systems for Health Agency offers the following comments on the Report Card Paper.

• In general, I think that you have put together a well-thought out paper.
• I like Page 5 where you have indicated that each eHealth program can apply flexible weights to the report card. Risk management tools need to be flexible to address different situations.
• The opportunity-risk matrix in Table 5 is also a nifty idea.
• I am very interested in seeing the paper put into real practice. In fact, application of the paper is a theme for me. Applying this to a pilot area of eHealth will be the true test of what works, what does not work and what needs to be refined a bit.

• The biggest potential obstacle that you may face is with the definition of risk.
• There are many competing definitions of the word risk out there. You have defined risk as the possibility that a threat will be realized resulting in harm or loss.
• The risk management world is slowly moving towards more of a risk definition that is uncertainty of outcome, with an outcome potentially having both positive and negative elements.
• I like the uncertainty definition much better because it inherently contains both the positive and negative definition of risk. It also forces the risk assessor / manager to focus on events and scenarios that have not occurred yet as opposed to issues that have already materialized.

I may seem to be quibbling but I have come to the epiphany that when I talk about risk management, often everyone in the room has different definitions of risk (a bad thing, uncertainty, an impact, a possibility, etc.). If you agreed with the semantics change, the language would change slightly in the paper. For example, in the title:

The eHealth Risk Report Card:

A practical approach to realizing opportunities in eHealth from understanding and managing risk

• Many in the audience for this paper could potentially have little understanding of risk management methodology. Hopefully that is not the case too often as a CIO should have at least a rough understanding of risk management. Still, you may want to have a brief primer on risk management in your back pocket or in an Appendix.

• Much of the material that was brought over to SSHA to help define the risk management framework came from the brilliant Australian Risk Management Standard AS/NZS 4360:2004 RISK MANAGEMENT. You will want to cite the framework in your end notes.

What can go wrong with eHealth?

The British Parliament's Committee of Public Accounts has published a report that is very critical of the NHS' National Program for IT. It identifies many issues which are captured in our eHealth Risk-Opportunity Report Card. This is a list of their conclusions and recommendations [words in square brackets are mine]:

1. The delivery of the patient clinical record, which is central to obtaining the benefits of the programme, is already two years behind schedule and no firm implementation dates exist. [project risk]
   
2. The Department has not sought to maintain a detailed record of overall expenditure on the Programme and estimates of its total cost have ranged from £6.2 billion up to £20 billion. [business risk]
   
3. The Department's investment appraisal of the Programme did not seek to demonstrate that its financial benefits outweighed its cost.[business risk]
   
4. The Department is maintaining pressure on suppliers but there is a shortage of appropriate and skilled capacity to diliver the systems required by the Programme, and the withdrawal of Accenture has increased the burden on other suppliers, especially CSC. [project and business risk]
   
5. The Department needs to improve the way it communicates with NHS staff, especially clinicians. [project and business risk]
   
6. We are concerned that the leadership of the Programme has focused too narrowly on the delivery of IT systems, at the expense of proper consideration of how best to use IT within a broader process of business change. [benefits realization risk]
   
7. The Department should clarify responsibility and accountability for the local implementation of the Programme. [business risk]
   
8. The use of only two major software suppliers may have the effect of inhibiting innovation, progress and competition. [project and business risk]
   
9. At the present rate of progress it is unlikely that significant clinical benefits will be delivered by the end of the contract period. [benefits realization and project risk]
   

It would be interesting to see what grade the NHS would get for their Programme.

Download White Paper

I have finally put together a complete White Paper on the eHealth Risk-Opportunity Report Card. It is available for download in its FIRST DRAFT form. I am looking for comments and criticisms on the concept and the approach to managing eHealth opportunity and risk. In particular:

1. Are there any serious inaccuracies?
2. Are there sections or terms that require more explanation?
3. Could this be used by competent people in average health care organizations?
4. Are there points that should be footnoted and sourced?
5. Are there alternative standards that would be better than the standards referenced in the paper?

I plan to post all comments on this blog. A second draft will be available in a couple of weeks after I've digested comments and criticisms from the first round of review.