ONC and Misdirection Regarding Mass Healthcare IT Failure

In my keynote address to the Health Informatics Society of Australia in Sydney recently, I cautioned attendees including those in government to be wary of healthcare IT hyper-enthusiast misdirection and logical fallacy (a.k.a. public relations).

In the LA Times story "Patient data outage exposes risks of electronic medical records" on the Cerner EHR outage I wrote of in my post "Massive Health IT Outage: But, Of Course, Patient Safety Was Not Compromised" (the title, of course, being satirical), Jacob Reider, acting chief medical officer at the federal Office of the National Coordinator for Health Information Technology is quoted.  He said:

"These types of outages are quite rare and there's no way to completely eliminate human error."

This is precisely the type of political spin and hyper-enthusiast misdirection I cautioned the Australian health authorities to evaluate critically.

As comedian Scott Adams humorously noted regarding irrelevancy, a hundred dollars is a good price for a toaster, compared to buying a Ferrari.

Further, when you're the patient harmed or killed, or the victim is a family member, you really don't care how "rare" the outages are.

Airline crashes are "rare", too.   So, shall they just be tolerated as a "cost of doing business" and spun away?

(As I once wrote, the asteroid colliding with Earth that caused the extinction of the dinosaurs was a truly "rare" event.)

It seems absurd for me to have to point out that paper, unless there is a mass outbreak of use of disappearing ink, or locally hosted clinical IT, do not go blank en masse across multiple states and countries for any length of time, raising risk across multiple hospitals greatly, acutely and simultaneously.   Yet, I have to point out this obvious fact in the face of misdirection.

Locally hosted health IT, of course, can only cause "local" chart disappearances.  "Local" is a relative term, however, depending on HC organization size, as in the example of a Dec. 2011 regional University of Pittsburgh Medical Center (UPMC) 14-hour outage affecting thousands here.

Further, EHR's and other clinical IT, whether hosted locally or afar, had better offer truly major advantages, without major risks and disadvantages, over older medical records technologies before exposing large numbers of patients to an invasive IT industry and the largest unconsented human subjects experiment in history.

Unfortunately, those basic criteria are not yet apparent with today's systems (see for instance this reading list).

EHR's and other clinical IT, forming in reality an enterprise clinical resource management and clinician workflow control apparatus, have introduced new risk modes including mass chart theft (sometimes tens of thousands in the blink of an eye); also, mass chart disappearances as in this case - all not possible with paper.

At the very least, if hospitals want enterprise clinical resource management and clinician workflow control systems, these should not be relegated to a distant third party.  Patients are not guinea pigs upon whom to test the ASP software model ("software as a service") that, upon failure for any reason, threatens their lives.

Finally, these complications are a further example why this industry cannot go on without meaningful oversight.  The unprecedented special medical device regulatory accommodations must end.

-- SS

Malpractice Attorney Puts ONC-Authorized Testing and Certification Bodies (ATCBs) at Risk of Litigation?

I am jet-lagged after returning from Sydney, Australia, where I delivered one of the keynote addresses at the Health Informatics Society of Australia annual conference, HIC 2012 (http://www.hisa.org.au/page/hic2012/).

My theme in a talk entitled "Critical Thinking on Building Trusted, Transformative Medical Information:  Improving Health IT as the First Step" was health IT trust and safety.  I was actually invited in 2011 but could not attend; I was helping care for my mother, who was severely injured due to a HIT-related mishap in 2010.  Her death in 2011 allowed me to attend now on re-invitation.

More on my presentation later.

A beautiful view of the Sydney Harbour Bridge and Opera House, taken with a mere Canon SX110IS.  Click to enlarge.

In the meantime, I returned to the U.S. to find that the defense attorney for the hospital where my mother was severely injured, and then died as a result, is once again raising an absurd issue in objections to the medical malpractice Complaint that was refiled within the Statute of Limitations for technical reasons.   The President Judge of the county where the case is filed had dismissed this complaint (among many others) some time ago:

(ii) Plaintiffs Software Design Defect Claims are Preempted by the Federal HITECH Act

... To the extent Plaintiff attempts to bring a common law product liability claim against [name redacted] Hospital for required use of EMR software, such a claim is barred due to Federal Preemption of this area with the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act. 42 U.S.C. 201, 300, et seq.

Specifically, the design, manufacture, specification, certification and sale of EMR in the United States is a highly regulated industry under the jurisdiction of the Department of Health and Human Services (HHS). The HHS draws its statutory authority to design and certify EMR as safe and effective under the HITECH act as amended. Id.

The Supremacy Clause of the United States Constitution, article VI, clause 2, preempts any state law that conflicts with the exercise of federal power. Fid. Fed. Say. & Loan Ass’n v. de la Cuesta, 458 U.S. 141, 102 S. Ct. 3014 (1982). “Pre-emption may be either express or implied, and ‘is compelled whether Congress’ command is explicitly stated in the statute’s language or implicitly contained in its structure and purpose.” Matter of Calun Elec. Power Co-op., Inc., 109 F.3d 248, 254 (5th Cir. 1997) citing Jones v. Rath Packing Co., 430 U.s. 519, 525 (1977).

In this case, to impose common law liability upon [name redacted] Hospital for using certified EHR technology, which was in compliance with federal law and regulations for Health Information Technology, would directly conflict with Congress’ statutory scheme for fostering and promoting the implementation and use of EHR 

I really don't think Congress intended HIT to maim and kill patients with impunity.  In any case, this assertion was thrown out in its entirety several months ago, but here it is again in a new set of objections.  I find its reappearance remarkable.  I also wonder if the industry is behind it.

As per numerous posts in this blog, such assertions are false - and likely knowingly so in this situation.  (In that case, this would be an even more serious matter.)

For example as I pointed out at my Feb. 2012 post Hospitals and Doctors Use Health IT at Their Own Risk - Even if "Certified", ONC-Authorized Testing and Certification Bodies (ATCB's) answered my questions about safety, legal indemnification etc.  Their work has nothing to do with certifying HIT as safe by their own admission.

Also, as in my April 2011 post FDA Decides Regulating Implantable Defibrillator Medical Devices a "Political Hot Potato"; Demurs and my Nov. 2011 post IOM Report - "Health IT and Patient Safety: Building Safer Systems for Better Care, the HIT industry is unregulated.

On the HIT regulation issue, IOM has itself stated in no uncertain terms that HIT is non-regulated (not "a highly regulated industry") in their report to HHS.  For instance, in the aforementioned 2012 report they state (as one example):

... If the Secretary [of HHS] deems it necessary for the FDA to regulate EHRs and other currently nonregulated health IT products, clear determinations will need to be made about whether all health IT products classify as medical devices for the purposes of regulation. If FDA regulation is deemed necessary, the FDA will need to commit sufficient resources and add capacity and expertise to be effective.

I won't even address the claim that the HITECH Act represents or intended to represent Federal pre-emption of state common law rights.   It's without merit, and actually absurd.

Worst of all, statements in legal dockets that "HHS draws its statutory authority to design and certify EMR as safe and effective under the HITECH Act" (in reality, private non-governmental ONC-Authorized Testing and Certification Bodies or ATCB's are appointed by ONC to "certify" HIT features and functionality to be compliant with "Meaningful Use" guidelines and do not test for safety or efficacy) potentially puts those private ATCB's at risk for being named defendants in lawsuits where HIT was found unsafe and/or ineffective if upheld.

I am sure the ATCB's and ONC would not be happy about that.

-- SS

WSJ "There's a Medical App for That—Or Not" - Misinformation on Health IT Safety Regulation?

There's a health IT meme that just won't die (patients may, but not the meme).

It's the meme that health IT "certification" is a certification of safety.

I expressed concern about the term "certification" being misunderstood even before the meme formally appeared, when the term was adopted by HHS with regard to evaluation of health IT for adherence to the "meaningful use" pre-flight features checklist.  See my mid-2009 post "CCHIT Has Company" where I observed:

HIT "certification." ... is a term I put in quotes since it really is "features qualification" at this point, not certification such as a physician receives after passing Specialty Boards.

The "features qualification" is an assurance that the EHR functions in way that could enable an eligible provider or eligible hospital to meet the Center for Medicare & Medicaid Services' (CMS) requirements of "Meaningful Use."  No rigorous safety testing in any meaningful sense is done, and no testing under real-world conditions is done at all.

I've seen the meme in various publications and venues.  I've even seen it in legal documents in medical malpractice cases where EHR's were involved, as an attempted defense.

Now the WSJ has fallen for the health IT Certification meme.

An article "There's a Medical App for That—Or Not" was published on May 29, 2012.  Its theme is special regulatory accommodation for health IT in the form of opposition to FDA regulation of devices such as "portable health records and programs that let doctors and patients keep track of data on iPads."

In the article, this assertion about health IT "certification" is made:

... The FDA's approach to health-information technology risks snuffing out activity at a critical frontier of health care. Poor, slow regulation would encourage programmers to move on, leaving health care to roil away for yet another generation, fragmented, disconnected and choking on paperwork.

The process already exists for safeguarding the public for computers in health care. It's not FDA premarket review but the health information technology certification program, established under President George W. Bush and still working fine under the Obama Health and Human Services Department. The government sets the standards and an independent nonprofit [ATCB, i.e., ONC Authorized Testing and Certification Bodies - ed.] ensures that apps meet those standards. It's a regulatory process as nimble as the breakout industry it's meant to monitor. That is where and how these apps should be regulated.

It's a wonderful meme.  Unfortunately, it's wrong.  Dead wrong.

Certification by an ATCB does not "safeguard the public."   Two ONC Authorized Testing and Certification Bodies (ATCB's) admitted this in email, as in my Feb. 2012 post "Hospitals and Doctors Use Health IT at Their Own Risk - Even if Certified".  I had asked them, point-blank:

"Is EHR certification by an ATCB a certification of EHR safety, effectiveness, and a legal indemnification, i.e., certifying freedom from liability for EHR use of clinical users or organizations? Or does it signify less than that?"

I received two replies from major ONC ATCB's indicating that "certification" is merely assurance that HIT meets a minimal set of "meaningful use" guidelines, not that it's been vetted for safety.  For instance:

From: Joani Hughes (Drummond Group)
Sent: Monday, March 05, 2012 1:06 PM
To: Scot Silverstein
Subject: RE: EHR certification question

Per our testing team:

It is less than that. It does not address indemnification although a certification could be used as a conditional part of some other form of indemnification function, such as a waiver or TOA, but that is ultimately out of the scope of the certification itself. Certification in this sense is an assurance that the EHR functions in way that could enable an eligible provider or eligible hospital to meet the CMS requirements of Meaningful Use Stage 1. Or to restate it more directly, CMS is expecting eligible providers or eligible hospitals to use their EHR in “meaningful way” quantified by various quantitative measure metrics and eligible providers or eligible hospitals can only be assured they can do this if they obtain a certified EHR technology.

Please let me know if you have any questions.

Thank you,
Joani.

Joani Hughes
Client Services Coordinator
Drummond Group Inc.

The other ATCB, ICSA Labs, stated that:

... Certification by an ATCB signifies that the product or system tested has the capabilities to meet specific criteria published by NIST and approved by the Office of the National Coordinator. In this case the criteria are designed to support providers and hospitals achieve "Meaningful Use." A subset of the criteria deal with the security and patient privacy capabilities of the system.

Here is a list of the specific criteria involved in our testing:
http://healthcare.nist.gov/use_testing/effective_requirements.html

In a nutshell, ONC-ATCB Certification deals with testing the capabilities of a system, some of them relate to patient safety, privacy and security functions (audit logging, encryption, emergency access, etc.).

What was suggested in the email below (freedom from liability for users of the system, etc.) would be out of scope for ONC-ATCB testing based on the given criteria. [I.e., certification criteria - ed.] I hope that helps to answer your question.

I had noted that:

... My question was certainly answered [by the ATCB responses]. ONC certification is not a safety validation, such as in a document from NASA on aerospace software safety certification, "Certification Processes for Safety-Critical and Mission-Critical Aerospace Software" (PDF) which specifies at pg. 6-7:

In order to meet most regulatory guidelines, developers must build a safety case as a means of documenting the safety justification of a system. The safety case is a record of all safety activities associated with a system throughout its life. Items contained in a safety case include the following:

• Description of the system/software
• Evidence of competence of personnel involved in development of safety-critical software and any
safety activity
• Specification of safety requirements
• Results of hazard and risk analysis
• Details of risk reduction techniques employed
• Results of design analysis showing that the system design meets all required safety targets
• Verification and validation strategy
• Results of all verification and validation activities
• Records of safety reviews
• Records of any incidents which occur throughout the life of the system
• Records of all changes to the system and justification of its continued safety

A CCHIT ATCB juror, a physician informatics specialist, has also done a guest post in Jan. 2012 on HC Renewal about the certification process, reproducing his testimony to HHS on the issue.  That post is "Interesting HIT Testimony to HHS Standards Committee, Jan. 11, 2011, by Dr. Monteith."  Dr. Monteith testified (emphases mine):

... I’m “pro-HIT.” For all intents and purposes, I haven’t handwritten a prescription since 1999.

That said and with all due respect to the capable people who have worked hard to try to improve health care through HIT, here’s my frank message:

ONC’s strategy has put the cart before the horse. HIT is not ready for widespread implementation. 

... ONC has promoted HIT as if there are clear evidence-based products and processes supporting widespread HIT implementation.

But what’s clear is that we are experimenting…with lives, privacy and careers.

... I have documented scores of error types with our certified EHR, and literally hundreds of EHR-generated errors, including consistently incorrect diagnoses, ambiguous eRxs, etc.

As a CCHIT Juror, I’ve seen an inadequate process. Don’t get me wrong, the problem is not CCHIT. The problem stems from MU.

EHRs are being certified even though they take 20 minutes to do a simple task that should take about 20 seconds to do in the field.  [Which can contribute to mistakes and "use error" - ed.] Certification is an “open book” test. How can so many do so poorly?

For example, our EHR is certified, even though it cannot generate eRxs from within the EHR, as required by MU.

To CCHIT’s credit, our EHR vendor did not pass certification. Sadly, our vendor went to another certification body, and now they’re certified.

MU does not address many important issues. Usability has received little more than lip-service. What about safety problems and reporting safety problems? What about computer generated alerts, almost all of which are known to be ignored or overridden (usually for good reason)?

 

The concept of “unintended consequences” comes to mind.

All that said, the problem really isn’t MU and its gross shortcomings, it is ONC trying to do the impossible:

ONC is trying to artificially force a cure for cancer, basically trying to promote one into being, when in fact we need to let one evolve through an evidence-based, disciplined process of scientific discovery and the marketplace.

Needless to say, as was learned at great cost in past decades, a "disciplined process" in medicine includes meaningful safety regulation by objective outside experts.

Further, the certifiers have no authority to do important things such as forcibly remove dangerous software from the market.  An example is the forced Class 1 recall of a defective system as I wrote about in my Dec. 2011 post "FDA Recalls Draeger Health IT Device Because This Product May Cause Serious Adverse Health Consequences, Including Death".   Class 1 recalls are the most serious type of recall and involve situations in which there is a reasonable probability that use of these products will cause serious adverse health consequences or death.

In that situation, the producer had been simply advising users (in critical care environments, no less) to "work around the defects" that could indicate incorrect recommended dosage values of critical meds, including a drug dosage up to ten times the indicated dosage, as well as corrupt critical cardiovascular monitoring data.  As I observed:

... I find a software company advising clinicians to make sure to "work around" blatant IT defects in "acute care environments" the height of arrogance and contempt for patient safety.

Without formal regulatory authority to take actions such as this FDA recall, "safeguarding the public" is a meaningless platitude.

It's also likely the ATCB's, which are private businesses, would not want the responsibility of "safeguarding the public."  That responsibility would open them up to litigation when patient injuries or death were caused, or were contributed to, by "certified" health IT.

I have in the past also noted that the use of the term "certification" might have been deliberate, to mislead potential buyers exactly into thinking that "certification" is akin to a UL certification of an electrical appliance for safety, or an FAA approval of a new aircraft's flight-worthiness.

The WSJ needs to clarify and/or retract its statement, as the statement is misinformation.

At my Feb. 2012 post "Health IT Ddulites and Disregard for the Rights of Others" I observed:

Ddulites [HIT hyper-enthusiasts - ed.] ... ignore the downsides (patient harms) of health IT.

This is despite being already aware of, or informed of patient harms, even by reputable sources such as FDA (Internal FDA memo on H-IT risks), The Joint Commission (Sentinel Events Alert on health IT), the NHS (Examples of potential harm presented by health software - Annex A starting at p. 38), and the ECRI Institute (Top ten healthcare technology risks), to name just a few.

In fact, the hyper-enthusiastic health IT technophiles will go out of their way to incorrectly dismiss risk management-valuable case reports as "anecdotes" not worthy of consideration (see "Anecdotes and medicine" essay at this link).

They will also make unsubstantiated, often hysterical-sounding claims that health IT systems are necessary to, or simply will "transform" (into what, exactly, is usually left a mystery) or even "revolutionize" medicine (whatever that means).

Health IT is a potentially dangerous technology.   It requires meaningful regulation to "safeguard the public."  How many incidents like this and this will it take before that is understood by the hyper-enthusiasts?

I've emailed the ATCB's that had responded to my aforementioned query for clarification on the WSJ assertion about their role, being that the statement is in contradiction to their earlier replies to me.  I also advised them of the potential liability issues.

However, if it turns out to be true that the ONC-ATCB's do intend themselves as the ultimate watchdog and assurer of public safety related to EHR's, that needs to be known by the public and their representatives.

-- SS

EHRs and test ordering: Health Affairs authors reply to ONC

At my March 9, 2012 post "Increased Lab Ordering with EHR's?" I refuted ONC's response to a Harvard-based research study "Giving Office-Based Physicians Electronic Access To Patients’ Prior Imaging And Lab Results Did Not Deter Ordering Of Tests" that may contradict the notion that HIT reduces medical costs.

Now the authors themselves have responded to ONC. There are many similar themes in their response. I've bolded them below:

The Effect Of Physicians’ Electronic Access To Tests: A Response To Farzad Mostashari

March 12th, 2012
by Danny McCormick, David Bor, Stephanie Woolhandler, and David Himmelstein

Our recent Health Affairs article linking increased test ordering to electronic access to results has elicited heated responses, including a blog post by Farzad Mostashari, National Coordinator for Health IT. Some of the assertions in his blog post are mistaken. Some take us to task for claims we never made, or for studying only some of the myriad issues relevant to medical computing. And many reflect wishful thinking regarding health IT; an acceptance of deeply flawed evidence of its benefit, and skepticism about solid data that leads to unwelcome conclusions.

Dr. Mostashari’s critique of our paper, will, we hope, open a fruitful dialogue. We trust that in the interest of fairness he will direct readers to our response on his agency’s site. [If not, HC Renewal posts get highly ranked by Google - ed.]

Our study analyzed government survey data on a nationally representative sample of 28,741 patient visits to 1187 office-based physicians. We found that electronic access to computerized imaging results (either the report or the actual image) was associated with a 40% -70% increase in imaging tests, including sharp increases in expensive tests like MRIs and CT scans; the findings for blood tests were similar. Although the survey did not collect data on payments for the tests, it’s hard to imagine how a 40% to 70% increase in testing could fail to increase imaging costs.

Dr. Mostashari’s statement that “reducing test orders is not the way that health IT is meant to reduce costs” is surprising, and contradicts statements by his predecessor as National Coordinator that electronic access to a previous CT scan helped him to avoid ordering a duplicate and “saved a whole bunch of money.” A Rand study, widely cited by health IT advocates including President Obama, estimated that health IT would save $6.6 billion annually on outpatient imaging and lab testing. Another frequently quoted estimate of HIT-based savings projected annual cost reductions of $8.3 billion on imaging and $8.1 billion on lab testing.

We focused on electronic access to results because the common understanding of how health IT might decrease test ordering is that it would facilitate retrieval of previous results, avoiding duplicate tests. Indeed, it’s clear from the extensive press coverage that our study was seen as contravening this “conventional wisdom”.

Nonetheless, Dr. Mostashari criticizes us for analyzing the impact of physicians’ electronic access to imaging and test results, but not other aspects of electronic health record (EHR) use. We did, however, analyze the relationship of EHRs to test ordering in a subsidiary analysis. While physicians use of a full EHR was associated with a 19% increase in image ordering, as we noted in the paper this finding was not statistically significant. While we cautiously (and properly) interpreted this as a “null” finding, these data are inconsistent with Mostashari’s optimistic view that use of a full EHR reduces costs.

He asserts that our 2008 data are passe, and that health IT meeting today’s “meaningful use” criteria definitely saves money. The data we analyzed were the latest available data when we initiated the study. While the proportion of outpatient physicians utilizing health IT has grown since 2008, we are unaware of any “game changing” health IT developments in the past four years that are would produce substantially different results if the study were repeated today. The EHR vendors that dominated the market in 2008 remain, by and large, today’s market leaders, and their products have undergone mostly modest tweaks. Mostashari’s contention that 2012 EHRs – incorporating decision support and electronic information exchange – save money in ways not possible in 2008 should be tested through additional research but remains merely a hypothesis. We hope that some day his predicted savings can be achieved.

Dr. Mostashari offers his own explanation for our findings, suggesting that doctors who are inclined to order more tests are also inclined to purchase health IT for viewing test results electronically rather than on paper. He offers no evidence for this assertion and ignores the fact that we explored (and rejected) this explanation by analyzing subgroups of doctors who are unlikely to be the decision maker for IT purchases – e.g. employed physicians, those working in an HMO setting etc. In other words, electronic access to results predicted more test ordering whether or not the ordering physician was responsible for health IT purchases.

He incorrectly states that our analysis did not take into account patients’ severity of illness, physicians’ level of training, and the nature of physicians’ financial arrangements. In fact, we reported subsidiary multivariate analyses that included several serious diagnoses; all of our models included physician specialty (which we specified in several different ways); and all models included adjustment for an extensive list of indicators of financial arrangements (e.g. whether the physician owned the practice or was an employee; the type of office; whether the practice was owned by a hospital; whether the physician was a solo practitioner; whether the physician’s compensation was based, in part or whole on “profiling”; and whether the practice was predominantly prepaid). We also performed a series of subsidiary analyses that explored whether physicians with a proclivity to “self refer” patients for imaging tests accounted for our finding; they didn’t.

Dr. Mostashari criticizes us for failing to assess whether health IT improved the quality or appropriateness of care. Of course, these were not the topic of our research. Those are different studies for a different time. However, we would note that other large-scale studies have found no, or trivial quality improvements associated with HIT outside of a few flagship institutions^4-6.

Dr. Mostashari’s strongest claim is that observational studies like ours (and most other health policy studies, including some by Dr. Mostashari himself) cannot prove causation. This is surely true. As long time teachers of evidence based medicine we took care to couch our conclusions in cautious terms, stating only that “Computerization, whatever its other benefits, remains unproven as a cost control strategy.”

But Dr. Motashari is less cautious, asserting that the case for HIT is closed. The paper he cites to buttress this claim (authored by members of his own agency) culled studies reporting any impact of HIT on virtually any aspect of care, and accepted authors’ claims of benefit without regard to study quality or statistical niceties. Thus, a focus group’s impressions of benefit are accorded the same weight as nationwide studies of Medicare data showing virtually no impact of computerization on quality measures. Reports of a reduction from 70% to 38 % in “missed billing opportunities” or a $7,000 reduction in office supply costs are among the 92% of studies judged “positive”. While the literature review he cites is interesting, nothing in it contradicts our findings.

Dr. Mostashari is also correct in reiterating that randomized trials are the best way to assess health IT. In fact, no randomized trial has ever been published that examines patients’ outcomes or costs associated with off-the-shelf health IT systems that dominate the U.S. market. No drug or new medical device could pass FDA review based on such thin evidence as we have on health IT. Yet his agency is disbursing $19 billion in federal funds to stimulate the adoption of this inadequately evaluated technology. Dr. Mostashari is perhaps the only person in our nation who commands the resources needed to mount a well done randomized controlled trial to fairly assess the impact of health IT, and the comparative efficacy of the various EHR options.

Finally, Dr. Mostashari’s unbridled faith in technology is mirrored by his belief that ACOs are the next panacea for health costs and quality. That health policy flavor-of-the-month also remains wholly unproven.

I think readers comparing my response to the authors', and who are familiar with the many posts at HC Renewal going back to 2004, will recognize the themes I have bolded in the author's response to HHS.

-- SS

Increased Lab Ordering with EHR's?

ONC has once more proffered typical politically-motivated spin with regard to the Harvard study "Giving Office-Based Physicians Electronic Access To Patients’ Prior Imaging And Lab Results Did Not Deter Ordering Of Tests" that I wrote about at "Another Health IT Mythbuster: Doctors order more X-rays, not fewer, with computer access."

That study had the rather tame, reasonable conclusion:

... These findings raise the possibility that, as currently implemented, electronic access does not decrease test ordering in the office setting and may even increase it, possibly because of system features that are enticements to ordering. We conclude that use of these health information technologies, whatever their other benefits, remains unproven as an effective cost-control strategy with respect to reducing the ordering of unnecessary tests.

The ONC response (posted here at present) to such a mild conclusion about an experimental technology seemed very splenetic.

As I mentioned at the aforementioned post, ONC has thrown good science under the bus before, for political purposes in my view:

... On the other hand, coming from a political office that clearly does not understand how to conduct qualitative research and creates political promotion pieces masquerading as "research", such a statement is not surprising. See "ONC: "The Benefits Of Health Information Technology: A Review Of The Recent Literature Shows Predominantly Positive Results" at this link, where essential research methodologies were thrown under the bus for publication in Health Affairs.

At least the deviations from rigorous research methodologies were admitted:

“... Our findings must be qualified by two important limitations: the question of publication bias [e.g., bias in evidence selection - ed.], and the fact that we implicitly gave equal weight to all studies regardless of study design or sample size.”

Unfortunately, the media, politicians, financial decisionmakers and others are likely not to really comprehend, in-depth, the full significance of that sentence.

I am in the unfortunate situation of again having to stuff ONC's - um, stuff - back into the bull.

First, hat tip to Histalk where I found the link below:

National Coordinator for HIT Farzad Mostashari, MD takes issue with the recently published report that found doctors with online access to patients’ charts ordered more tests. Mostashari disputes the study, which raised questions as to whether or not EHRs cut costs. Mostashari’s contends that the study was based on 2008 data and before the start of the Meaningful Use program and thus does not address certified EHRs’ capabilities for data exchange and clinical data support.

The fatal passage in the linked ONC piece at http://www.healthit.gov/buzz-blog/meaningful-use/study-facts/ is this:

"Also, the study data were from 2008, before the passage of the HITECH Act and the linking of payment incentives to the meaningful use of EHRs."

This seems a variation of the typical excuse-making in IT - "they were using v. 1.0; it's all fixed in the later version."

The outcomes of (Orwellian-named) "Meaningful Use" (MU) have not been studied, to my knowledge. Futher, the criteria chosen for "Meaningful Use" were primarily best guesses as to what could be beneficial. ("Meaningful Use" should have been more accurately termed "good faith use.")

Claiming that 2008 data on EHR-related test ordering is invalid because "Meaningful Use" was not in effect at the time is, in fact, jumping to an unsupported conclusion that "Meaningful Use" will counter whatever multiple medical/social factors caused the increased ordering in the first place -- because, of course, MU is "Meaningful" and deterministically guaranteed to work out, nationally, as planned, among all outpatient and inpatient settings.

This seems a form of "begging the question":

Begging the Question is a fallacy in which the premises include the claim that the conclusion is true or (directly or indirectly) assume that the conclusion is true. This sort of "reasoning" typically has the following form.

- Premises in which the truth of the conclusion is claimed or the truth of the conclusion is assumed (either directly or indirectly).
- Claim C (the conclusion) is true.

This sort of "reasoning" is fallacious because simply assuming that the conclusion is true (directly or indirectly) in the premises does not constitute evidence for that conclusion. Obviously, simply assuming a claim is true does not serve as evidence for that claim. This is especially clear in particularly blatant cases: "X is true. The evidence for this claim is that X is true."

I note that the MU criteria are themselves evolving and not finalized. Making predictions about the future is the domain of fortune tellers with crystal balls, not scientists:

Health IT WILL reduce costs and improve caregivers’ decisions and patients’ outcomes. It is written in the stars!

ONC seems to think it is capable of such certainty, as I wrote in mid-2010 at "Science or Politics? The New England Journal and "The 'Meaningful Use' Regulation for Electronic Health Records":

... The widespread use of electronic health records (EHRs) in the United States is inevitable. EHRs will improve caregivers’ decisions and patients’ outcomes. Once patients experience the benefits of this technology, they will demand nothing less from their providers. Hundreds of thousands of physicians have already seen these benefits in their clinical practice.

In its current piece, ONC goes on to state:

Reducing Test Orders Is Not the Way that Health IT Is Meant to Reduce Costs

The ultimate impact of EHRs on reducing cost will be through improvements in the coordination and quality of care, and the prevention of unnecessary and costly complications and hospitalizations. [Note the mysterious disappearance of the word "tests" - ed.] Providers who are embracing new delivery and payment models such as Accountable Care Organizations and Patient-Centered Medical Homes know that meaningful use of EHRs is a critical foundation for being able to improve quality while reducing cost.

That is, simply, a lie. Reducing test utilization has long been claimed as a benefit of EHR's.

For example, from http://www.healthit.gov/patients-families/benefits-health-it:

EHRs reduce unnecessary tests and procedures. Have you ever had to repeat medical tests ordered by one doctor because the results weren’t readily available to another doctor? Those tests may have been uncomfortable and inconvenient or have posed some risk, and they also cost money. Repeating tests—whether a $20 blood test or a $2,000 MRI--results in higher costs to you in the form of bigger bills and increased insurance premiums. With EHRs, all of your care providers can have access to all your test results and records at once, reducing the potential for unnecessary repeat tests.

It is also another example of "moving the goalposts", a defense often used by those without a sound argument; by politicians; and sometimes by - scoundrels.

(I observed another "moving of the goalposts" at my aforementioned post "Another Health IT Mythbuster: Doctors order more X-rays, not fewer, with computer access." In that post I observed Michael Furukawa, a health economist in the ONC office, stating that the researchers’ focus was not "deep enough" to support the study’s conclusions. He wrote: “The data are sound, the methods are appropriate, but the focus is limited,” he said. “They only looked at one piece of health IT.”)

Well, yes, and the piece they looked at should set off red flags that the assumptions about health IT and savings might be erroneous - not generate excuses that the goalposts were too close, and need to be moved further away.

This is not to mention that it appears most healthcare errors have little to do with documentation, as I outlined in my Dec. 2010 post "Is Healthcare IT a Solution to the Wrong Problem?"

I particularly take issue with the ONC statement that:

... this was not a randomized trial, but an observational study (the National Ambulatory Medical Survey) that was not designed to answer the question of cost, or associations between EHRs and quality. As a result, many other variables that could affect physician behavior could not be examined in this study

Yet in the aforementioned ONC "predominantly positive results" literature survey described at this link, ONC "implicitly gave equal weight to all studies regardless of study design or sample size." Their review thus included qualitative studies that were probably not meant to be evaluative, and observational studies subject to severe methodological bias, yet all were weighted equally. Further, studies contradicting the 'narrative' of health IT efficacy and beneficence such as in this list appear to have been excluded.

This appears a prime example of, at best, "the pot calling the teakettle black", and raises doubts as to ONC's objectivity and perhaps even intellectual honesty. Of course, they are a political office with a mission, so perhaps this can be understood. (Not excused, but understood.)

ONC also views risk management-critical reports of health IT-caused harm as "anecdotal", a scientific and ethical faux pas (or is it willful blindness?) of major proportions.

As I wrote at my April 2001 post "Making a Stat Less Significant: Common Sense on "Side Effects" Lacking in Healthcare IT Sector":

... This view [of negative HIT reports being 'anecdotes'] extends all the way up to the Director of the Office of the National Coordinator for Health IT, who glibly stated per the Aug. 2010 Huffington Post Investigative Fund article FDA, Obama Digital Medical Records Team at Odds over Safety Oversight that FDA's own reports of health IT related injuries and deaths were “anecdotal":

ONC director Blumenthal, the point man for the administration, has called the FDA’s injury findings “anecdotal and fragmentary.” He told the Investigative Fund that he believed nothing in the report indicated a need for regulation.

Those "injury findings" appear in an FDA Internal Memo made available by the aforementioned Huffington Post Investigative Fund and archived at the following link:

Internal FDA memorandum on HIT risks (PDF) to Jeffrey Shuren MD JD (Director, Center for Devices and Radiological Health). Health Information Technology (H-IT) Safety Issues. "This is an Internal Document Not Intended for Public Use." Feb. 23, 2010.

(My description/summary of the memorandum is at my Aug. 2010 post "Internal FDA memorandum of Feb. 23, 2010 to Jeffrey Shuren on HIT risks. Smoking gun?")

The definitive take-down of the "anecdote" canard is at this link.

One could wonder if a criteria for work at ONC is a Ddulite disposition ('Luddite' with first four characters reversed):

Ddulites: Hyper-enthusiastic technophiles who either deliberately ignore or are blinded to technology's downsides, ethical issues, and repeated local and mass failures.

Instead of logical fallacy and "spin", perhaps the ONC would be better served by sponsoring some (independent, objective) researchers to actually conduct research supporting their claims about the effects of the "Meaningful Use" program.

Finally, perhaps ONC should ask my mother what she thinks about EHRs. She has specially-acquired firsthand expertise. They can find her here.

-- SS

Addendum 3/9/12:

At my Feb. 2011 post "Does EHR-Incited Upcoding (Also Known as "Fraud") Need Investigation by CMS, And Could it Explain HIT Irrational Exuberance " is another side effect on costs of EHR's in inpatient settings, specifically, the ED.

-- SS

Addendum 3/11/12:

Another article in 2010 by Himmelstein, Woolhandler & Wright had reached related conclusions:

Hospital Computing and the Costs and Quality of Care: A National Study

David U. Himmelstein, MD, Adam Wright, PhD,Steffie Woolhandler, MD, MPH

The American Journal of Medicine Volume 123, Issue 1 , Pages 40-46, January 2010

Background

Many believe that computerization will improve health care quality, reduce costs, and increase administrative efficiency. However, no previous studies have examined computerization's cost and quality impacts at a diverse national sample of hospitals.

Methods

We linked data from an annual survey of computerization at approximately 4000 hospitals for the period from 2003 to 2007 with administrative cost data from Medicare Cost Reports and cost and quality data from the 2008 Dartmouth Health Atlas. We calculated an overall computerization score and 3 subscores based on 24 individual computer applications, including the use of computerized practitioner order entry and electronic medical records. We analyzed whether more computerized hospitals had lower costs of care or administration, or better quality. We also compared hospitals included on a list of the “100 Most Wired” with others.

Results

More computerized hospitals had higher total costs in bivariate analyses (r=0.06, P=.001) but not multivariate analyses (P=.69). Neither overall computerization scores nor subscores were consistently related to administrative costs, but hospitals that increased computerization faster had more rapid administrative cost increases (P=.0001). Higher overall computerization scores correlated weakly with better quality scores for acute myocardial infarction (r=0.07, P=.003), but not for heart failure, pneumonia, or the 3 conditions combined. In multivariate analyses, more computerized hospitals had slightly better quality. Hospitals on the “Most Wired” list performed no better than others on quality, costs, or administrative costs.

Conclusion

As currently implemented, hospital computing might modestly improve process measures of quality but does not reduce administrative or overall costs.

-- SS

Another Health IT Mythbuster: Doctors order more X-rays, not fewer, with computer access

This piece is in the Washington Post about a just-released piece in Health Affairs (at this link) by Danny McCormick, David H. Bor, Stephanie Woolhandler and David U. Himmelstein of Harvard Medical School.

(Woolhandler and Himmelstein have written on such topics before, such as in the Nov. 2009 article “Hospital Computing and the Costs and Quality of Care: A National Study”, Amer J Med 123:1; 40-46.)

In the abstract of the new article, the researchers note:

Giving Office-Based Physicians Electronic Access To Patients’ Prior Imaging And Lab Results Did Not Deter Ordering Of Tests

Health Aff March 2012 vol. 31 no. 3 488-496

Abstract

Policy-based incentives for health care providers to adopt health information technology are predicated on the assumption that, among other things, electronic access to patient test results and medical records will reduce diagnostic testing and save money.

To test the generalizability of findings that support this assumption, we analyzed the records of 28,741 patient visits to a nationally representative sample of 1,187 office-based physicians in 2008. Physicians’ access to computerized imaging results (sometimes, but not necessarily, through an electronic health record) was associated with a 40–70 percent greater likelihood of an imaging test being ordered. The electronic availability of lab test results was also associated with ordering of additional blood tests.

The availability of an electronic health record in itself had no apparent impact on ordering; the electronic access to test results appears to have been the key. These findings raise the possibility that, as currently implemented, electronic access does not decrease test ordering in the office setting and may even increase it, possibly because of system features that are enticements to ordering. We conclude that use of these health information technologies, whatever their other benefits, remains unproven as an effective cost-control strategy with respect to reducing the ordering of unnecessary tests.

The WaPo offered these observations in their article:

Doctors order more X-rays, not fewer, with computer access

By Lena H. Sun, Published: March 5

In the debate over the high cost of health care, federal policymakers have always claimed that one way to cut costs is for doctors to use electronic medical records and other information technology. Doing so, they say, avoids duplication and saves money.

But new research suggests that may not be the case.

Doctors who have easy computer access to results of X-rays, CT scans and MRIs are 40 to 70 percent more likely to order those kinds of tests than doctors without electronic access, according to a study to be published in the March issue of the journal Health Affairs.

This is not surprising to me. Click a button, order an test, click a button, get the results. What could be easier?

What the pundits failed (and continually fail) to take into account is the social context of health IT. It is not used on a robotic assembly line. There are incentives, both proper and perverse (e.g., "defensive medicine", profits, patient demands) that figure into how any IT will "play out" in healthcare.

Making the blanket statement that "IT in healthcare will save money" without solid, sustained evidence (as opposed to the anecdotal evidence that now exists), while simultaneously ignoring contrary evidence is simply promoting an industry-created meme, a.k.a. wishful thinking.

“On average, this is comparing doctors who had electronic medical records and those who didn’t,” said lead author Danny McCormick, a physician and assistant professor of medicine at Harvard Medical School.

Researchers say the findings challenge a key premise of the nation’s multibillion-dollar effort to promote the widespread adoption of health information technology.

“This should give pause to those making the argument,” McCormick said. Instead of saving money, that effort could drive costs higher, he said.

For the study, researchers at the Cambridge Health Alliance, a health system in Cambridge, Mass., and the City University of New York analyzed data from a 2008 federal government survey. The data included information collected from 28,741 patient visits to 1,187 office-based doctors. The information included the type of doctor, their office computerization and the tests ordered at each visit. About half of the doctors’ offices surveyed had computerized access to results of X-rays and other imaging tests.

Researchers found that doctors who did not have computerized access ordered imaging tests in 12.9 percent of visits, while doctors with electronic access ordered imaging in 18 percent of visits, a 40 percent greater likelihood. Doctors with computerized access were even more likely — about 70 percent more likely — to order advanced imaging tests, such as PET scans, which experts said are most commonly used to detect cancer, heart problems, brain disorders and other central nervous system disorders.

The study found the results hold true even after taking into account other factors, such as patient demographics, doctor specialty and physician self-referral.

Unfortunately, nothing up to and including HIT-related patient deaths will give pause to health IT pundits pushing national rollout. They have too much skin in the game in the health IT bubble, psychologically (link) and financially.

The pundits need to be opposed by medical and legal professionals in an organized, systematic fashion until such time as this industry follows the National Academies' direct recommendations on getting health IT "right."

... Researchers were not able to determine why physicians ordered the imaging tests, or whether in those cases, physicians had looked at patients’ prior chest X-rays. Nor were they able to assess whether the increased imaging helped or harmed patients.

That needs to be followed up upon, obviously. Will it? Probably not.

At some flagship hospitals with customized health information technology systems, for example, doctors get specific feedback about diagnostic tests that can result in ordering fewer tests.

Note the word "can." (After decades there is no rigorous, unopposed proof, yet we are embarking on a national rollout with penalties for non-adopters?) This rationalization is both the "it will be better in version 2.0" and wishful thinking combined.

Michael Furukawa, a health economist in the Office of the National Coordinator for Health Information Technology, the administration’s health IT czar, said the researchers’ focus was not deep enough to support the study’s conclusions.

“The data are sound, the methods are appropriate, but the focus is limited,” he said. “They only looked at one piece of health IT.”

"But the focus is limited?"

So what?

That was all they intended to look at, and the result was concerning. Moving the goalposts is not a valid refutation of research.

On the other hand, coming from a political office that clearly does not understand how to conduct qualitative research and creates political promotion pieces masquerading as "research", such a statement is not surprising. See "ONC: "The Benefits Of Health Information Technology: A Review Of The Recent Literature Shows Predominantly Positive Results" at this link, where essential research methodologies were thrown under the bus for publication in Health Affairs.

At least the deviations from rigorous research methodologies were admitted:

“... Our findings must be qualified by two important limitations: the question of publication bias [e.g., bias in evidence selection - ed.], and the fact that we implicitly gave equal weight to all studies regardless of study design or sample size.”

Unfortunately, the media, politicians, financial decisionmakers and others are likely not to really comprehend, in-depth, the full significance of that sentence.

I can only imagine the reaction to such an excuse made by pharma for the mainstreaming of a risk-prone drug.

I personally take with the greatest of skepticism anything coming from ONC, an increasingly politicized government office in my observation.

You should, too, in my opinion.

-- SS

Is ONC Stonewalling on the issue of HIT Certification, Safety and Liability?

At my Feb. 16, 2012 post "Hospitals and Doctors Use Health IT at Their Own Risk - Even if Certified" I wrote that an ONC-ATCB (Authorized Testing and Certification Body) replied to my email inquiry about health IT certification, safety and liability indemnification by stating that:

What was suggested in the email below (freedom from liability for users of the system, etc.) would be out of scope for ONC-ATCB testing based on the given criteria.

[That is, the criteria used in testing here - ed.]

What I did not include in that post was the fact that some months ago, I had emailed ONC directly with the same questions, and then called them on the phone with those questions at about the same time as I inquired of the ATCB.

ONC itself never responded.

There are several possibilities:

• They don't know the answer.
• They don't want to respond.
• They don't care to respond.

Dismissing possibility #1, these civil servants appear to be stonewalling on the issue.

It would be nice to hear ONC itself admit the term "certification" is a gossamer guarantee of health IT safety, efficacy and indemnification of purchasers, implementers and users from potential EHR-related liability.

I am not holding my breath.

-- SS

Addendum:

An ONC representative did get back to me on Feb. 27, but I told them my question had already been answered by ONC ATCB's.

Generic Leadership Fine for EHR Implementation in High Acuity, High Risk Areas Such as an Emergency Department?

At my Feb. 16, 2012 post "Hospitals and Doctors Use Health IT at Their Own Risk - Even if 'Certified'" I established that buyers and users of EHR systems do so at their own risk with regard to implementation problems or outright defects resulting in patient harm.

As I've also written at my April 2011 post "FDA Decides Regulating Implantable Defibrillator Medical Devices a Political Hot Potato; Demurs", FDA has admitted health IT is a medical device that they shy away from regulating because it's politically expedient not to do so. (The area is a "hot potato" per the Director of FDA's CDRH, the Center for Devices and Radiological Health, so FDA stays away.)

Yet FDA knows of the stakes. As I observed at my Aug. 2010 post "Internal FDA memorandum of Feb. 23, 2010 to Jeffrey Shuren on HIT risks" and elsewhere, patient harm is occurring due to these devices and FDA is aware. Due to admitted impediments to information diffusion, however, even FDA, IOM, The Joint Commission (due to a "dearth of data" as in their 2008 Sentinel Events Alert On Healthcare IT) and others do not know the true magnitude of the problem.

From the FDA internal memo at the preceding FDA memorandum link:

Limitations of the MAUDE [Manufacturer and User Facility Device Experience database] search and final subset of Medical Device Reports (MDRs) include the following:

1. Not all H-IT safety issue MDRs can be captured due to limitations of reporting practices including

... (a) Vast number of H-IT systems that interface with multiple medical devices currently assigned to multiple procodes making it difficult to identify specific procodes for H-IT safety issues;
... (b) Procode assignments are also affected by the ability of the reporter/contractor to correctly identify the event as a H-IT safety issue;
... (c) Correct identification by the reporter of the suspect device brand name is challenged by difficulties discerning the actual H-IT system versus the device it supports.

2. Due to incomplete information in the MDRs, it is difficult to unduplicate similar reports, potentially resulting in a higher number of reports than actual events.

3. Reported death and injury events may only be associated with the reported device but not necessarily attributed to the device.

4 Correct identification by the reporter of the manufacturer name is convoluted by the inability to discern the manufacturer of the actual H-IT system versus the device it supports.

5 The volume of MDR reporting to MAUDE may be impacted by a lack of understanding the reportability of H-IT safety issues and enforcement of such reporting.

... The results of this data review suggest significant clinical implications and public safety issues surrounding Health Information Technology. The most commonly reported H-IT safety issues included wrong patient/wrong data, medication administration issues, clinical data loss/miscalculation, and unforeseen software design issues; all of which have varying impact on the patient’s clinical care and outcome, which included 6 death and 43 injuries. The absence of mandatory reporting enforcement of H-IT safety issues limits the number of relevant MDRs and impedes a more comprehensive understanding of the actual problems and implications.

Finally, as I pointed out in my Dec. 2009 post "ONC Defines a Taxonomy of Robust Healthcare IT Leadership", the Office of the National Coordinator for Health IT at HHS (ONC) had published the following with regard to the qualification of those who could "lead the successful deployment and use of health IT to achieve transformational improvement in the quality, safety, outcomes, and thus in the value, of health services in the United States":

Clinician/Public Health Leader: By combining formal clinical or public health training with training in health IT, individuals in this role will be able to lead the successful deployment and use of health IT to achieve transformational improvement in the quality, safety, outcomes, and thus in the value, of health services in the United States. In the health care provider settings, this role may be currently expressed through job titles such as Chief Medical Information Officer (CMIO), Chief Nursing Informatics Officer (CNIO). In public health agencies, this role may be currently expressed through job titles such as Chief Information or Chief Informatics Officer. Training appropriate to this role will require at least one year of study leading to a university-issued certificate or master’s degree in health informatics or health IT, as a complement to the individual’s prior clinical or public health academic training. For this role, the entering trainees may be physicians or other clinical professionals (e.g. advanced-practice nurses, physician assistants) or hold a master’s or doctoral degree(s) in public health or related health field. Individuals could also enter this training while enrolled in programs leading directly to degrees qualifying them to practice as physicians or other clinical professionals, or to master’s or doctoral degrees in public health or related fields (such as epidemiology). Thus, individuals could be supported for training if they already hold or if they are currently enrolled in courses of study leading to physician, other clinical professional, or public-health professional degrees.

Here, though, is a more typical reality in today's hospitals, the background of an EHR deployment leader in one of the most risk-prone and difficult environments in healthcare, the Emergency Department (ED) in a large hospital of which I am aware:

Clinical Systems Analyst

January 2004 – Present

Project manager for implementation of an ED EHR [vendor name redacted]
Responsible for all aspects of implementation including build and process design, report build, testing, training, and go-live support.
Extensive knowledge ADT processes, HL7, and CPOE.
Manage software patches and upgrades.
Experience in HTML, SQL and SAP.

Financial Systems Analyst
November 2001 – January 2004

Project Team Lead for implementation of Patient Accounting system.
Responsible for system development and system build, training, testing, and reporting.

Education

[Regional university in the lower 50% of US News rankings in the region, name redacted]
B.S., Management Information Systems
2000 – 2003

[Name redacted] Community College

Associates Degree, Business Administration

So, ED physicians (with many years of doctoral and post-doctoral training and experience, boards testing their competence, etc.) may be placing the lives of patients and their careers in the hands of:

• An experimental medical device,
  
• unregulated by anyone,
  
• in a field whose qualifications for leadership are unregulated by anyone,
  
• devices known to be risky,
  
• but without anyone knowing the magnitude of that risk,
  
• whose implementation is led by a generic 'clinical' systems analyst "responsible for all aspects of implementation" with a Bachelor's degree in MIS from a second-tier institution, an Associates degree from a community college, and no medical or Medical Informatics education or experience whatsoever.

Further, that project lead jumped from financial systems analyst to 'clinical' analyst/leader of an ED EHR implementation.

One should also ask

• Who was the hiring manager who hired such a person and put them in the role?
• On what credentials and experience was the decision based?
  
• What were the hiring manager's own credentials?
  

I leave it to the reader to decide if this is an appropriate arrangement, and if they would place their trust in an ED whose activities center around a cybernetic system implemented in this manner.

-- SS

Feb. 17, 2012 Addendum:

At least in this case the person has degrees.

Per several prominent healthcare IT recruiters in the past in the article "Who's Growing CIO's" in the journal Healthcare Informatics, as mentioned at my Sept. 2009 post "Correcting historical information from the recruiter component of the Health IT Ecosystem", the School of Hard Knocks was plenty good for health IT leadership roles:

I don't think a degree gets you anything," says healthcare recruiter Lion Goodman, president of the Goodman Group in San Rafael, California about CIO's and other healthcare MIS staffers. Healthcare MIS recruiter Betsy Hersher of Hersher Associates, Northbrook, Illinois, agreed, stating "There's nothing like the school of hard knocks." In seeking out CIO talent, recruiter Lion Goodman "doesn't think clinical experience yields [hospital] IT people who have broad enough perspective. Physicians in particular make poor choices for CIOs. They don't think of the business issues at hand because they're consumed with patient care issues," according to Goodman.

-- SS

Hospitals and Doctors Use Health IT at Their Own Risk - Even if "Certified"

Due to my observations of confusion about health IT certification [1], and due to vague or incomplete seller language that could be misinterpreted by buyers (perhaps by design), I recently asked several ONC-ATCBs (HHS's Office of the National Coordinator for Health IT-Authorized Testing and Certification Bodies) the following.

I sent this question via email to their "questions" email addresses:

"Is EHR certification by an ATCB a certification of EHR safety, effectiveness, and a legal indemnification, i.e., certifying freedom from liability for EHR use of clinical users or organizations? Or does it signify less than that?"

One ONC-ATCB provided the following in response to my request for information.

From: Trivedi, Amit V (ICSA Labs)
Sent: Thursday, February 16, 2012 11:22 AM
To: Scot Silverstein
Subject: RE: Form submission from: Contact Us

Hello Scot,

Thanks for your email. Certification by an ATCB signifies that the product or system tested has the capabilities to meet specific criteria published by NIST and approved by the Office of the National Coordinator. In this case the criteria are designed to support providers and hospitals achieve "Meaningful Use." A subset of the criteria deal with the security and patient privacy capabilities of the system.

Here is a list of the specific criteria involved in our testing:
http://healthcare.nist.gov/use_testing/effective_requirements.html

In a nutshell, ONC-ATCB Certification deals with testing the capabilities of a system, some of them relate to patient safety, privacy and security functions (audit logging, encryption, emergency access, etc.).

What was suggested in the email below (freedom from liability for users of the system, etc.) would be out of scope for ONC-ATCB testing based on the given criteria. [I.e., certification criteria - ed.] I hope that helps to answer your question.

Thanks,

Amit

Amit Trivedi
Program Manager - Healthcare
ICSA Labs, an Independent Division of Verizon Business

My question was certainly answered. ONC certification is not a safety validation, such as in a document from NASA on aerospace software safety certification, "Certification Processes for Safety-Critical and Mission-Critical Aerospace Software" (PDF) which specifies at pg. 6-7:

In order to meet most regulatory guidelines, developers must build a safety case as a means of documenting the safety justification of a system. The safety case is a record of all safety activities associated with a system throughout its life. Items contained in a safety case include the following:

• Description of the system/software
• Evidence of competence of personnel involved in development of safety-critical software and any
safety activity
• Specification of safety requirements
• Results of hazard and risk analysis
• Details of risk reduction techniques employed
• Results of design analysis showing that the system design meets all required safety targets
• Verification and validation strategy
• Results of all verification and validation activities
• Records of safety reviews
• Records of any incidents which occur throughout the life of the system
• Records of all changes to the system and justification of its continued safety

Health IT testing conspicuously lacks attention to most of the aerospace software safety points above. I note that there appears to be no reasonable excuse for such omissions.

IOM has recently studied the issue of HIT safety. IOM states in a Nov. 2011 report that HIT safety and safety testing is unsatisfactory, and has recommended HHS study it as well. IOM recommends HHS annually re-evaluate whether regulation is needed to improve safety, although IOM favors industry self-policing [2].

Thus, buyers and users of even "ONC certified" health IT are not indemnified from liability due to medical errors or problems caused by the health IT.

Sellers who exaggerate the value of certification or imply its meaning is akin to FDA device approval, likewise, could be faulted for making false representations about their products.

It would appear the sellers could potentially be sued for doing so by purchasers/users who themselves get into legal hot water due to EHR defects or other problems.

-- SS

Note:

[1] I believe confusion about EHR "certification" is in part due to the term itself. I raised objections to this term when it was first proposed based on my experience in pharma, suggesting what I felt was the more accurate expression "features qualification" instead.

[2] "Health IT and Patient Safety: Building Safer Systems for Better Care", Institute of Medicine of the National Academies, Nov. 2011, http://www.iom.edu/Reports/2011/Health-IT-and-Patient-Safety-Building-Safer-Systems-for-Better-Care.aspx

-----------------

Addendum March 6, 2012:

I received a response from another ONC-ATCB, the Drummond Group:

From: Joani Hughes (Drummond Group)
Sent: Monday, March 05, 2012 1:06 PM
To: Scot Silverstein
Subject: RE: EHR certification question

Per our testing team:

It is less than that. It does not address indemnification although a certification could be used as a conditional part of some other form of indemnification function, such as a waiver or TOA, but that is ultimately out of the scope of the certification itself. Certification in this sense is an assurance that the EHR functions in way that could enable an eligible provider or eligible hospital to meet the CMS requirements of Meaningful Use Stage 1. Or to restate it more directly, CMS is expecting eligible providers or eligible hospitals to use their EHR in “meaningful way” quantified by various quantitative measure metrics and eligible providers or eligible hospitals can only be assured they can do this if they obtain a certified EHR technology.

Please let me know if you have any questions.

Thank you,
Joani.

Joani Hughes
Client Services Coordinator
Drummond Group Inc.

These are direct and clear statements.

-- SS