NY Times/Steve Lohr asks "Why the Economic Payoff From Technology Is So Elusive." The answer in medicine is obvious.

In a June 5, 2016 article, New York Times reporter Steve Lohr (http://topics.nytimes.com/top/reference/timestopics/people/l/steve_lohr/index.html), who reports on technology, business and economics, asked the following question:

Why the Economic Payoff From Technology Is So Elusive
New York Times, Business Day
By STEVE LOHR
JUNE 5, 2016
http://www.nytimes.com/2016/06/06/business/why-the-economic-payoff-from-technology-is-so-elusive.html

Your smartphone allows you to get almost instantaneous answers to the most obscure questions. It also allows you to waste hours scrolling through Facebook or looking for the latest deals on Amazon.  More powerful computing systems can predict the weather better than any meteorologist or beat human champions in complex board games like chess.

But for several years, economists have asked why all that technical wizardry seems to be having so little impact on the economy. The issue surfaced again recently, when the government reported disappointingly slow growth and continuing stagnation in productivity. The rate of productivity growth from 2011 to 2015 was the slowest since the five-year period ending in 1982.

Healthcare becomes the gravamen of the article:

One place to look at this disconnect is in the doctor’s office. Dr. Peter Sutherland, a family physician in Tennessee, made the shift to computerized patient records from paper in the last few years. There are benefits to using electronic health records, Dr. Sutherland says, but grappling with the software and new reporting requirements has slowed him down. He sees fewer patients, and his income has slipped.

Unfortunately, the advisors who helped him with the article may have provided incomplete information:

... “The government funding has made a huge difference,” said Dr. Ashish Jha, a professor at the Harvard School of Public Health. “But we’re seeing little evidence so far that all this technology has had much effect on quality and costs.”

In the face of, among many others, a stunning letter from 40 medical societies to HHS in 2015 that the technology is unfit for purpose (http://mb.cision.com/Public/373/9710840/9053557230dbb768.pdf), known and hair-raising defects (http://hcrenewal.blogspot.com/2011/01/maude-and-hit-risk-mother-mary-what-in.html), and many other complaints from physicians and nurses (e.g., http://hcrenewal.blogspot.com/2013/11/another-survey-on-ehrs-affinity-medical.html, http://hcrenewal.blogspot.com/2013/07/rns-say-sutters-new-electronic-system.html, query link http://hcrenewal.blogspot.com/search/label/glitch as just a few examples), such a statement is anserine.

Why would anyone expect (good) effects on "quality and costs" of healthcare when the technology is so unfit for purpose in design and implementation that it has alienated most of its users?  

I've written previously about Jha's views in a May 27,  2009 post "Harvard's EMR Justification: We Just Have To Do Something" (http://hcrenewal.blogspot.com/2009/05/harvards-emr-justification-we-just-have_27.html):

 ... "I'm not suggesting EHR is going to be a panacea, but the one thing that is absolutely true is there is nothing else out there now that has any more political appeal," Jha says. "Everybody agrees, whether you are a conservative, moderate, or liberal, that we have to do something about healthcare. So the one place where we can all come to agreement is we have to do something about electronic records."

I do not think "political appeal" is a good justification for a multi-billion dollar cybernetic experiment in medicine, where the risks of the technology are considerable and where basic healthcare needs are not being well met among the poor and underprivileged.

Former ONC Chair David Brailer is quoted:

“People confuse information automation with creating the kind of work environment where productivity and creativity can flourish,” said Dr. David J. Brailer, who was the national health technology coordinator in the George W. Bush administration. “And so little has gone into changing work so far.”

Brailer was little better than Jha, and moves the goalposts with a type of circular logic.  He appears to be saying that technology that will revolutionize medicine can't work until we change how things are done in medicine so the technology can revolutionize medicine. 

The article then quotes one Tennessee physician, a Dr. Sutherland, who is "happy" to accept bad health IT, a resultant pay cut, and increased work:

... Today, Dr. Sutherland’s personal income and the medical group’s revenue are about 8 percent below where they were four years ago. But in 2015, both his earnings and the revenue of Healthstar, which employs 350 people in 10 clinics, increased slightly, by nearly 3 percent from 2014.

... Dr. Sutherland bemoans the countless data fields he must fill in to comply with government-mandated reporting rules, and he concedes that some of his colleagues hate using digital records. Yet Dr. Sutherland is no hater. Despite the extra work the new technology has created and even though it has not yet had the expected financial payoff, he thinks it has helped him provide better information to patients.

He values being able to tap the screen to look up potentially harmful drug interactions and to teach patients during visits. He can, for example, quickly create charts to show diabetes patients how they are progressing with treatment plans, managing blood glucose levels and weight loss.

He is working harder, Dr. Sutherland says, but he believes he is a better doctor. Blunt measures of productivity, he added, aren’t everything. “My patients are better served,” he said. “And I’m happier.”

While being able to provide fancy charts and check drug-drug interactions (for which a massive and expensive EHR is certainly not needed; a PDA will suffice) is fine.

However, anyone who gladly accepts a pay cut, and inconvenience, and harder work due to bad health IT, and is a happy camper with that state of affairs, either suffers from the Stockholm syndrome or has a lot of discretionary income and free time to spare that many clinicians do not.  

The article fails to mention the hundreds of thousands of other US docs and others in other lands (e.g., http://hcrenewal.blogspot.com/2016/05/hit-mayhem-canadian-style-nanaimo.html) who aren't happy at all with health IT as it is today.  

---------------------------------------

I sent this email to Mr. Lohr.

From: S Silverstein
To:Steve Lohr 
Date: Tue, Jun 7, 2016 at 10:02 AM
Subject: Re: Why the Economic Payoff From Technology Is So Elusive

Dear Mr. Lohr,

In medicine, the answer to this question is straightforward.  I don't know if Ashish Jha brought this to your attention, or if he himself is aware of it.

This letter from nearly 40 different medical societies to HHS about bad health IT is specific about how bad the current health IT is:

http://mb.cision.com/Public/373/9710840/9053557230dbb768.pdf

You should be aware of the letter's contents.  I've also attached it to this email.

In academic Medical Informatics, such matters are often ignored, as they run contrary to the narrative that IT will "revolutionize medicine"; I know, as I was Yale faculty in Medical Informatics myself. 

The assumption in academic circles and in the Administration (unfortunately) is that "all health IT is good health IT." 

Unfortunately, it is not.  From my own site "Contemporary Issues in Medical Informatics: Good Health IT, Bad Health IT, and Common Examples of Healthcare IT Difficulties" at http://cci.drexel.edu/faculty/ssilverstein/cases/ :

Definitions authored by myself and Australian informatics expert Dr. Jon Patrick:

Good Health IT ("GHIT") is defined as IT that provides a good user experience, enhances cognitive function, puts essential information as effortlessly as possible into the physician’s hands, can be easily, substantively and cost-effectively customized to the needs of medical specialists and subspecialists, keeps eHealth information secure, protects patient privacy and facilitates better practice of medicine and better outcomes. 

Bad Health IT ("BHIT") is defined as IT that is ill-suited to purpose, hard to use, unreliable, loses data or provides incorrect data, is difficult and/or prohibitively expensive to customize to the needs of different medical specialists and subspecialists, causes cognitive overload, slows rather than facilitates users, lacks appropriate alerts, creates the need for hypervigilance (i.e., towards avoiding IT-related mishaps) that increases stress, is lacking in security, is lacking in evidentiary soundness, compromises patient privacy or otherwise demonstrates suboptimal design and/or implementation. 

It comes as no surprise not to find productivity gains, but instead hundreds of thousands of angry physicians (and nurses), when health IT is mostly bad IT.

The health IT industry itself needs serious remediation before its products will be a boon to medicine.

Sincerely,

Scot Silverstein, MD

Drexel University, Philadelphia

http://drexel.edu/now/experts/Overview/silverstein-scot/

p.s. I have not even broached the matter of health IT patient harms. 

Patients are being harmed and dying of bad health IT.  See for instance the CRICO insurance report at http://www.cci.drexel.edu/faculty/ssilverstein/PSQH_MalpractClaimsAnalyConfirRisksEHR.pdf

 

---------------------------------------  

I will add an addendum if I receive a reply.

-- SS

Clinical Practice Guidelines: Still Conflicted After All These Years

Background: Untrustworthy Clinical Practice Guidelines

Since the 1990s, clinicians have been exhorted to follow clinical practice guidelines (CPGs) to improve their decision-making and patients' outcomes.  When Dr Wally Smith and I started teaching a short course (often at the Society for Medical Decision Making meetings) about changing physician behavior, we naively set out to improve decisions by increasing adherence to such guidelines.  We first thought that physicians' apparent shortcomings in guideline adherence were due to their lack of knowledge of the guidelines and the underlying evidence, and to their human cognitive limitations.

That approach, however, failed to yield ways to improve adherence.  After a while, it occurred to us that there might be other reasons physicians did not follow guidelines, including their lack of trust of the guidelines or the evidence supposedly incorporated within them.  We learned that integrity of  the clinical evidence base has been severely degraded due to the manipulation of clinical studies by those with vested interests, and the outright suppression of trials for which manipulation does not produce the desired results.

Furthermore, as we became less naive, we learned that the integrity of the guideline development process was also suspect, again due to the influence of those with vested interest.  No wonder physicians were suspicious of guidelines, and resisted adhering to them.

In 2011, the prestigious Institute of Medicine released a report on the development of  better standards to produce more trustworthy guidelines (Clinical Practice Guidelines We Can Trust.  Link here.)  We posted about that report here, but noted that it was receiving little other attention, an example of the anechoic effect.  The IOM report cited conflicts of interest (COIs) affecting the guideline development process as a major reason such guidelines might be untrustworthy, and suggested a variety of ways to reduce COIs affecting guidelines.

These included in particular:
- Guideline committee members should disclose their conflicts of interest in detail, and these should appear in the guidelines (Standard 2.1, 2.2)
- The number of conflicted committee members should be minimized, and in no case should be a majority of the committee (2.3, 2.4)
- Funders should have no role in guideline development (2.4)

This week, an original article(1) and editorial(2) in PLoS Medicine discussed how guidelines published since the report was produced addressed conflicts of interest.

Article Summary

Below I summarized the article, but presented results somewhat differently than did the authors.

Design

Campsall et al did a cross-sectional study of clinical practice guidelines that appeared in the US Agency for Healthcare Research and Quality (AHRQ) National Guideline Clearinghouse in 2012 from national or international organizations, excluding guidelines produced by organizations restricted to allied health professionals, guidelines produced by corporations, or guidelines that had no specific recommendations.  They obtained information from the guidelines, the organizations' websites, and surveys sent to the organizations.

Study Population

The study population was 290 guidelines produced by 95 organizations. The response rate for the surveys was 68%.  The organizations were primarily professional societies (67%) or disease advocacy groups (21%).

My comment is that therefore this study assessed guidelines mainly produced by non-profit organizations which ostensibly uphold health care professionals' values and/or primarily support patients' interests.

Industry Funding

63% of organizations reported receiving funds from biomedical companies.  (It was not clear whether the remainder denied receiving such funding, or provided no information about funding.)

My comment is that the majority of such ostensibly "do-gooder" organizations are nonetheless at least somewhat supported by commercial firms that market health care products, presumably mainly drugs or devices.

Conflict of Interest Policies

Please note that the authors provided results to emphasize the positive, for example, they provided the proportion of organizations which provided specific types of information about COIs or had particular policies to reduce COIs.  In most cases, I simply did subtractions to emphasize the negative.  Furthermore, the authors further emphasized the positive by choice of denominator, as noted below.

20% of organizations did not provide any information about conflict of interest policies, and 7% asserted the existence of such policies, but did not disclose them.  A majority, 55%, did not have disclosed policies that specifically dealt with with conflicts of interest affecting guidelines.

My comment is thus that a majority of these do-gooder organizations did not apparently even begin to address, at least in terms of clear, transparent written policy, the recommendations of the IOM report on trustworthy guidelines.

Measures to Improve Guidelines Trustworthiness by Decreasing Influence of Conflicts of Interest

The authors provided the number of organizations with disclosed policies that addressed particular issues of concern to the IOM in their report on trustworthy guidelines, but made the proportions of such organizations seem larger by using as a denominator the number of organizations that disclosed their entire policies, rather than the total number of organizations which produced the guidelines.

Therefore, they found that of the 60 (out of 95 total) organizations that fully disclosed their conflict of interest policies:
98% required committee members to disclose conflicts of interest;
90% required review of conflicts of interest before guideline publication; 
68% required that a majority of guideline committee members be free of conflicts of interest;
92% reported publishing committee members' conflicts of interest within guidelines;
78% did not allow direct industry funding of guideline development
82% did not allow industry participation in selection of committee members
67% did not allow industry to review guidelines prior to publication.

That all sound good, but consider what happens when one uses the total sample of organizations in the denominators.

Therefore, the article also found that of all 95 organizations that produced the guidelines of interest:

38% did not require CPG committee members to disclose conflicts of interest
43% did not require review of committee members' conflicts of interest prior to guidelines development
57% did not require that a majority of guideline committee members be free of conflicts
44% did not report publishing committee members' conflicts of interest within guidelines
48% did not report preventing direct industry funding of guidelines
51% did not report preventing industry participation in selection of committee members
61% did not report preventing industry review of guidelines prior to publication.

My comment is that thus the results suggested that substantial numbers of organizations that produced CPGs did not demonstrate that they were committed to measures that would improve guidelines trustworthiness by reducing the influence of conflicts of interest.

Disclosure of Conflicts of Interest Within Guidelines

Again, it was necessary to recalculate some proportions using the full population of guidelines reviewed as the denominator.

Of the 290 guidelines reviewed:
65% "included disclosure statements regarding direct funding and support"
37% specifically disclosed the absence or presence of direct funding from biomedical companies
However,
49% did not disclose committee members' financial relationships
99% did not disclose relevant financial relationships of the organization (that is, institutional conflicts of interest) that produced the guidelines as a whole

My comment is that substantial numbers of published guidelines were not clearly produced so as to increase their trustworthiness by reducing the influence of conflicts of interest on them. 

Did the Guidelines Follow the Organizations' Stated Policies?


The authors commendably provided information about whether the guidelines produced by organizations that disclosed conflict of interest policies followed those polices.  They found:

16% of organizations "that reported that committee member conflicts of interested were published in guidelines" produced guidelines that did not include committee members' COI disclosure

61% of organizations that "reported the majority of committee members must be free of conflicts of interest" produced guidelines by committees with majorities of conflicted members

6% of organizations that "reported that industry partners were not permitted to directly fund clinical practice guideline development" produced guidelines which disclosed such funding.

My comment is that thus even of the organizations that stated they had policies in place to improve the trustworthiness of the guidelines they produced by reducing the influence of conflicts of interest, they allowed guidelines to be produced that did not follow those policies.

Authors' Conclusion

Financial relationships between organizations that produce clinical practice guidelines and the biomedical industry appear to be common. These relationships are important because they may influence, through guideline usage, the practice of large numbers of healthcare providers. We believe that to effectively manage conflicts of interest, organizations that produce clinical practice guidelines need to develop robust conflict of interest policies that include procedures for managing violations of the policy, make the policies publicly available, and disclose all financial relationships with biomedical companies.

Editorialists' Conclusion

The editorial by Bastian(2) was sharper in tone.  It included an observation that reinforced my confession of naivete above:

With hindsight, I think those of us encouraging better methodology for guideline development in the 1990s took the issue of disclosure of financial interests too much for granted. It seemed so self-evident, it got barely a mention even in national policy on guideline development

She also noted that the Campsall study did not address all the ways conflicts of interest can influence guideline development:

Stelfox and colleagues focus particularly on the organizational conflicts of interest of guideline producers and their policies. They examine the financial interests of the organizations, but not of the individuals employed within those organizations. This same blind spot is evident when it comes to policies about committee members; the financial interests of the organizations that individuals represent tend to be disregarded. Yet these can be substantial, including for patients’ organizations.

She noted that the failure of guidelines to report their developers' conflicts of interest is a continuing problem.  The rate of failure to report found by Campsall et al was

about the same rate that Taylor and Giles found in 2004 and Norris and colleagues found in 2010.

Finally, she emphasized that we are still a long way from having guidelines that health care professionals and patients can trust:

Guideline processes without adequate financial conflict management have to become unacceptable to a far wider circle. They need to become unacceptable to influential committee members, to the medical journals that lend so many guidelines additional standing and reach, and to the membership of the professional societies that produce them. Until that happens, for guidelines as for clinical research, it’s a case of caveat lector: let the reader beware.

My Summary

So should physicians trust clinical practice guidelines?  At least this article suggests they ought to be very very skeptical of them.  The IOM report meant to improve the trustworthiness of practice guidelines seems to be honored mainly in the breach.  The likelihood that any given guideline was produced so as to reduce the influence of conflicts of interest on it is low.  Even organizations that ostensibly put professional values and patients first in their efforts to develop guidelines seem to often be financially beholden to companies that want to sell drugs and devices.  Physicians and patients ought to be concerned that most new clinical practice guidelines may be as much about marketing commercial products as improving medical practice or patients' outcomes.

The current US presidential race has made it evident that we have a lot of disgruntled citizens, many of whom believe our system is rigged to favor the "establishment."  I suggest that the more we know about clinical practice guidelines, the more it appears that the health care system is rigged to favor certain parts of the "establishment," the big corporations that market drugs and devices, and their paid part-time hands within medical societies and patient advocacy groups who have turned these ostensibly idealistic, do gooder organizations into part-time marketing machines.

The huge and complex web of individual and institutional conflicts of interest that binds much of the health care system, the government, and industry may be good for the insiders, but is stifling improvement in our dysfunctional health care system.  True health care reform would first expose these conflicts, then reduce or better yet, eliminate them, and make health care more about helping patients and less about making money by marketing commercial products.

Musical Interlude

To dispel the darkness a bit, Paul Simon, "Still Crazy After All These Years," 1992 acoustic version:





References

1.  Campsall P et al.  Financial relationships between organizations that produce clinical practice guidelines and the biomedical industry: a cross-sectional study.  PLoS Medicine 2016; DOI;10.1371/journal.pmed.1002029    Link here.

2.  Bastien H. Nondisclosure of financial interest in clinical practice guideline development: an intractable problem? PLoS Medicine 2016;  DOI;10.1371/journal.pmed.1002030  Link here.