Absence of an Ideal Observer II
The Agonizing Search for Experts Without a Conflict of Interest
The concept of an ideal observer, advanced by Green,1 proposes that a truth exists beyond the distortions created by our preconceptions and prejudices, and that it can be discerned by a “fully informed, imaginative, impartial and calm observer.” Our current ethos depends profoundly on the presumption that an objective reality can be perceived and described by an unbiased and knowledgeable person. My prior exploration of this concept focused on the lack of an ideal observer in the planning, execution, and interpretation of clinical trials.2 This article explores the difficulties of finding an ideal observer in our efforts to identify trustworthy, reliable, and unconflicted experts.
We are obsessed with conflicts of interest, and we should be. When we seek experts, we want them to be ideal observers, but we know that their guidance may be imperfect. We accept imperfections if they are related to a lack of wisdom or experience, but we are repulsed if they are because of financial involvements that benefit the expert. We are comfortable if an expert speaks wrongly because of ignorance but not because of self-interest. Being deceived is acceptable if it is innocent but not if it is motivated.
Our Fondness for Focusing on the Paper Trail
How can we discern whether a motivation to distort or deceive exists? If we were travelers in a strange land and asked for directions, we would be annoyed but not aggrieved if we received the wrong guidance from another (presumably equally ill informed) visitor. Yet, we would be offended if the poor advice were provided by a native (whom we assume should know better). Is the difference in our emotional response reasonable? How do we know that the other tourist made an innocent error? How can we assume that the native knew the geography of the city? We cannot know motivations, and so we focus on the only information available to us (ie, the place of residence). It is an unreliable surrogate, but it is all we have. We embrace an ecological fallacy, which is appealing and wrong. If there is discoverable information, we depend on it; we forget that the truth may be invisible to us and may not leave a paper trail.
This is how we deal with conflicts of interest. We focus only on traceable conflicts. If an expert has received significant sums of money from a company and proffers an opinion that is favorable to its product, we conclude that that opinion was distorted by the financial reward to the expert. There is no doubt that this happens, and it is detestable, but can there be an alternative explanation? Is it possible that experts develop their opinions independently based on their own diligent work, and the company searched for and found someone whose independent opinion it liked and made a financial arrangement with them? If so, then the money was the result of the opinion, not the cause of it. Both explanations are plausible; we cannot know which is true, but we consider only one possibility. Because the opinion and money are associated, we assume that they are causally related and only in one direction. Accordingly, we routinely reject financially conflicted experts (even if their opinions are true and unbiased) and keep searching for the ideal observer.
Our reliance on the absence of a paper trail as an identifier of a lack of conflict of interest is similarly illogical. Years ago, I was asked by the US Food and Drug Administration (FDA) to participate as a reviewer on an advisory panel. The FDA staff was desperate to find unconflicted experts, and I seemed perfect. I was knowledgeable about the experimental and regulatory issues at hand, and I had no financial ties with the sponsor. However, the drug had saved the life of a family member, and so I recused myself, although the FDA officials were comfortable with my involvement because there was no monetary paper trail. Nonfinancial conflicts are commonplace; members of FDA advisory committees are often asked to evaluate presentations made by mentors, journal editors, or friends. These personal influences may be more powerful than money.
Recently, I was asked to review a manuscript that was critical of a drug that I had helped to develop. I had led the major trial with the drug, but I had not had any financial relationship or communication with the sponsor for years. Because of my knowledge, the editors felt (in the absence of a financial link) that I was an ideal reviewer. Yet I declined because I could not provide an unbiased review. My intellectual conflict of interest was more impactful than any financial ties.3 The editors disagreed because there was no traceable monetary connection.
The Unintended Consequences of Our Focus on Traceable Conflicts
Once it becomes widely known that the presence or absence of a personal financial paper trial is the decisive factor, it is simple to hide real financial conflicts of interest. A consultant who wishes to conceal evidence of inappropriate influence can easily find ways to do so. Instead of a personal bank account, the money can be directed as a contribution to a research foundation or presented as a grant to the investigator’s institution in a way that benefits the physician. Similarly, the funds can be donated to a charitable organization, which yields personal tax advantages for the consultant.4 Under these circumstances, the investigator can claim that there is no financial relationship because the flow of money is untraceable on a personal level; he or she is deemed to be unconflicted, although the bias is real.
The problem of an undisclosed conflict exists even among physicians who routinely express contempt for those with the financial ties to industry. Some have established a reputation (and often a career) by pointing out the presumed chicanery of drug and device manufacturers; the stature and self-esteem of these critics are highly dependent on sustaining their role as professional cynics. Consequently, these individuals have a conflict of interest that is more influential than most financial involvements.5 Some people even profit monetarily from their cynicism. They may receive research grants to promote an opposing viewpoint, and they may provide expensive services to attorneys who file legal suits against pharmaceutical and device companies. These personal and financial influences may or may not be disclosed on the usual conflict of interest statement.
We must be honest about conflicts of interest. If we avoid listening to those who have them and report them openly and truthfully to us, then we may end up relying on those who have consciously or inadvertently concealed their bias or who have no conflicts, conceivably because they have little expertise to offer. We cannot assume that conflicts are absent among highly qualified people who have no personal financial paper trail. Similarly, we should hesitate to embrace physicians with little insight or experience, although they can be readily designated as unconflicted experts because industry has not sought and paid for their advice. Those without a traceable financial conflict do not necessarily deserve the confidence we place in them. Yet we provide it anyway because we desperately need to believe in the ideal observer, although in truth it is hard to know or prove that one exists.1
During the past 3 years, Dr Packer has consulted for Admittance, Amgen, AstraZeneca, Bayer, BioControl, Boehringer Ingelheim, Cardiorentis, CardioKinetix, Celyad, Daiichi Sankyo, Ferring, Gliead, NovoNordisk, Novartis, Relypsa, Sanofi, Takeda, and ZS Pharma.
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
Circulation is available at http://circ.ahajournals.org.
- © 2017 American Heart Association, Inc.
- Green RF
- Packer M
- Akl EA,
- El-Hachem P,
- Abou-Haidar H,
- Neumann I,
- Schünemann HJ,
- Guyatt GH