We Often See Quite a Lot of Interesting Research Findings, but How Many of Them Are Trustworthy?

Experimenter bias plagues research publications every year.

Experimenters, their studies, and their results are far from perfect.

We have all heard of the academic misconduct, intentional manipulation, and researchers who blatantly lie in their research.

However, it is safe to assume that most researchers have good intentions when performing experiments and writing publications.

Despite good intentions, it is important to understand that all researchers are subject to one major downfall: experimenter bias.

Understanding Experimenter Bias ^[1]

In its simplest form, bias is when our mind tends to favor something.

We all have our own set of bias, including our political views, our ideology, or what we expect from someone or something.

These biases influence how we speak, what we do, or who we vote for.

This isn’t only the case in everyday living, but in research as well.

Experimenters struggle to keep their preconceived notions out of their experiments. Unfortunately, this can happen during their experiment and influence the results.

This process is termed experimenter bias.

How Experimenter Bias Happens

When experimenters interact too closely with their subjects, or have preconceived notion of what to expect, biases start influencing the experiment. These effects are usually subtle, and often times even unintentional.

In fact, most researchers may get so caught up in their research that they get trapped in their own hypothesis.

For instance, a researcher might over explain the intended results to their subjects, and the knowledge the subjects gain can influence their behavior.

Experimenter and subject interaction isn’t the only source experimenter bias either.

Experimenter bias can also be in the form of the design. Becoming too infatuated with their outcome can cause them to manipulate the experiment.

Examples of Experimenter Bias ^[2]

We are all familiar with the bodybuilding supplement industry. They often show their products producing incredible strength gains or weight loss results through multiple “studies”, while a different study with the same ingredients fail to indicate anything. If these are considered clinical studies, they can often times be an example of experimenter bias.

Essentially, the researchers altered specific aspects of their experiments to produce the results they were hoping to see, either with participants they choose, the way they interacted with their subjects, or by the way they designed the testing.

However, not all types of experimenter bias are intentional.

One of the most popular examples of experimenter bias was done by Rosenthal and Fode in 1963 (2). In this example, two groups of students received rats to analyze. These rats were suppose to be judged on their ability to navigate a maze. One group was told their rats were “bright” while another was told their rats were “dull”, although in reality both groups were randomized without any different characteristics.

The students who analyzed the “bright group” rated their rats more highly then did the “dull group”. In essence, the group who anticipated their rats to perform well, influenced their actions to prove it. Rosenthal and Fode noted that this may have even been done unconsciously.

How Researchers Reduce Experimenter Bias ^[3]

Extensive Peer Review Process

If enough qualified “eyes” review the publication, then hopefully the biases become identified and the experiment isn’t published.

Blind Data Collectors

This is achieved by having data collecting personnel unaware of the subjects (both the control and experimenter group) and unaware of the hypothesis. Therefore, they don’t know what the expected outcome is when they perform the experiment and collect the results.

Double-Blind Experimenter Design

With double-blind studies, both the experimenter and subjects are unaware of which group is controlled and which group is experimental. In addition, the design of the experiment can also be done by someone who is unaware of the hypothesis.

How You can Identify Experimenter Bias as a Reader

Look for key aspects including:

• A control group
• It is a “double-blind” experiment, both the experimenter and subjects are withheld from knowing which group is the control and which is the experimental
• The funder isn’t influencing or interacting with the experiment
• Evidence that the publication went through a rigid review process
• That the selection of applicants was randomized
• Assure that the control group was evaluated as thoroughly as the experimental group

If any of these criteria aren’t meant, you should start analyzing the publication more rigidly and start questioning its quality and you should begin questioning if its worth citing.

What Should Be Taken Away From This?

• It is necessary to read the entire research publication and not just the abstract and results.
• It is essential as readers that we can identify and disseminate when a bias is occurring.
• Understanding the context of the experiment is just as important as understanding the results.
• Even the best researchers with the greatest intentions are still susceptible to bias errors

As readers, it is just as much our responsibility to interpret and understand the literature as it is for researchers to produce honest and quality literature.

The next time you hear somebody talk about a “study” or “research” make sure to question them on experiment, and don’t be afraid to discuss biases.

Reference

[1]	^	Brent Strickland: Experimenter Philosophy: the Problem of Experimenter Bias in Experimental Philosophy
[2]	^	PLOS: Evidence of Experimental Bias in the Life Sciences: Why We Need Blind Data Recording
[3]	^	Pergamon Press Ltd: A Longitudinal Study of the Effect of Experimenter Bias on the Operant Learning of Laboratory Rats

The post We Often See Quite a Lot of Interesting Research Findings, but How Many of Them Are Trustworthy? appeared first on Lifehack.