Intermediatemethods

Experimental Design in Biology: Building Tests That Work

Overview

A brilliant hypothesis fails with a sloppy experiment. This lesson covers the architecture of good experimental design: defining measurable outcomes, replication, randomization, blinding, sample size and power, and the common design flaws that invalidate biological studies.

Learning objectives

  • State the components of a well-designed experiment.
  • Explain biological vs technical replication.
  • Explain randomization and blinding and the biases they prevent.
  • Discuss sample size and statistical power.
  • Identify common design flaws: confounding, pseudoreplication, HARKing.

Design before data

Good design starts before any pipetting: state the question precisely, define the measurable outcome, identify the comparison that answers it, and plan the analysis. The most expensive mistake in biology is collecting data that cannot answer the question — weeks of work producing numbers that prove nothing. Design is where experiments are won or lost.

The design question

'What comparison, with what measurement, would convince a skeptic?' Answer that on paper before touching a pipette.

Replication: biological versus technical

Repeating a measurement on the same sample is technical replication — it estimates measurement noise. Repeating on independent biological samples is biological replication — it estimates real variability and is what lets conclusions generalize. Three pipetting replicates of one culture is n=1 with extra precision, not n=3: a classic trap called pseudoreplication.

Randomization and blinding

Randomization — random assignment of subjects to treatments — breaks the link between treatment and confounding variables (cage position, handling order, genetic background). Blinding — keeping experimenters unaware of which sample is which during measurement — prevents unconscious bias in scoring. Together they defend against the experimenter's own expectations, the subtlest confounder of all.

Bias is usually unconscious

Blinding isn't about distrust — it's about recognizing that expectation subtly shapes measurement, even in honest researchers.

Sample size and power

Too few samples and real effects hide in noise (low power); too many wastes resources and animals. Power analysis uses the expected effect size and variability to compute the n needed for a reliable answer. Pilot experiments estimate variability; ethics committees demand the minimum n that still answers the question — especially for animal work, where the 3Rs (replace, reduce, refine) apply.

Classic design flaws

Confounding: the treatment groups differ in more than the treatment (e.g., drug group also handled more). Pseudoreplication: treating non-independent measurements as independent. HARKing: hypothesizing after results are known — presenting exploratory findings as predicted. Cherry-picking: reporting only supportive data. Each has the same cure: pre-register the design, randomize, blind, replicate biologically, and report everything.

HARKing corrupts the logic

Testing a hypothesis suggested by the same data that 'confirms' it is circular — predictions must precede the results they explain.

Key points

  • Design the comparison and analysis before collecting data.
  • Biological replication (independent samples) generalizes; technical replication only measures precision.
  • Randomization defeats confounders; blinding defeats observer bias.
  • Power analysis sets sample size from effect size and variability.
  • Avoid confounding, pseudoreplication, HARKing, and cherry-picking.

Common misconceptions

✗ Myth: More data always fixes a bad design.

✓ Truth: No sample size rescues confounding or pseudoreplication — design flaws are structural, and more data just measures the wrong thing more precisely.

✗ Myth: Technical replicates count as independent samples.

✓ Truth: Repeated measurements of one sample estimate instrument noise, not biological variability — treating them as n inflates false confidence.

Practice questions

Q1. A researcher measures one treated culture three times and reports n=3. This is:
  1. biological replication
  2. pseudoreplication
  3. randomization
  4. blinding
Answer: B.

Three technical measurements of one biological sample are n=1 — treating them as independent inflates confidence spuriously.

Q2. Blinding the person scoring tissue slides primarily prevents:
  1. contamination
  2. observer bias from knowing treatment assignments
  3. random error
  4. equipment failure
Answer: B.

Expectation unconsciously shapes judgment; blinding keeps scoring honest by hiding which sample received which treatment.

Q3. The main purpose of a power analysis before an experiment is to:
  1. guarantee significance
  2. determine the sample size needed to reliably detect the expected effect
  3. choose the statistical software
  4. write the hypothesis
Answer: B.

Power analysis translates expected effect size and variability into the n required — avoiding underpowered inconclusive studies and wasteful oversampling.

Click a question to reveal the answer and explanation.

Frequently asked questions

What is the difference between biological and technical replication?

Technical replication repeats measurements on the same sample (estimates measurement precision). Biological replication uses independent biological samples (estimates true variability and supports generalization).

Why randomize treatments?

Random assignment distributes confounding variables — cage position, handling order, genetic background — evenly across groups, so differences can be attributed to the treatment rather than hidden factors.

What is pseudoreplication?

Treating non-independent measurements as independent replicates — e.g., counting three measurements of one culture as n=3 — which underestimates variability and overstates statistical confidence.

What is HARKing and why is it a problem?

Hypothesizing After Results are Known: presenting a post-hoc explanation as if it were a prior prediction. It's circular — the hypothesis was fitted to the same data used to 'test' it.