Order effects and counterbalancing

When the levels of a factor (test conditions) are assigned within-subjects, participants are tested with one condition, then another condition. and so on. In such cases, interference between the test conditions may result due to the order of testing, as noted above.

In most within-subjects designs, it is possible -- in fact, likely -- that participants' performance will improve as they progress from one test condition to the next. Thus participants may perform better on the second condition simply because they benefited from practice on the first. They become familiar with the apparatus and procedure, and they are learning to do the task more effectively.

Practice, then, is a confounding variable, because the amount of practice increases systematically from one condition to the next. This is referred to as a practice effect or a learning effect.

Although less common in HCI experiments, it is also possible that performance will worsen on conditions that follow other conditions. This may follow from mental or physical fatigue -- a fatigue effect. In a general sense, then, the phenomenon is an order effect or sequence effect and may surface either as improved performance or degraded performance. depending on the nature of the task, the inherent properties of the test conditions, and the order of testing conditions in a within-subjects design.

If the goal of the experiment is to compare the test conditions to determine which is better (in terms of performance on a dependent variable), then the confounding influence of practice seriously compromises the comparison. The most common method of compensating for an order effect is to divide participants into groups and administer the conditions in a different order for each group. The compensatory ordering of test conditions to offset practice effects is called counterbalancing.