Oliver C. Mudford, Neil T. Martin, Jasmine K. Y. Hui, & Sarah Ann Taylor.
Assessing observer accuracy in continuous recording of rate and duration: Three algorithms compared.
Journal of Applied Behavior Analysis,
42, 527-539.
The three algorithms most frequently selected by behavior-analytic
researchers to compute interobserver agreement with continuous
recording were used to assess the accuracy of data recorded from video
samples on handheld computers by 12 observers. Rate and duration of
responding were recorded for three samples each. Data files were compared
with criterion records to determine observer accuracy. Block-by-block and
exact agreement algorithms were susceptible to inflated agreement and
accuracy estimates at lower rates and durations. The exact agreement
method appeared to be overly stringent for recording responding at
higher rates (23.5 responses per minute) and for higher relative duration
(72% of session). Time-window analysis appeared to inflate accuracy
assessment at relatively high but not at low response rate and duration
(4.8 responses per minute and 8% of session, respectively).
DESCRIPTORS: continuous recording, interobserver agreement, observer accuracy, observational data, recording and measurement