Assessing interrater agreement on binary measurements via intraclass odds ratio.

TitreAssessing interrater agreement on binary measurements via intraclass odds ratio.
Publication TypeJournal Article
Year of Publication2016
AuthorsLocatelli, I, Rousson, V
JournalBiom J
Volume58
Issue4
Pagination962-973
Date Published07/2016
DOI10.1002/bimj.201500109
ISSN1521-4036
Mots-clésBinary measurements, Biometry/methods, Concordances and discordances, Data Interpretation, Humans, Interrater agreement, Intraclass correlation, Intraclass odds ratio, Observer Variation, Odds Ratio, Reproducibility of Results, Statistical
Abstract

Interrater agreement on binary measurements is usually assessed via Scott's π or Cohen's κ, which are known to be difficult to interpret. One reason for this difficulty is that these coefficients can be defined as a correlation between two exchangeable measurements made on the same subject, that is as an "intraclass correlation", a concept originally defined for continuous measurements. To measure an association between two binary variables, it is however more common to calculate an odds ratio rather than a correlation. For assessing interrater agreement on binary measurements, we suggest thus to calculate the odds ratio between two exchangeable measurements made on the same subject, yielding the concept of "intraclass odds ratio". Since it is interpretable as a ratio of probabilities of (strict) concordance and discordance (between two raters rating two subjects), an intraclass odds ratio might be easier to understand for researchers and clinicians than an intraclass correlation. It might thus be a valuable descriptive measure (summary index) to evaluate the agreement among a set of raters, without having to refer to arbitrary benchmark values. To facilitate its use, an explicit formula to calculate a confidence interval for the intraclass odds ratio is also provided.

Notes

Publication Status: aheadofprint

Alternate URL

http://www.ncbi.nlm.nih.gov/pubmed/26988408?dopt=Abstract

WOS ID (UT)

000379929300014

Alternate JournalBiom J
Citation Key / SERVAL ID6967
PubMed ID26988408

                         

IUMSP | www.iumsp.ch
Institut universitaire de médecine sociale et préventive
Route de la Corniche 10, 1010 Lausanne - Switzerland
+41 21 314 72 72 | iumsp@chuv.ch

Go to top