Controlling for response order effects in ranking items using latent choice factor modeling

IPM Vriens, GBD Moors, J Gelissen, JK Vermunt

Research output: Contribution to journalArticleScientificpeer-review

4 Citations (Scopus)

Abstract

Measuring values in sociological research sometimes involves the use of ranking data. A disadvantage of a ranking assignment is that the order in which the items are presented might influence the choice preferences of respondents regardless of the content being measured. The standard procedure to rule out such effects is to randomize the order of items across respondents. However, implementing this design may be impractical and the biasing impact of a response order effect cannot be evaluated. We use a latent choice factor (LCF) model that allows statistically controlling for response order effects. Furthermore, the model adequately deals with the known issue of ipsativity of ranking data. Applying this model to a Dutch survey on work values, we show that a primacy effect accounts for response order bias in item preferences. Our findings demonstrate the usefulness of the LCF model in modeling ranking data while taking into account particular response biases.
Original languageEnglish
Pages (from-to)218-241
Number of pages24
JournalSociological Methods and Research
Volume46
Issue number2
DOIs
Publication statusPublished - 2017

Cite this

@article{274aaeb1c3f84978a441e24585fcec5b,
title = "Controlling for response order effects in ranking items using latent choice factor modeling",
abstract = "Measuring values in sociological research sometimes involves the use of ranking data. A disadvantage of a ranking assignment is that the order in which the items are presented might influence the choice preferences of respondents regardless of the content being measured. The standard procedure to rule out such effects is to randomize the order of items across respondents. However, implementing this design may be impractical and the biasing impact of a response order effect cannot be evaluated. We use a latent choice factor (LCF) model that allows statistically controlling for response order effects. Furthermore, the model adequately deals with the known issue of ipsativity of ranking data. Applying this model to a Dutch survey on work values, we show that a primacy effect accounts for response order bias in item preferences. Our findings demonstrate the usefulness of the LCF model in modeling ranking data while taking into account particular response biases.",
author = "IPM Vriens and GBD Moors and J Gelissen and JK Vermunt",
year = "2017",
doi = "10.1177/0049124115588997",
language = "English",
volume = "46",
pages = "218--241",
journal = "Sociological Methods and Research",
issn = "0049-1241",
publisher = "SAGE Publications",
number = "2",

}

Controlling for response order effects in ranking items using latent choice factor modeling. / Vriens, IPM; Moors, GBD; Gelissen, J; Vermunt, JK.

In: Sociological Methods and Research, Vol. 46, No. 2, 2017, p. 218-241.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Controlling for response order effects in ranking items using latent choice factor modeling

AU - Vriens, IPM

AU - Moors, GBD

AU - Gelissen, J

AU - Vermunt, JK

PY - 2017

Y1 - 2017

N2 - Measuring values in sociological research sometimes involves the use of ranking data. A disadvantage of a ranking assignment is that the order in which the items are presented might influence the choice preferences of respondents regardless of the content being measured. The standard procedure to rule out such effects is to randomize the order of items across respondents. However, implementing this design may be impractical and the biasing impact of a response order effect cannot be evaluated. We use a latent choice factor (LCF) model that allows statistically controlling for response order effects. Furthermore, the model adequately deals with the known issue of ipsativity of ranking data. Applying this model to a Dutch survey on work values, we show that a primacy effect accounts for response order bias in item preferences. Our findings demonstrate the usefulness of the LCF model in modeling ranking data while taking into account particular response biases.

AB - Measuring values in sociological research sometimes involves the use of ranking data. A disadvantage of a ranking assignment is that the order in which the items are presented might influence the choice preferences of respondents regardless of the content being measured. The standard procedure to rule out such effects is to randomize the order of items across respondents. However, implementing this design may be impractical and the biasing impact of a response order effect cannot be evaluated. We use a latent choice factor (LCF) model that allows statistically controlling for response order effects. Furthermore, the model adequately deals with the known issue of ipsativity of ranking data. Applying this model to a Dutch survey on work values, we show that a primacy effect accounts for response order bias in item preferences. Our findings demonstrate the usefulness of the LCF model in modeling ranking data while taking into account particular response biases.

U2 - 10.1177/0049124115588997

DO - 10.1177/0049124115588997

M3 - Article

VL - 46

SP - 218

EP - 241

JO - Sociological Methods and Research

JF - Sociological Methods and Research

SN - 0049-1241

IS - 2

ER -