University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Preventing Rater Biases in 360-Degree Feedback by Forcing Choice

Brown, A, Inceoglu, I and Lin, Y (2017) Preventing Rater Biases in 360-Degree Feedback by Forcing Choice Organizational Research Methods.

[img]
Preview
Text
Brown%2c Inceoglu%2c & Lin (in press) Preventing rater biases in 360-degree feedback ORM.pdf - Accepted version Manuscript
Available under License : See the attached licence file.

Download (702kB) | Preview
[img]
Preview
Text
Brown%2c Inceoglu%2c & Lin (in press) SUPPLEMENT.pdf - Accepted version Manuscript
Available under License : See the attached licence file.

Download (188kB) | Preview
[img]
Preview
PDF (licence)
SRI_deposit_agreement.pdf
Available under License : See the attached licence file.

Download (33kB) | Preview

Abstract

We examined the effects of response biases on 360-degree feedback using a large sample (N=4,675) of organizational appraisal data. Sixteen competencies were assessed by peers, bosses and subordinates of 922 managers, as well as self-assessed, using the Inventory of Management Competencies (IMC) administered in two formats – Likert scale and multidimensional forced choice. Likert ratings were subject to strong response biases, making even theoretically unrelated competencies correlate highly. Modeling a latent common method factor, which represented non-uniform distortions similar to those of “ideal-employee” factor in both self- and other assessments, improved validity of competency scores as evidenced by meaningful second-order factor structures, better inter-rater agreement, and better convergent correlations with an external personality measure. Forced-choice rankings modelled with Thurstonian IRT yielded as good construct and convergent validities as the bias-controlled Likert ratings, and slightly better rater agreement. We suggest that the mechanism for these enhancements is finer differentiation between behaviors in comparative judgements, and advocate the operational use of the multidimensional forced-choice response format as an effective bias prevention method.

Item Type: Article
Subjects : Surrey Business School
Divisions : Faculty of Arts and Social Sciences > Surrey Business School
Authors :
AuthorsEmailORCID
Brown, AUNSPECIFIEDUNSPECIFIED
Inceoglu, IUNSPECIFIEDUNSPECIFIED
Lin, YUNSPECIFIEDUNSPECIFIED
Date : 19 February 2017
Copyright Disclaimer : Copyright 2016 The Authors. This is the authors' accepted version.
Uncontrolled Keywords : multisource feedback, halo effect, rater biases, forced choice, Thurstonian IRT model, ideal-employee factor
Related URLs :
Depositing User : Symplectic Elements
Date Deposited : 19 Aug 2016 10:20
Last Modified : 26 Sep 2016 10:07
URI: http://epubs.surrey.ac.uk/id/eprint/811741

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800