Social Self-awareness Validation – A Modern (and Better) Alternative to Traditional 360 Surveys

Statistics indicate that currently the number of people using social media is over 4.95 billion worldwide.  Connectivity is ranked as one of the most significant benefits of social media, linking countless users at any time, everywhere. Users use their connections to share memories, exchange information and opinions, and keep current. They gather inputs on everything including validating their own perceptions. It would seem logical that their perception of their own personal strengths and development needs could also be validated through social means. We will examine this approach as an alternative to traditional 360 surveys. 

TRADITIONAL 360 SURVEYS

A traditional 360 assessment survey consists of a series of behavioral statements associated with job relevant competencies. Ratings are provided for each behavior as a measure of the performance of an individual on the behaviors and the underlying competencies. The individual being rated completes a behavioral survey on themselves while others are also completing surveys at the same time. Different perspectives are included such as manager, peers, direct reports, etc. The ratings are aggregated across raters within a given perspective to generate perspective measure.

Traditional multi-rater (360) assessments have been around for some time. The underlying assumption behind developmental 360 feedback systems is that an individual’s self -awareness and perceived need for change will be enhanced by a systematic process of introspection and the review and comparison of ratings from others (Church & Bracken, 1997). It is hypothesized that:

1. Multiple raters provide a more accurate identification and understanding of development needs.
2. An individual’s self-awareness and motivation to change will be heightened by sharing and reviewing the rating results from different perspectives.

Research and practical experience with Traditional 360’s have not fully supported these assumptions:

1. Measurement quality – One of the big complaints of 360’s is the administrative burden and time demands it places on raters that participate in multiple surveys.  In a traditional 360, everyone independently rates the behavioral performance of the individual using a common survey.  However, an individual is often a rater for multiple people, and is required to complete multiple lengthy surveys.  In these circumstances, raters can suffer from “survey fatigue”.  The diligence and attention of the rater can become compromised and raises concerns about the reliability and accuracy of the ratings that result from the process.
2. Enhanced awareness and motivation to develop – In a traditional 360, research has shown that an individual’s self-ratings are typically significantly higher than the ratings of others (Harris & Schaubroeck, 1988). This presents a challenging feedback situation where the individual is first confronted with the reality that others rate them much less favorably than they rate themselves.  Individuals often discount the ratings of other’s or become defensive and demotivated by the lower rating values.

The inflated self-ratings and low self-other agreement with the traditional 360 approach is a problem since research has shown that high self-other agreement is associated with several positive outcomes. Some of these positive outcomes include perceived need for change (London & Smither, 1995), performance improvement after feedback (Atwater & Yazmmarino, 1992; Atwater et al., 2005; Johnson & Ferstl, 1999) and leadership effectiveness (Atwater, Rouch, & Fischthal, 1995).

Does the low level of self-other agreement in traditional 360 surveys mean that most individuals have low self-awareness? It may be that the low level of agreement between self and others is not because individuals have such little self-awareness but has more to do with the traditional 360 process.

SOCIAL SELF-AWARENESS VALIDATION

OMNIview has developed an alternative process that overcomes the problems of rater fatigue and inflated self-ratings. The process basically mirrors many modern social media interactions where an individual uses their social network to validate their self-observations: In this case, it would be: “Hey everyone, I am trying to work on some of my competency gaps. This is where I see myself doing well and these are some areas where I think I need to develop… Do you agree or have other suggestions”

The actual process is:

1. The individual rates their performance in comparison to structured performance standards for each behavior within a competency.
2. The self-ratings are then shared with the other raters along with the performance standards to gather their perceptions of agreement or disagreement with the ratings.
3. Raters indicate their agreement at the competency level and adds helpful, actionable comments.

It was our belief that having an individual share their self-ratings with others would make them more introspective and thoughtful about their true levels of competency performance and predispose them to receiving confirming or disconfirming feedback. It was also thought that having raters provide agreement ratings at the competency level rather than the behavioral level would greatly reduce rater fatigue.

Our experience has confirmed this to be the case and we also conducted research to scientifically evaluate the superiority of the approach over the traditional 360 process. The results were amazing. There was a highly significant relationship between self-ratings and the ratings by others and no significant differences in their means. In fact, the self-ratings were slightly lower than the ratings of others. It appears the Social Self-Awareness Validation process corrected the past problems with inflated self-ratings and low self-other agreement.

BENEFITS OF NEW APPROACH

The research study provided supporting evidence for the benefits of sharing self-ratings and performance standards with other raters as a stimulus for gathering 360 ratings. This process seems to correct past problems of overly inflated self-ratings and low self-other rating agreement reported for traditional 360 processes. It suggests that the transparency of the self-rating and the structure of performance standards may drive a more reflective approach that heightens the self-awareness of the individual.

The benefits of this approach are most likely to be seen in performance feedback sessions and subsequent developmental planning. These sessions will not have the difficult task of overcoming potential defensiveness associated with presenting data showing other’s perceptions being much lower than the individual’s self-perceptions. The higher agreement between self and others should also make it easier for the individual to accept identified development needs as being accurately measured which should increase their willingness to act on the data.

Finally, the more thoughtful introspection driven by the Social Self-Awareness Validation process should help predispose the individual for receiving feedback since they have already thought carefully about their true strengths and development needs.

There are likely to be other benefits to higher self-other agreement including a higher willingness to participate in future 360 surveys. The process should be seen as less threatening by the target individuals. The higher efficiency of this process for gathering others’ ratings should also increase the overall willingness of all participants to use 360 surveys for tracking performance improvements and guiding developmental planning efforts.

---

REFERENCES

Atwater, L.E, Rouch, P., & Fischtal, A. (1995). The influence of upward feedback on self and follower raters of leadership. Personnel Psychology, 48: 34-60.

Atwater, L.E., Waldman, D., Ostroff, C., Robie, C., & Johnson, K.M. (2005). Self-other agreement: Comparing its relationship with performance in the U.S. and Europe. International Journal of Selection and Assessment. 13: 25-40.

Atwater, L.E., & Yammarino, F.J. (1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personnel Psychology, 45: 141-164.

Church, A.H., & Bracken, D.W. (1997). Advancing the state of the art of 360-degree feedback: Guest editors’ comments on the research and practice of multirater assessment methods. Group & Organization Management, 22: 149-191.

Harris, M.M., & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. Personnel Psychology, 41: 43-62.

Hauenstein, P.C. (2013). An Evaluation of the Impact of Sharing Self Ratings and Performance Standards with Other Raters as a Stimulus for Gathering 360 Ratings. Omniview Publication.

Johnson, J.W., & Ferstl, K.L. (1999). The effects of interrater and self-other agreement on performance improvement following upward feedback. Personnel Psychology, 52: 271-303.

London, M., & Smither, J.W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performance-related outcomes? Theory based applications and directions for research. Personnel Pzsychology, 48: 803-839.