blog

I read an interesting article from the Wall Street Journal entitled “On the Internet, Everyone’s a Critic, But They’re Not Very Critcial “. In it, Geoffrey A. Fowler and Joseph De Avila note that the “average grade for things online is about 4.3 stars out of five”. This may be a surprise to many (or at least it was to me) as a good part of the criticism I hear about web - especially social web - is in regard to the capability and aptitude of people to write/speak negatively. We hear about the ability of one negative experience-turned-review to snowball into a train wreck begging for disaster relief (see Pete Blackshaw’s Tell 3000 ).

Grade inflation (Positive Acquiescence Bias - thanks Bryce ) seems to be prevalent across the web - YouTube and Amazon are both reporting it, and averages are higher in the UK (4.4) than the US.

While I don’t see any critical problems with positive acquiescence bias on the public web (buyer/browser/analyst beware & get smart), it made me think about the manifestation of this behavior on internal social networks - especially as more organizations are exploring and deploying mechanisms and/or re-engineering for some degree of social business design (Alimeter Group , Dachis Group ).

Significant capital outlay for technology, change management, HR moves and more means that social business design is an investment. The investment is worth it, according to McKinsey survey results , but as companies push deeper into the space and begin to rely more heavily on information and insights gleaned from digital environments, I think we need to be aware of patterns and possible skews.

Why?

Because connecting a workforce is proving valuable: real-time feedback and data mean fast learning, course correction and innovation. Reputational systems applied to knowledge, resources, and options can quickly gauge a global and disparate organization’s sentiment and needs, allowing for informed business decisions… ‘informed’ being the operable word here. If feedback is inflated (one way or another), organizations need to be wary of making decisions based on it. The opportunity to gather and act on data is certainly there. It’s the qualification of that data, per the tendencies being reported in similar environments, that must be remembered.

So…

As Brian Link says, “sample sizes and % participation and correlated results from different data sets are key to interpreting these kinds of things” but I’m also wondering - Do we design against/for it? Do we coach against it? I’m not even close to an expert on reputational systems (again, see Bryce )) and haven’t researched inflation results beyond the WSJ article, but it made me think:

1. Are grades inflated?
2. Are there reputational system design considerations which can be made to combat or normalize this behavior or the data?
3. Is there group behavior coaching or leadership that can modify this these patterns?

Thoughts?