8 isn’t good enough?

20100325982

I believe in a normal distribution when it comes to rating things. Sure, it’s flawed, but so is every other scoring paradigm. If a “C” is really supposed to indicate average, then 9 out of 10 students shouldn’t be getting A’s and B’s.

It sometimes seems most people don’t take scoring that literally. If something met the requirements, it gets an A. If just meeting the requirements is head over heels better than all the competition, then maybe it really is an A but I’d also argue that perhaps the requirements aren’t reasonable. There should be room about just meeting the requirements for a higher score so that when something unexpected comes along and blows you away, you aren’t trying to rate it 11 on a 10 point scale. And when that thing does come along, average gets shifted and a new scale is created making room for the new blow-me-away feature.

Star ratings on websites are hard for a number of reasons. Ten 5 star ratings along with ten 1 star ratings suggests and average product as the rating will work out to average – 2.5. But it’s not actually seen as average to any of the raters. It’s either fantastic or terrible. On top of that, those stars are very subjective. I call average actually average, meaning room for improvement, meaning not a top rating. Another person might consider average as meeting all expectations and therefore deserving of the highest possible honor.

Catch me in the right mood and I sometimes fill out those surveys that get promoted at the bottom of shopping receipts. I’m not so much interested in actually rating the store or experience or explaining how likely I am or am not to recommend the store to a friend but rather in seeing what sort of questions, wording, format and technology are used. Nerding for my benefit, not corporate’s.

I recently filled out a Survey for Sears. All the questions were rate on 10 point scale, 10 being the highest. I always rate stuff slightly higher than what I really consider honest due to being used to my 8 being more like someone else’s 10. The survey was all about that one particular experience, not combination of all my Sears shopping experiences. So I rated it. It was a perfectly fine experience. There was nothing memorable about it – good or bad. A lot of 8′s.

Near the end of the survey it asked I select another store and rate that experience. I thought this was a really good idea as it would put my scores into perspective as it would give them something to compare to. Perhaps allows for adjustment of the scores making the ratings less subjective. Chances are someone is either going to choose a store they had a really good experience at or a really bad experience at which should show what sort of score they give in those situations. I ended up just choosing one with a similarly average experience (seriously, I’m typically satisfied with shopping experiences, rarely blown away or extremely annoyed) and gave it similar ratings. There, now they know that when I say 8 I really mean that as what I typically find in a store experience.

I left contact information so I could win a prize. I got contacted, but not about a prize.

“Why all the 8s?” they asked, suggesting only 9s and 10s are good enough and anything lower is cause for concern.

I tried to explain that I consider an 8 to be a very high score, even higher than it should be for an everyday average experience. Sears did everything I expected them to on that shopping occasion.

“Then why not a 10?”

If average is 10, then what is better than average? When you surprise me with something fantastic that goes beyond my expectations, you’ll get your 10. And that answer of a 9 or 10 to “How likely are you to shop here again?” should have been a pretty good indicator I was in no way disappointed.

I’ve seen arguments against rating widgets and surveys. Seems essays are the only way to go. But who has time for that? Which, of course, brings up another pitfall of ratings: rushing through them just to get done.

Comments

  1. Andy or matches or whatever 20100326

    How do you feel about Best Buy’s surveys? If I recall, they use a (strongly) agree/disagree scale. I think that makes more sense. I can “strongly agree” that I found the prices to be satisfactory (uh, I wouldn’t have shopped there otherwise). But if you’re asking me to rate your prices, the only way you’re getting a 10 is if the item I bought was on some ridiculous sale. MSRP does not equal a 10.

  2. waytoocrowded 20100326

    I’ve never been given the chance to fill out a Best Buy survey. I don’t even remember the last time I bought something there. That sounds like it’d be less prone to different interpretations though. And price questions are silly always, especially if not asking about convenience or urgency – I shop at Sears because I used to live 3 blocks away and even now it is on my daily commute. It could be Home Depot instead so long as the location was the same.

  3. majafa 20100327

    You got contacted for all 8s? haha. Now for some reason I feel that if you were to give all 2s or 3s, they probably wouldn’t contact you asking “why all the 2s?”. If you gave all 2s they’d probably feel you’re a disgruntled shopper and trying to get you to come back to their store would be a lost cause. But, you gave 8s, giving them hope and showing them you like the store, but there’s room for improve. It’s kinda like teasing a dog by holding a treat just out of their reach…they’re so close to getting what they want, but not quite haha.

  4. waytoocrowded 20100328

    I like the “teasing a dog” idea. Not straight 8′s. Probably some 7′s and even a 9 or two in there. And most certainly answered 10 to the will-you-shop-here-again question.

  5. matches 20100329

    I read that as “leasing a dog”, which is funny in a whole new way.


Skateboards and bikes are better at nights

Skateboards and bikes are better at nights. 0 comments.


Search WayTooCrowded
The Header Should Always Point Home