Over-rated and Under-rated courses

[sidewinding]

Assuming that Average=2.0, or even Average=2.5, it's just not possible by the definition of average that >50% of the courses are above a 2.5 rating.

We are dealing with two different definitions for average. The first definition is the center of a zero to five scale. The second definition is how you would actually rate something on that scale.

Let me give an example. If we looked at the average speed for all drivers on the interstate on a scale from 0-70mph would most drivers be driving at 35mph with a nice bell curve out from there?. No >50% would be driving faster than 35mph.

Let me give another example. If I asked you to rate the beauty of all the women you know on a scale of 1-10 would most of them be 5's with a nice bell curve out from their. No > 50% would be above 5 because most women are prettier than a 5.

Let me give another example. If we weighed all members of DGCR on a scale of 0 to 300 pounds would most of them be 150 pounds with a nice bell curve out from there. No > 50% would be >150 pounds. There would be a few at or above 300 but none at 0 pounds.

 

[ERicJ]

All your examples are using a numerical, linear scale. DGCR ratings were not implemented that way.

As Tim has said: the scale was originally like this:

Poor
Average
Good
Excellent
Best of the best

If you take away the numbers and use Tim's definition of Average, i.e. common... most courses will be "Average". His scale provides more room at the top to differentiate the really good courses. And I liked it that way.

However, you need some metric of averaging the ratings so numbers get used. Once that happens people have this desire to want to put the average in the middle of the range.

 

[timg]

Unless "Average" wasn't called "Average" of course

 

[JR Stengele]

I think we need to move on...anyone else feel the same way? I think it is difficult to make it perfect but what Timg is saying makes the most sense, and since he made the site lets follow his lead. I feel like this question is like asking what came first...the chicken or the egg. Eventually it doesn't matter, we are just happy to have it. Maybe I am wrong and I think there are a lot of valid points but I think we need to just stick with what it is. = ) By the way it was the egg. = )

 

[garublador]

We are dealing with two different definitions for average. The first definition is the center of a zero to five scale. The second definition is how you would actually rate something on that scale.

Let me give an example. If we looked at the average speed for all drivers on the interstate on a scale from 0-70mph would most drivers be driving at 35mph with a nice bell curve out from there?. No >50% would be driving faster than 35mph.

Yeah, but the average wouldn't be 35mph, either. I'll be there's still a bell curve, too.

Let me give another example. If I asked you to rate the beauty of all the women you know on a scale of 1-10 would most of them be 5's with a nice bell curve out from their. No > 50% would be above 5 because most women are prettier than a 5.

Yeah, but the average isnt' 5, either. I'll be there's still a bell curve (so to speak), too.

Let me give another example. If we weighed all members of DGCR on a scale of 0 to 300 pounds would most of them be 150 pounds with a nice bell curve out from there. No > 50% would be >150 pounds. There would be a few at or above 300 but none at 0 pounds.

Yeah, but the average wouldn't be 150lbs, either. I'll bet there's still a bell curve, too.

Generally when you talk about average in statistics you are talking about the mean, not the median. Analysis based on the median isn't nearly as useful as analysis based on the mean in situations like this.

Yes, if you take the mean of the possible scores you'll get 2.5. However, if you take the actual ratings of all the courses out there I'm guessing you won't get exactly 2.5 unless you purposly skew the results that way. That would mean that even if every course in existance were what most of us could call a 5 now that the wost of them would be a 0 and the very best would be a 5. Half of them would fall between 0.63 and 4.4. The question is do we want it to be that way or would we rather they all be called 5's?

In other words, should we rate the courses based on how good other coures are or based on how good the course is on its own? Do we want to force it to a bell curve where the mean is 2.5 or do we want to set up what each value represents and see what the mean comes out to? I honestly don't know, but putting the word "average" in there will make some people do the former and some people do the latter which will make the results less useful.

An example of what might happen if we force the average to be 2.5 would be if all of a sudden DG got super popular and a ton of really, really good courses got made. That would actually force the ratings of many 4's and 5's to be lowered to fit the curve. Would we rather see that happen or would we rather just see the actual average get higher? If you were doing a presentation to sell the idea of disc golf to a friend, which scale would you rather use, the one where 2.5 is the average or the one where the average was raised because of all the great courses being installed?

Not necessarily. The word "Average" has a semantic range with several meanings.

But it's still very ambiguous, especally when you're talking about gathering data on a site like this, which is my point. Do you really want ambiguous terms used when people come up with their ratings? That will skew some, but not all ratings based on how people interpret that one word.

Well my intention was for it to mean "b: not out of the ordinary"

Then label 2.5 as "ordinary." Personally I think "decent" is a better term to use because it has less of a statistical implication, but it's not my site.

 

[ERicJ]

Ok.. how about everyone weighs in on this

0.0 - Abysmal
0.5 - Very Poor
1.0 - Poor
1.5 - Passable
2.0 - Fair
2.5 - Average
3.0 - Good
3.5 - Very Good
4.0 - Excellent
4.5 - Phenomenal
5.0 - Best of the Best

Fair and Average pretty much mean the same thing so it's open to interpretation but I thought it would satisfy those that have a problem with the label positions while those that have rated courses according to the existing labeled scale can leave their ratings alone.

Tim, if you want to change the precident to move the "Average" value to the middle of the range I'm okay with that. It's probably what more users than not are expecting.

In my mind Fair is obviously less than Average. If this scale goes into effect I'll be re-rating courses to bump up 2.0's to 2.5's and adjust others accordingly.

We could also remove the word "Average" all together which I kind of like more. Again, no score adjustments necessary.

0.0 - Abysmal
0.5 - Very Poor
1.0 - Poor
1.5 - Passable
2.0 - Fair
2.5 - Decent
3.0 - Good
3.5 - Very Good
4.0 - Excellent
4.5 - Phenomenal
5.0 - Best of the Best

Keep the word Average. I suspect that users new to this site will want to establish the baseline at Average.

Please do not drop "Average". Like Olorin, I think "Average" sets a baseline for reviewers' frame of reference. Otherwise most of the other terms are pretty subjective. A new reviewer could possibly view "Good" to be their "Average" value.

I don't think this topic will ever completely die, but if you remove "Average" we'll see threads like "What should the average course be rated?".

The word "Average" has a semantic range with several meanings.

The MW dictionary says,
Function:adjective 1: equaling an arithmetic mean 2 a: being about midway between extremes <a man of average height> b: not out of the ordinary : common <the average person>

I think that "being about midway between extremes" is exactly what I take average to mean on DGCR

Well my intention was for it to mean "b: not out of the ordinary"

That too is how I've interpreted "average", i.e. most courses are like this. In pure statistical terms I treat this as the Mode.

 

[DGtourist]

I just wanted to say,

that I don't care, but keep uniform.

I want to never speak of this again.
&
Whomever neg voted my 2 disc rated courses is a jerk. (I know it could be anybody, but trust my ratings on these ones if you are traveling)
-Love JD

 

[timg]

Argh.. why is it so difficult to please everyone?

 

[garublador]

I don't think this topic will ever completely die, but if you remove "Average" we'll see threads like "What should the average course be rated?".

IMO, you'll get better data if you set it up so you get questions like that. The answer would be, "You can't possibly know unless you've played a significant number of courses." My guess is the only people qualified to really say what the "average" course is like won't be the one asking the question.

Here's something to think about if you're for putting "average" in as one of the rating descriptions. How do you readjust all the ratings if the quality of the average course changes? If new courses that are installed improve over time, who's going to go back and change the ratings of all the courses that have already been rated to reflect the new average? In other words, if courses improve over time, what's "average" now, a 2.5, wouldn't be "average" anymore and might only be a 2.0 after the standards have been raised.

If you let the average move over time, you don't have to worry about that.

Although it might be advantageous to know just how incorrectly people are scoring courses. If your average isn't 2.5 you know that your data is bad.

 

[timg]

Ok, I just changed the scale (again..). I'm not sure if it's the right decision but hopefully it will put an end to the subject. Here it is:

0.0 - Abysmal
0.5 - Very Poor
1.0 - Poor
1.5 - Passable
2.0 - Fair
2.5 - Decent/Average
3.0 - Good
3.5 - Very Good
4.0 - Excellent
4.5 - Phenomenal
5.0 - Best of the Best

You probably don't have to re-adjust scores but if you do, I apologize.

 

[garublador]

Argh.. why is it so difficult to please everyone?

I don't care how it's done, but I'd like to think I can help improve the process by pointing out the implications of using statsitcal terms when you aren't intending the statistical definition to be used. I can tell you right now that unless you want my ratings to fit so that it's a bell curve about whatever value is described as "average," using that word will cause me to enter bad data. If that's what you want then that's OK. If it's not then you'll have the people who understand statistics the best giving the worst data. I just want to make sure you're getting the data you want.

FWIW, I think the descriptions are more useful than the actual ratings. I'm more likely to choose what course I go to based on the pros and cons people give than what they rate the course.

 

[timg]

FWIW, I think the descriptions are more useful than the actual ratings. I'm more likely to choose what course I go to based on the pros and cons people give than what they rate the course.

That's the way most of us feel. The number is just a quick barometer for the most part.

 

[JR Stengele]

Even though you didn't have this scale to begin with, I have always seen it this way. I think that it has and continues to make sense. Thanks timg for your time and endless help to our questions.

 

[garublador]

2.5 - Decent/Average
3.0 - Good

So the average course isn't good?

Now I'm just giving you a hard time.

All the stuff I said only applies if you want to do statistical analysis on this at any time. I'd hate for you to go back and wish it was done differently.

 

[timg]

Eh, can't please everyone? I probably would have removed the "Average" label but some like it, some don't so I thought this was an ok compromise. I doubt I'll ever do any statistical analysis on the data so that concern isn't really part of the equation.

 

[ERicJ]

Flashbacks of Clinton and what is the meaning of "is"...

Here's something to think about if you're for putting "average" in as one of the rating descriptions. How do you readjust all the ratings if the quality of the average course changes? If new courses that are installed improve over time, who's going to go back and change the ratings of all the courses that have already been rated to reflect the new average? In other words, if courses improve over time, what's "average" now, a 2.5, wouldn't be "average" anymore and might only be a 2.0 after the standards have been raised.

If you let the average move over time, you don't have to worry about that.

This is possibly the best argument against using "Average" that I've read. At least it is the most optomistic for the sport.

But you still have an aspect of that problem that exists today and pertains to individual courses. I played WillCo back in January and rated it based on the conditions at the time. I've heard that since then concrete tee pads have been installed. That could up my rating as the mud-factor was one of the detractors at the time I played. So even if the general trend of course quality doesn't improve you can still have isolated cases where a course has improved and older ratings are now out of whack.

Although it might be advantageous to know just how incorrectly people are scoring courses. If your average isn't 2.5 you know that your data is bad.

Yeah, I got a dollar that says this site has some serious rating inflation.

I don't care how it's done, but I'd like to think I can help improve the process by pointing out the implications of using statsitcal terms when you aren't intending the statistical definition to be used. I can tell you right now that unless you want my ratings to fit so that it's a bell curve about whatever value is described as "average," using that word will cause me to enter bad data. If that's what you want then that's OK. If it's not then you'll have the people who understand statistics the best giving the worst data. I just want to make sure you're getting the data you want.

I still like a term that tries to set some sort of baseline or expectation. E.g. your definition of "Good" and mine may be pretty different; say you like trees and give an overly wooded course high marks, when in reality it's not a very balanced course.

But you make good points about people who take the view that "average" means the statistical mean rating.

What do people think about "Typical" instead of "Average"?

It still has that meaning of a typical/common course baseline without the strict statistics of an arithmetic mean.

FWIW, I think the descriptions are more useful than the actual ratings. I'm more likely to choose what course I go to based on the pros and cons people give than what they rate the course.

That's the way most of us feel. The number is just a quick barometer for the most part.

While this site is still relatively young and most courses only have a few reviews, if any, it's not too hard to check the pros/cons in addition to the numbers. But as the site grows and courses accumulate vast quantities of reviews it will become overwhelming for someone planning a trip to easily/quickly decide which two of the ten courses at their destination they want to play by reading all the reviews. At that point the numeric barometer becomes more of a factor.

ERic

 

[DGtourist]

Flashbacks of Clinton and what is the meaning of "is"...


Yeah, I got a dollar that says this site has some serious rating inflation.



While this site is still relatively young and most courses only have a few reviews, if any, it's not too hard to check the pros/cons in addition to the numbers. But as the site grows and courses accumulate vast quantities of reviews it will become overwhelming for someone planning a trip to easily/quickly decide which two of the ten courses at their destination they want to play by reading all the reviews. At that point the numeric barometer becomes more of a factor.

ERic

Those first two comments made me laugh!

As far as trip planning goes, I feel like a person doing the sort of research that would take you to this site wouldn't mind reading a good handfull of those reviews.

 

[sidewinding]

A historian, an engineer and a statistician are duck hunting. a duck rises from the lake. the historian fires first, and shoots 10' over the duck. then the engineer shoulders the shotgun and shoots 10' under the duck. the statistician exclaimed "got him!".

 

[timg]

I wouldn't mind "Typical" versus "Average". I'm open to making that change. "Decent/Typical"?

 

[timg]

A historian, an engineer and a statistician are duck hunting. a duck rises from the lake. the historian fires first, and shoots 10' over the duck. then the engineer shoulders the shotgun and shoots 10' under the duck. the statistician exclaimed "got him!".

Hah