[Request] Let's Settle The Ratings Mystery

[DiscFifty]

So for a few years now, I've been asking about a simulated scenario similar to this:

MPO Division: 50 players all rated 1000.
REC Division: 50 players all rated 800.

Both play the same course, except REC plays 1 hole shorter. This way each division is rated on it's own, which is the key for this scenario.

Both divisions, all players shoot the same score of 54. What is the round rating going to be for each division?

The public answer from those in the know, has always been "this scenario will never happen due to the difference in skills sets". Well... what IF it actually happened?

I talked to a TD today who is going to work with me and try to get a clear answer, but it's not going to happen for a few weeks or so because he is out of town. (He's very curious about this as well. lol..) So I thought I would post this here and see if any other TD wants to plug in this scenario using their TD tools and post the results. opcorn:

 

[Armus Patheticus]

So for a few years now, I've been asking about a simulated scenario similar to this:

MPO Division: 50 players all rated 1000.
REC Division: 50 players all rated 800.

Both play the same course, except REC plays 1 hole shorter. This way each division is rated on it's own, which is the key for this scenario.

Both divisions, all players shoot the same score of 54. What is the round rating going to be for each division?

The public answer from those in the know, has always been "this scenario will never happen due to the difference in skills sets". Well... what IF it actually happened?

I talked to a TD today who is going to work with me and try to get a clear answer, but it's not going to happen for a few weeks or so because he is out of town. (He's very curious about this as well. lol..) So I thought I would post this here and see if any other TD wants to plug in this scenario using their TD tools and post the results. opcorn:

Then there would be a lengthy and confusing playoff.

 

[Jugular]

For MPO the SSA would be 54. Unless there's a constant/fudge factor applied, or there's a possibility this special case isn't well dealt with in the calculations and no SSA could be calculated.
For REC I could imagine that there would be no SSA calculated since the curve of best fit (or equivalent calculation) doesn't have sufficient information, but it's possible there is a default rating points/stroke of 10 that would mean the SSA would be 34.

I'm not sure where this takes you though.

 

[DavidSauls]

All the MPOs shot 1000-rated rounds, and all the RECs shot 800 rated rounds.

If that sounds like a ludicrous result, it's because it comes from a ludicrous scenario. GIGO, in other words.

 

[DiscFifty]

All the MPOs shot 1000-rated rounds, and all the RECs shot 800 rated rounds.

If that sounds like a ludicrous result, it's because it comes from a ludicrous scenario. GIGO, in other words.

Of course it's ludicrous, all this is, is an experiment to see what the ratings algorithm would spit out.

"All the MPOs shot 1000-rated rounds, and all the RECs shot 800 rated rounds."

Is that the final answer then?

 

[Jugular]

It's what the algorithm should return but, like I say, with such reductive data some of the calculations may encounter errors.

 

[Horsman]

All pros would shoot 1000 and all ams would shoot 800. Its unfortunate but I think its right, who knows

(50*1000)/50=1000 Average Player rating is 1000
(50*54)/50=54 Average round score is 54

Therefore a round of 54=1000 and all pros would shoot 1000 rated round

 

[krupicka]

The algorithm will spit out the average rating for the average score. That is the way it is designed. You don't need to put in any tools to figure that part out.

 

[Cgkdisc]

If a course was short enough to where a group of 1000 rated players and a group of 800 rated players both shot a score of 30 for 18 holes, the math would produce the same ratings for both groups. So we're talking about a putter type "course" where each player threw at least 6 aces.

The ratings system can mathematically produce the numbers but they don't effectively resolve skill levels of our highest rated players when the SSA of a course falls below 41 due to severe birdie score compression with 36 being a realistic bottom limit. This compression effect is why the PDGA for the past 15 years or so has recommended courses with SSA over 49 be used for MPO in higher tier competitions.

 

[araytx]

Of course it's ludicrous, all this is, is an experiment to see what the ratings algorithm would spit out.

"All the MPOs shot 1000-rated rounds, and all the RECs shot 800 rated rounds."

Is that the final answer then?

PDGA Player Ratings is NOT a criterion-referenced system. Therefore, the "hypothetical scenario" you set up here is not an appropriate use of statistical analysis. AKA, it doesn't fit the parameters of statistics from which a conclusion can be drawn. Can you "plug in these numbers you set up into the formula and have it spit out something"? Yes. Will that information, once spat, be valid and reliable? No.

You need to hook up with Brian T. Usher. We have this same conversation all the time. He thinks that when he TDs an event, he can physically see the "ratings change" as he enters more and more data. But nope. They didn't "change." Until all the data was entered, they weren't ratings -- they were something else.

 

[DiscFifty]

Thanks for the replies. So.....let's say the top elite players ONLY decide to play DGPT events. And these events only have 2 divisions, MPO & FPO. We know each division is going to be rated separately.

It seems like there is the possibility of some kind of bubble effect where the average player rating would always go up and never down.

For instance.....player A wins a few events, over time his rating increases. Now player A cools down a bit, doesn't win but he still shoots well enough that it really doesn't affect his rating. But...here comes player B (or C, D) with a win (or 2) and it causes his rating to rise.

So..if the above scenario repeats over and over....in effect the top elite players taking turns near the top, it just seems all of them will benefit because the "ratings points in" will continue to rise, hence... their player ratings will also rise. opcorn:

 

[Jugular]

Thanks for the replies. So.....let's say the top elite players ONLY decide to play DGPT events. And these events only have 2 divisions, MPO & FPO. We know each division is going to be rated separately.

It seems like there is the possibility of some kind of bubble effect where the average player rating would always go up and never down.

For instance.....player A wins a few events, over time his rating increases. Now player A cools down a bit, doesn't win but he still shoots well enough that it really doesn't affect his rating. But...here comes player B (or C, D) with a win (or 2) and it causes his rating to rise.

So..if the above scenario repeats over and over....in effect the top elite players taking turns near the top, it just seems all of them will benefit because the "ratings points in" will continue to rise, hence... their player ratings will also rise. opcorn:

The only way to win is to shoot the best round. The best round will be the highest rated round. Given the way it's described it appears it is only the rating of the round that impacts the player rating over time. Whether or not each round represented a win or not doesn't matter beyond the rating assigned to the round.

Is there any evidence of a ratings bubble? Is PMcB's rating that much higher than ratings possible in 1998? Ratings drift is possible in many rating systems but either by good design or luck it doesn't appear to be the case in ratings for the DG population as a whole.

 

[Rastnav]

So..if the above scenario repeats over and over....in effect the top elite players taking turns near the top, it just seems all of them will benefit because the "ratings points in" will continue to rise, hence... their player ratings will also rise.

I'm not saying you are wrong, but I don't understand from whence you are drawing the idea that "rating points in will continue to rise". I think the only way that could happen is if the existing pros never fell in ratings and dropped out, and the only players who came into the bubble and popped out were lower tier players who came in, underperformed their rating when they came in, and then went back "down" to a different tier (and therefore their ratings points lost were added to the pool).

But that effect should be offset by new players coming in and grabbing rating points from existing players who are on the decline, but then take the rest of their points out of the pool. They won't leave all of their points in the MPO/FPO pool.

(Again, if I'm following all this correctly).

 

[Dave242]

Well... what IF it actually happened?

I talked to a TD today who is going to work with me and try to get a clear answer, but it's not going to happen for a few weeks or so because he is out of town. (He's very curious about this as well. lol..) So I thought I would post this here and see if any other TD wants to plug in this scenario using their TD tools and post the results. opcorn:

I used to do the score keeping for a pretty good number of PDGA tournaments, and I used real life results to run through the TD tools very similar to what you are suggesting. I did all sorts of scenarios to analyze the typical questions (myths....some of which are showing up here) and also to see how different course styles challenged different skill groups, and to see how different holes/designs scored.

Overall, given a group of over say 12-15 scores, things were very stable across pretty much every scenario. While some people got all torqued about variations of 5 points, I had to keep reminding them that is only half a throw/stroke.

There were indeed a few outlier scenarios/results (1-1.5 throws worth of ratings/SSA points).....but it seemed about half of them went the direction opposite of what conventional wisdom would dictate.

Tinkering with this stuff is really interesting and fun to do. I'll be curious to see what you discover.

 

[Hampstead]

Par talk > player rating talk

 

[DavidSauls]

Thanks for the replies. So.....let's say the top elite players ONLY decide to play DGPT events. And these events only have 2 divisions, MPO & FPO. We know each division is going to be rated separately.

It seems like there is the possibility of some kind of bubble effect where the average player rating would always go up and never down.

For instance.....player A wins a few events, over time his rating increases. Now player A cools down a bit, doesn't win but he still shoots well enough that it really doesn't affect his rating. But...here comes player B (or C, D) with a win (or 2) and it causes his rating to rise.

So..if the above scenario repeats over and over....in effect the top elite players taking turns near the top, it just seems all of them will benefit because the "ratings points in" will continue to rise, hence... their player ratings will also rise. opcorn:

I believe that if it's the always the same players, it's a zero-sum game. The average rating will never change. Wins have no effect; only scores.

 

[Countchunkula]

When have we settled anything on dgcr?

 

[Putt for D'oh]

When have we settled anything on dgcr?

When there isn't actually any Mystery?

 

[araytx]

I'll say this til I'm am blue in the face (and if you know me you realize what a really long time that is):

PDGA Player ratings are not directly and independently related to the course. They have an SSA for the day (layman's terms is "how the course played"), but that is NOT independently about the course. Never.

 

[DiscFifty]

I'm not saying you are wrong, but I don't understand from whence you are drawing the idea that "rating points in will continue to rise".

Let's say Paul plays 5 tournaments, shoots the best score and ultimately his rating goes up. Now he skips the next 5, your rating doesn't change if you don't play. So now someone else (or several people) play well enough during the next 5 tournaments (that Paul doesn't play) and their rating goes up. So...the next time all of them play in a tournament the "ratings points in" has increased. Is this not plausible?

I think I understand now why some local pros say "c tiers are rating killers". Probably because you have so many chances for low rated ams to shoot lights out which would affect the pros rated round if they shot the same thing. Probably also explains why they would prefer MPO to play longs and the ams from the shorts to keep the rounds rated separately. hmmm....