Ratings Saga Continues

[blarmour]

I don't know about anyone else, but I get tired of hearing and reading about the ratings. Having said that, I just had to post my feelings on this issue I encountered after the latest ratings update.
My friend and I recently played in a charity event, "Support the Troops". This was the only event that either of us played in since the last ratings update. My friend who has played much longer and in more overall tournaments that I, has sat out most of the year working on his doctorate.
Before this event, he was rated at 899 and I at 858. For this tournament, he shot rounds of 936 & 965 (Total of 103 points above his rating) and I shot an 878 & an 864 (Total of 26 points above my rating).
When this last ratings came out on Dec 12th, my rating jumped up 2 whole points while his rating skyrocketed up 1 whole point! How does this happen???
To clarify, I'm not complaining about my rating. I'm totally confused about his rating only going up 1 point after shooting 103 total points above his previous rating!
His last tournament prior to this one was Ledgestone in Aug of 2016, while I had many in this past year. His rating was based on 11 rounds while mine was based on 43 rounds. I know this comes into play in the calculations, but not sure how or why it would affect the rating that much.
I don't have anything against Chuck Kennedy or anyone else at the PDGA, but how long are we going to struggle along with a ratings system that only 1 person on the entire planet understands? I believe this is one of the top things holding our sport back.
As much as people want to say players join the PDGA to support our sport, ratings is the real reason players join the PDGA. If they don't understand the ratings or they don't seem fair, then players will stop joining.
Isn't it time to do away with the current system and come up with a system that makes sense and is easy to understand? I don't have the answer and don't like throwing questions out there without giving a suggestion on how to fix it, but this issue just seems ridiculous to me. Rant over...

 

[brutalbrutus]

This is one reason why I would like to have a way to quickly look up old rounds, instead of shuffling through old tourney pages. There should be a way to compare your previous ratings detail pages with the current one. My guess is that your friend dropped some pretty good rounds when these latest ones became official. Also, he may have had a lower rated round not being counted because its was outside his standard deviation. Then when the older rounds dropped off his deviation changed and that round changed to being counted. Yes the ratings system isn't perfect but with all the variation in courses I don't think it would be feasible to do a slope rating like golf has. That means we have to have a player based rating system. Standard deviation isn't that hard to calculate, especially if you just let one of the websites do it for you.

 

[brutalbrutus]

I looked at that event and went to his PDGA page, it looks like he dropped 12 rounds from his rating. So you can go back and figure out which rounds he dropped and do some math. I was a little curious but not that much...

With that many rounds dropping, they wouldn't even need to be great rounds just a bunch of good rounds would change the average.

 

[brutalbrutus]

I looked at that event and went to his PDGA page, it looks like he dropped 12 rounds from his rating. So you can go back and figure out which rounds he dropped and do some math. I was a little curious but not that much...

With that many rounds dropping, they wouldn't even need to be great rounds just a bunch of good rounds would change the average.

Ok so I am bored.(Not bored enough to go back and do all the math but close...)

I meant he dropped 14 rounds, then added 2 new ones. Ratings can get screwy when dealing with a time off from adding new events. Usually its only events that were within the last year from your latest round. But when you take time off they will include your most recent 8 rounds. It looks like in his case that 8th round was at BG Ams so they are using all 4 rounds from that event, giving him 11 total on his rating right now. One round from that event is 839, so if he had a bunch of rounds in those 14 between lets say 900 and 930, that 839 might fall outside the deviation. Then when those dropped his deviation would have changed quite a bit(if my quick math is right). In that case the 839 gets added back in and balances out the 965 and that's where you only get a 1 point rise in rating.

 

[wericsson]

Wait, so a larger, more current dataset would yield less pathological results? Color me shocked.

 

[DavidSauls]

I don't have anything against Chuck Kennedy or anyone else at the PDGA, but how long are we going to struggle along with a ratings system that only 1 person on the entire planet understands? I believe this is one of the top things holding our sport back.

Quite the exaggeration. Lots of people understand the ratings, to a large degree---the really fine, under-the-hood part that we don't know, generally has a minor effect in our day-to-day dealing with ratings.

It's interesting that you seem to be putting faith in the individual round ratings, but not in how they're averaged to produce a player's rating. Most detractors complain about the round ratings themselves.

Brutalbrutus pretty much covered it, but when you get a surprising update, it's not complicated math. It generally comes from 3 sources: (1) whether recent rounds were submitted in time to be included, (2) what rounds dropped off due to age, and (3) what bad rounds during the period are excluded (that "standard deviation" stuff).

 

[bnbanbury]

So here are the ratings that were included in your friend's rating calculation:
839
890
923
906
895
857
874
862
851
936
965

The average for this is 890.73. The only reason that he gained anything is because the last two rounds are double weighted, so if you add 936 and 965 to the to this list again and divide by 13 you get 899.92 or 900. I was able to do this calculation in a couple of minutes and it worked out to exactly what it should be.

The OP is filled with a lot of hyperbole - ratings are what's holding the sport back and there is only one person on the planet that understands ratings, ect. Figuring out ratings is not akin to hunting theoretical particles, its pretty knowable and relatively straight forward.

The more appropriate question was probably, why are so many old rounds included in this rating update? The PDGA does keep rounds longer than the 12 month period noted if there are zero rounds during a 12 month period. Rest assured though, given a few more rated rounds and updates, those old rounds will fall off and you will see a big rating jump if he continues to play at a 950ish level.

 

[bnbanbury]

I didn't go through the exercise with your rating calculation, only because you have a ton of rated rounds in your calculation. I am confident however, that if you take the rounds that are part of your calculation, add them up, double weight the most recent, and divide that it will come out to your current rating. Your rating is supposed to represent how well you play on average, not how well you played over the last few weeks.

 

[McLovin]

Ratings don't matter...

 

[_MTL_]

As much as people want to say players join the PDGA to support our sport, ratings is the real reason players join the PDGA. If they don't understand the ratings or they don't seem fair, then players will stop joining.

I disagree with this. The real reason most people join the PDGA is it's financially smart too. Why pay $10 5 times for 5 entry fees when you can $50 once, save the $10 fee and get a lot of benefits.

 

[_MTL_]

Ratings don't matter...

Disagree 100%.

Ratings matter for the following reasons:

1. Large pro events have sign ups based on ratings.
2. The pro worlds qualification is rating based.
2A. Even if you say "I'm not a pro" eventually, larger amateur events will have to
follow this model
3. Ratings tell you what division you can't play.

 

[Discette]

I don't know about anyone else, but I get tired of hearing and reading about the ratings. ...
I don't have anything against Chuck Kennedy or anyone else at the PDGA, but how long are we going to struggle along with a ratings system that only 1 person on the entire planet understands? I believe this is one of the top things holding our sport back.
As much as people want to say players join the PDGA to support our sport, ratings is the real reason players join the PDGA. If they don't understand the ratings or they don't seem fair, then players will stop joining.
Isn't it time to do away with the current system and come up with a system that makes sense and is easy to understand? I don't have the answer and don't like throwing questions out there without giving a suggestion on how to fix it, but this issue just seems ridiculous to me. Rant over...

Sorry you don't understand how the ratings work. However, just because YOU don't understand how the PDGA ratings work, doesn't mean other players don't understand it. Chuck is not the only person on the entire planet that understands ratings. Others here did some "quick" math to prove the ratings generated for you and your friend were not mysterious nor flawed.

Next, I do not share the opinion that ratings are holding our sport back. I think ratings are a nice benefit of PDGA membership. Ratings may be one reason some players join, but certainly not the only (or "REAL") reason for most members to join or renew.


In my opinion, it is not time to do away with the current rating system since it is neither ridiculous nor does it need fixing. Most of the stuff you did not understand was explained pretty easily by others on here. You could have also looked here to get answers: https://www.pdga.com/faq

 

[GMcAtee]

I prefer my rating to stay low so I can enjoy the young hipsters getting butt hurt when I beat them.

 

[GoobyPls]

I prefer my rating to stay low so I can enjoy the young hipsters getting butt hurt when I beat them.

You are Kramer beating up on kids at karate class.

 

[brutalbrutus]

You are Kramer beating up on kids at karate class.

More like the Yankees pitcher from Major League who threw at his own kid in a father-son game...

 

[BogeyNoMore]

FTFY...

The only ratings that matter are those from 0.0 - 5.0

 

[GMcAtee]

I get to play against people my age or older now. So now the kids won't get their potential sponsorships ruined.

 

[jeverett]

Hi Blarmour,

Bnbanbury nicely explaining how the actual calculation works, but if it helps any what you're referring to is a phenomenon of rating systems called rating lag. Basically, as players "get better", their rating improves.. but how responsive is the rating system to player improvement? In all rating systems, a player's existing rating has a certain amount of 'weight', which helps determine how much new (different) data/rounds should affect their rating. The PDGA system does this by including (nearly-equally) potentially up to two years' worth of round data in your current player rating.

Switching over to my game design hat briefly, interestingly players actually often don't like 'true' rating systems, as opposed to 'progression' systems. As humans, we like the feeling of accomplishment/improvement.. we like being able to see our rating go up. In 'true' rating systems, however, as the amount of data/rounds increases, the weight of a players' existing rating gets stronger and stronger, until eventually (theoretically) their rating doesn't change at all anymore (barring things like injury/aging): all reasonable new data just confirms the player is performing "as expected" for their rating in the rating system. In progression systems, by contrast, there is much less of an upper limit on how much a player can improve their rating. Many video game rating systems, for example, look much more like progression systems than rating systems.

 

[Nubes]

Disagree 100%.

Ratings matter for the following reasons:

1. Large pro events have sign ups based on ratings.
2. The pro worlds qualification is rating based.
2A. Even if you say "I'm not a pro" eventually, larger amateur events will have to
follow this model
3. Ratings tell you what division you can't play.

I could see this being a huge factor. Ive never played in a disc golf tournament but grew up playing this sport and got back into it big time recently. I want to play tournaments this summer and possibly join a league? I raced bicycles on and off for over half my life and have seen my fair share of sandbaggers and have even been accused of it myself! lol You would get guys who should have been racing Pro/Expert for mtn bike or a Cat 1/2 roadie, racing with the cat 3 guys because they weren't as fit as they should have been or just didn't feel like finishing mid pack or toward the back of the pack that day?? Doping is the biggest problem at the top level of pro cycling but for guys like me who had a day job, the sandbagger may be your biggest threat come race day? If you don't have a way to measure a persons performance level or what they are capable of then you will always have the occasional sandbaggers dropping down a class to win.

 

[ru4por]

Wow....nice rant.

Just because YOU don't understand the ratings system, certainly does not mean we should scrap it for the rest of us who do. I play advanced, so rating don't really mean anything to me, but they are a nice tournament tool. If you don't understand something, try researching the issue and learn about it. Heck, you are lucky here, all you really had to do was ask. Or better yet, search for the couple existing threads on the topic.

I am curious though....what are rating "holding our sport back" from?