Course Modification Possibilities, Plantation Ruins

[Steve West]

Enjoy.

 

[bikinjack]

I just skimmed through that and I'll have to revisit it when I'm more awake, but its quite interesting.

 

[BraveThrower43]

nevermind

 

[grodney]

Assuming hole #13 is designed as a 1-shot hole, then the only 9% birdies would make some think it is a bad hole FOR THIS SKILL GROUP. Some cursory studies have shown that this skill range makes about 50% of their putts. If that's right, that means only about 18% of players were putting after their drive. Some would argue if that few players are getting to the "green" in 1 shot, then it's not a very good 1-shot hole. Yes, it has a wide scoring spread, but is it for a good reason?

Same argument (minus the scoring spread) would apply for #11 at only 11% and #14 at 15%.

 

[1978]

So you can use the synergy discussions to determine if holes are easier/harder based on similar features... so if I took every hole and made a feature list. Left turn, RT, up, down, fast green, open, narrow, tee box length, bench at tee pad, distance of bench to tee pad...etc... you could use this information to infer a lot about what makes up difficulty in DG hole...by assigning attributes to holes you could get a synergy score for 2 hole partners, 3 hole, 4 hole... using a matrix grouping them by shared attributes...?

Does that make sense?

 

[Dave242]

Enjoy.

Fascinating analysis!

I am afraid however that a large part of the analysis that revolves around luck/randomness/non-correlation is inherently flawed. The reason I say that is that "skill" is not a homogeneous data-point/variable.

Disc golf skill is made up of Distance, Accuracy, Putting and Course Management (self-awareness of the first 3 skills listed....and decision making based on odds-making of risk/reward choices).

Across a field of players, not all players have the same combination of these skills.....so Luck/non-correlation will possibly register much higher than reality. The coin-flip comments do not necessarily hold water.

What might be interesting/meaningful is to find the players that had the best scores on the longer/open holes and then do the same with the tight/technical holes. From there you have 2 groups of players - those with predominant skill of D and those with the predominant skill of accuracy. Then, run separate analysis of the course for both of these groups.

You still have not isolated the skills of Putting and Course Management, so that will cloud the results....but I still think that analysis would be a more fair assessment of the course.

 

[tom12003]

Enjoy.

I have to thank Steve for a massive effort in analyzing the detailed hole-by-hole #s that I sent him for the four MA1 rounds at Plantation Ruins during the 2012 Worlds (also have to thank BigDawg and the PDGA in sending me all the scorecards).

Many of the stats provided a different perspective, but I'm not really sure what all the statistics mean from a course design/improvement perspective. Case in point, the stats indicated that hole #14 showed a very low scoring separation; its about a 310' par 3 with either an open RHBH turnover path or a tighter, but still open, RHBH hyzer path. The hole was designed as "relief" from the previous 7 tight wooded holes--specifically from #13 (the hardest scoring hole). Stats indicated a desire to make #14 harder; our best option would be plant a tournament sleeve at about a 380-390' distance while keeping it a par 3. My opinion is that the 2012 MA1 Worlds field would have scored about the same (they were already a "select" field and added distance wouldn't have made any difference). From data that Steve did not have, is that the AM Masters, GrandMasters, and Sr GrandMasters fields showed a progressively higher scoring average but about the same separation (read as one ages distance adds throws). So increasing the length doesn't do much for separation statistics. Lengthening hole #14 to a 500' category would only make it a weak par 4.

I could say the identical comments about hole #3 (the other "open" hole on the course); tweaking the distance (by 80-100') probably would not change scoring separation (I've marked a test tee pad that makes this 301' hole to about 385'; test throw scores haven't radically change scoring).

Some of Steve's statistical analysis revealed needs for some tweaks on holes that might make sense, but not a lot. Sam Nicholson (designer) and myself (chief grunt) built options into each hole to make the course MUCH harder; many are doable, but I'm guessing they will not provide any significant separation of a given field of players. The Charlotte DGC recently weighed in with the opinion of WHY change--if players want the super challenging course that beats them into submission they should go 8 miles to play Renny Gold; if they want a fun course with challenges, the current Plantation Ruins course is about where it should be.

 

[Steve West]

...The Charlotte DGC recently weighed in with the opinion of WHY change--if players want the super challenging course that beats them into submission they should go 8 miles to play Renny Gold; if they want a fun course with challenges, the current Plantation Ruins course is about where it should be.

I think my analysis could be used as support for leaving the course as it is. I don't know for sure, because we don't have similar numbers from other courses to compare to.

But, if there were some urgent need to fiddle with the course, then the numbers may point to the holes that have the least chance of messing things up if they are tweaked, or at least indicate the other holes that should not be touched.

#13...Yes, it has a wide scoring spread, but is it for a good reason?

What's a "good reason"? I showed that it does the best job at sorting out these players. What else is a hole supposed to do during Worlds?

So you can use the synergy discussions to determine if holes are easier/harder based on similar features... so if I took every hole and made a feature list. Left turn, RT, up, down, fast green, open, narrow, tee box length, bench at tee pad, distance of bench to tee pad...etc... you could use this information to infer a lot about what makes up difficulty in DG hole...by assigning attributes to holes you could get a synergy score for 2 hole partners, 3 hole, 4 hole... using a matrix grouping them by shared attributes...?

Does that make sense?

I'd find it very difficult to classify holes that way. Especially after watching a group of five players all use different throws to take different lines. Even if you could, there would be too many attributes and too little data.

...I am afraid however that a large part of the analysis that revolves around luck/randomness/non-correlation is inherently flawed. The reason I say that is that "skill" is not a homogeneous data-point/variable.

Disc golf skill is made up of Distance, Accuracy, Putting and Course Management (self-awareness of the first 3 skills listed....and decision making based on odds-making of risk/reward choices).

..and knowledge of the rules, willingness to make calls, staying sober, Zen, sharp eyesight, depth perception, 3D mental modeling, stamina, focus, clothes that move well and keep you comfortable, getting a good night's sleep......

I agree with you as far as talking about using correlation to measure how good a hole is. It's OK for now. At our current primitive level of knowledge about how to design holes, it can probably teach us something.

However, there is an inherent danger in the long run. Say a hole is really good at testing one skill, so it is labeled a "good" hole, and then other holes are built like that, and they look "gooder" because more and more holes are built like that and they all agree with each other about which players are more "skilled". Relying purely on correlation, eventually we would get holes that only test one skill.

However, with Scoring Spread (and the measures of luck built on it), that doesn't happen. If an odd skill is tested by a certain hole, it shows up in a wider scoring spread for the total scores for the field (because Scoring Spread is a measure of how much information is in the system).

If a hole assigns scores randomly, it shows up as a narrower Scoring Spread.

If a hole tests the same skills as other holes, the Scoring Spread doesn't get as much wider as it would from a hole that tests a different skill.

I added the correlation charts just to show that Scoring Spread is not in general disagreement with correlation. I think Scoring Spread is a less-flawed lens to look through to find the same goal

 

[grodney]

What's a "good reason"? I showed that it does the best job at sorting out these players. What else is a hole supposed to do during Worlds?

A "good reason" that scoring spread exists would be that the hole is designed so that players can exhibit skill and separate themselves from those who don't.

A "bad reason" would be the hole is, for instance, too tight. (There are other bad reasons that a wide scoring spread might exist.)

I think we ran into this argument with you before.

If you have a 250 ft hole that looks like a plinko board, with small trees and smallish gaps between them, then you will get a HUGE scoring spread on that hole. Does that make it a good hole because, as you say, "it does the best job at sorting out these players"? Absolutely not.

If you could correlate the actual scores to the known skill of the players (as Chuck did at the course level), then maybe yes. But just scoring spread alone doesn't tell you what you need to know.

 

[Cgkdisc]

That's why Scoring Spread needs to be looked at in tandem with the correlation to see if the spread is random compared to rating/skill like a pinball hole or if the scores correlate with the player skill level.

 

[tom12003]

Grodney is referring to #13 at the Plantation Ruins. Its uphill (257' but plays in the low 300s') with a 9.5' wide gap about halfway (when it was designed that gap was only 5' but we realized that was too punitive and those cedars disappeared when the course opened last January). Many look at the hole and say I can deuce this (testosterone driven) but can often get in real trouble, really fast (resulting in 4s or worse); only a few succeed (I have seen it aced); if not accurate this hole will be frustrating and you'll call it luck. This hole tests course management skills more than any other--an accurate 160' drive in the middle followed by a decent upshot usually results in the hoped for par; although I'm a noodle arm, I manage my shots--average about 3.2 on this hole (missed putts or a few bad tree kicks; alternatively I have a few field deuces)--although I've played this hole hundreds of times. I see nothing wrong with one hole on a course that demands both precision and course management skills; otherwise we'd all be playing in open parking lots where one succeeds by simply out throwing their competition.

 

[Steve West]

If you have a 250 ft hole that looks like a plinko board, with small trees and smallish gaps between them, then you will get a HUGE scoring spread on that hole. Does that make it a good hole because, as you say, "it does the best job at sorting out these players"? Absolutely not.

If you could correlate the actual scores to the known skill of the players (as Chuck did at the course level), then maybe yes. But just scoring spread alone doesn't tell you what you need to know.

Actually, Scoring Spread alone does tell you if the hole is giving out scores according to skill. Less luck is one cause of a wider Scoring Spread (the specific statistical measure I calculate, not the general undefined notion of what a "wide" scoring spread is).

When I say "best job", that phrase includes sorting the players according to skill. Giving out a lot of different scores is one thing. Giving them to the players who can play that hole better is another. A hole will only rarely be able to widen the Scoring Spread of total scores much just by luck.

It is easier to see when you look at the contribution to the Scoring Spread of the total scores and how that relates to the contributions those same scores would have made based on pure luck.

Look at hole #13. As shown in the paper, if it assigned the exact same scores without regard to skill, it would have contributed anywhere from -.67 to .46 to the Scoring Spread of total scores – and 65% of the time the contribution would have been negative. The fact that it made a much larger positive contribution says that luck was not involved (or, more accurately, there is only a one in 10,000 chance that pure luck would generate this much contribution). We don't need correlation to know it wasn't assigning scores randomly. In fact, we know it was assigning scores less randomly than any other hole on the course.

Correlating scores to ratings only tells you if the hole is testing the predominant skills that went into the ratings. Most holes should do this. However, if there were a hole that tested a rarely-tested skill (like the wisdom to lay up, maybe?) it might not correlate well with ratings. However, that doesn't mean it is because of luck.

If you have the same group of players play the hole over and over, and you see the same players doing better more of the time, then that hole is testing a skill, not generating random scores. If it happens to not correlate well with ratings, then you know it is testing a skill that is rarely tested on existing courses. Scoring Spread analysis will find that it is testing a skill, without the need to look at correlation.

If there were such a thing as a hole that has gaps so small that it won't be the same players getting lower scores more of the time, then that hole would have a narrower contribution to the Scoring Spread of total scores (more precisely, a contribution within the range possible from pure luck).

What would probably happen way before a hole got to the point of having gaps too small to test skill (if that is even possible), would be that players would whine about it so much that it would get "fixed" by taking out some trees. Because there would be many players who think they are "good" because they don't face certain tests very often. It would irk them that other players who are supposedly "not as good" can consistently out-play them on that one hole.

If we rely on correlation with ratings as the measure of whether a hole is testing skill, we are doomed to designing only holes that re-test the most commonly tested skills.

 

[grodney]

.ok.