• Discover new ways to elevate your game with the updated DGCourseReview app!
    It's entirely free and enhanced with features shaped by user feedback to ensure your best experience on the course. (App Store or Google Play)

How should one analyze a hole's performance?

Doofenshmirtz

Double Eagle Member
Gold level trusted reviewer
Joined
Jul 6, 2012
Messages
1,312
This is a question that has grown out of my other thread about the course I just completed and its performance in its first tournament. I'll start out by sayting that I believe that part of the answer lies in a statistical analysis, but what I am looking for is a less mathematical and more intuitive explanation of what I should be looking for when trying to detemine whether a hole needs adjustment or tweakng (not HOW to tweak or adjust, but WHETHER to do so).

Much of my consideration of this was spurred on by jeverett, who did a statistical analysisi of the blue level (950 rated, or so) players and how they scored on the holes on that course, and to whom I am indebted. Also, some of my concern about that performance is also attributable to Steve West, with whom I disagree about most things par, but who nevertheless gives me much to consider on the subject of this thread.

One of the things that jeverett pointed out about my course was the performance of the different basket positions on a particular hole and its poor scoring spread width in one position and better scoring spread width in the other. The hole(s) are 17 and 17b. Hole 17 had 33.3% birdies and 66.75% pars. Hole 17b had 33.3% birdies, 53.3% pars and 13.3% bogies. Obviously, the better scoring spread width was seen by the "b" position due to the bogies.

The other hole that I wanted to consider from a conceptual standpoint was hole 6. Hole 6 is a gimmicky hole. It has a deep, OB creek within the circle, a few fee past the basket. The course is Kees Park if you want to look at photos of the hole. If you throw OB, there is a drop "spot," that is also just inside the circle, putting straight at OB. The reason for this drop zone was to give a high risk/reward opportunity to those who went OB on the throw and prevent a drop-in par to "reward" a poor tee shot. Conceptually, my thought was that an aggressive, very good throw would reward the player with a drop in birdie. A safe throw and easy lay up would allow a certain 3. A less than good throw by the aggressive player would result in and OB with the chance to make par with a gutsy putt, but with another OB penalty if you are OB on the putt. That was the idea.

So starting with the idea of scoring spread width, what is it exactly? I have been trying to figure out what I want out of a hole and I guess that what I am lookin for is one that requires a particular skill (or some creativity in the case of throws that I did not anticipte), and that results in some separation in scoring between those who make mistakes and those who do not. Should I be okay with a hole that results in 1/3 birdies and 2/3 pars. That strikes me as an okay result. Is it really better if the hole also forces a small number of bogies? Sure, I can see why, from a statistical standpoint, this is a better result, but is that more important than other considerations like the hole just being a fun one (as long as there was some separation in scores?

One of the first holes that I ever designed was a hole that was a dogleg right that began with a narrow fairway and was open after the dogleg. I made the hole too long. No mortal could reach it in two throws without divine intervention, and decent player could par it. It was a kind-of fun hole. Shortening it made possible to birdie and, as a result, more fun and more of a challenge. However, in tournaments, it tends to result in a few birdies and a lot of pars (I don't know the ratio off-hand). That hole has made me tend to equate "fun" with scoring spread to some extent. But danger, like on hole 6 (discussed above), also tends to equate to fun. Even my drop zone gimmickry drew no complaints from players at the tournament or locals who play it often.

What I used to look at the spread of scores was the Mean Absolute Deviation, because this looked like, to me, what I was trying to figure out with respect to a sort-of minimum performance of a hole. Why is that not the correct stat? Why is scoring spread width better? And am I wrong to think that 1/3 birdies and 2/3 pars isn't all that bad? This is certainly no criticism of that measure because I just really don't have a good enought grasp of that the different stats are telling me. But when I used the MAD to look at the scores, it told me roughly the same thing that the SSW told me with resped to the most problematic hole.

Another assumption that I am laboring under is that the lower the rating, the higher the SSW will be for any hole. Please correct me if I am wrong on this.

Finally, as I looked at the hole 7 stats, I saw a difficult hole with a decent width of scores but that was very heartening to me. There were few birdies. But the high number of bogies, to me, was an indication that the hole was encouraging better players to play aggressively and try for a birdie at the high risk of an OB throw. The reason I reached this conclusion is that this hole is an easy par for blue level players. Seriously, it is child's play to par this hole if you can throw over 200 feet with even moderate accuracy.

So if you guys can educate me on these issues (without resorting solely to mathematical equations), I would be deeply appreciative.
 
A. Unless a hole produces too many of one score (usually 3), it's OK. Too many is more than 66%. Less than that, you can start worrying about fun and aesthetics. Until big money is on the line, like DGPT or USDGC.


B. Differences between MAD and scoring spread:

Scoring spread will not give "extra credit" for gaps in the score. 50/50 3s and 4s will have MAD of .5 and SSW of 2. 50/50 2s and 4s will have a MAD of 1, and a SSW of 2. However, either hole only separates "good" from "bad". (You could use Relief Areas instead of OB to fill the gaps so MAD doesn't get confused.)

Another subtle difference is that there is more informational value in rare scores than MAD or intuition would tell you.

In terms of sorting players by performance, the order of the scores doesn't matter. 15/50/35 is the same as 50/15/35. MAD thinks there is a difference.

Having said all that, most holes are typical enough that you won't go wrong using MAD. As you've noticed.
 
With less math, more opinion, I'll start with my judgment that 33/67% split isn't bad.

Scoring spread may show how holes separate the competition but, after that, it takes observation to judge whether the holes are playing well. I have the benefit of being half-responsible for one course, a private course on which we've made plenty of mistakes, but had the freedom to tweak the holes until we liked them.

Houck talks about NAGS, and we were subject to that in the early days -- holes that looked cool, but ended up with a bunch of boring shots, usually long layups. Getting the distance wrong seems to be the biggest culprit, though making OB too severe, so players mostly play safe to avoid it, will do it too.

The other end of the spectrum on scoring spread is "luck", or at least, holes where the scoring spread doesn't favor the better players. You'd probably need a lot of numbers to quantify it, but ought to be able to sort it out with observations.

More casually, on a good hole, a player should stand on the tee uncertain of what his score will be, and that uncertainty based on his decisions and execution. Scoring spread may reveal the uncertainty, but I'm not sure it'll tell the reasons.
 
A few quicks thoughts, for what they're worth.

This is a question that has grown out of my other thread about the course I just completed and its performance in its first tournament. I'll start out by sayting that I believe that part of the answer lies in a statistical analysis, but what I am looking for is a less mathematical and more intuitive explanation of what I should be looking for when trying to detemine whether a hole needs adjustment or tweakng (not HOW to tweak or adjust, but WHETHER to do so).

I think you know this, but not everyone does, so it's probably worth saying again: good scores do not necessarily indicate a good hole. But good holes will generate good scores.

So...

Hole 17 had 33.3% birdies and 66.75% pars. Hole 17b had 33.3% birdies, 53.3% pars and 13.3% bogies. Obviously, the better scoring spread width was seen by the "b" position due to the bogies.

Those stats do not indicate that 17b is a better hole. You need to look at other factors. Again, I think you know this.

A few statements that always raise a red flag for me are

... is a gimmicky hole.

and

A safe throw and easy lay up would allow a certain 3.

and

Seriously, it is child's play to par this hole if you can throw over 200 feet with even moderate accuracy.

If yoiu can help it, you don't want those statements to be true about your holes.

I saw a difficult hole with a decent width of scores but that was very heartening to me. There were few birdies. But the high number of bogies, to me, was an indication that the hole was encouraging better players to play aggressively and try for a birdie at the high risk of an OB throw. The reason I reached this conclusion is that this hole is an easy par for blue level players.

I don't know the intricacies of your hole, but designers in this situation do need to be careful. There are holes where many players will go for the risky shot and bogey just because the safe play is too boring. These holes are not good holes.

Hope some of that is helpful to you. It's great to see that you're being so thoughtful working so hard to get it right. We need more designers to do that.
 
Here's an article I wrote a while ago on course design validation which pertains to the line of inquiry in this thread.

Nice read; thanks for sharing. Could you please help me understand this bit of score-distribution-analysis theory?

Holes in the 3.1-3.4 and 4.1-4.4 range primarily help separate the bottom third of the players from the top two thirds. However, for league and tournament play we want more holes to help separate the top third from the bottom two thirds. Holes with scoring averages in the 2.5-2.8, 3.5-3.8 and 4.5-4.8 range are much better at doing that.
 
Nice read; thanks for sharing. Could you please help me understand this bit of score-distribution-analysis theory?
First, let's assume Par has been set to match the skill level of the players competing. Let's also assume that the scores thrown on the holes correlate reasonably well with the skill levels of the players such that on average, the better the player, the better their score. Unfortunately, golf scoring is currently just integers. 1 (rare), 2, 3, 4, 5 primarily with sometimes 2 (open holes), 3 (moderate number of obstacles) or 4+ (wooded or significant OB) of these values thrown on a hole by players within a skill/ratings range.

Take the example where mostly 2 values are thrown on a mostly open hole say 2/3 3s and 1/3 4s for a 3.3 average which is in the X.1-x.4 range mentioned in my article. Most of your top-rated players are getting 3s and your lowest rated players in the field are getting 4s usually because they lack the distance of the higher rated players in the field. You probably know holes like this. Point being that this hole separates the lower part of the field from the expected contenders with almost no separation among the contenders except the few who really play poorly on getting down in 3.

The opposite occurs when the split is 1/3 2s and 2/3 3s with an average of 2.7 which is in my proposed better range of x.6-x.9. Statistically, the top contenders mostly get 2s and the remainder get 3s and are less likely to contend. Of course, we still need to separate the contenders who shoot 2s into the different paid positions (assume top 1/3 paid out). We do this by playing enough holes like this such that enough of the contenders will falter, one, two, three, four times such that a winner emerges who faltered the least number of holes.

Note that even with the shorter holes and courses in the 80s, 90s and early 2000s where there were more contenders who could reach all of the holes on a course in regulation, there are few events where there was a playoff required or with more than 2 players. Play enough holes and a single winner emerges, even without many holes with wider scoring separation. You'll also have more players staying in contention for a longer time with a higher chance of a playoff.

I think that's what spectators want to see with a big exception. I think we all want to see hole variety. So, the challenge for those designing courses for elite play on video is how to create a variety of holes where the elite players have a chance to birdie without wading though a repetitive series of open holes, especially those that artificially create pseudo separation with tons of OB.
 
Note that even with the shorter holes and courses in the 80s, 90s and early 2000s where there were more contenders who could reach all of the holes on a course in regulation...

Chuck raises an important scenario. I'd be interested to hear anyone's examples of good holes where not all contenders can reach the hole in regulation. For the purposes of this exercise, let's define a "contender" as someone who has a decent chance of cashing in his/her division on any given weekend. Thanks.
 
Chuck raises an important scenario. I'd be interested to hear anyone's examples of good holes where not all contenders can reach the hole in regulation. For the purposes of this exercise, let's define a "contender" as someone who has a decent chance of cashing in his/her division on any given weekend. Thanks.

[To trace the evolution of the idea here a bit]

Per Chuck's article (& general course design theory), one characteristic of a good disc golf hole is that it hands out a "good mix of different scores" to players of an intended skill level.

So, for some target player group, if the reachable-in-regulation (RIR) holes produce a greater mix of different scores than the not-reachable-in-regulation (NRIR) holes, then the RIR holes probably contribute more to sorting the players. Some evidence that this might be true is:

Note that even with the shorter holes and courses in the 80s, 90s and early 2000s where there were more contenders who could reach all of the holes on a course in regulation, there are few events where there was a playoff required or with more than 2 players. Play enough holes and a single winner emerges

The question then arises: do "good holes where not all contenders can reach the hole in regulation" exist? This question becomes especially significant if we assume that a trend of modern course design is toward more NRIR holes. (This seems to be true for the everyday amateur, but I'm not sure if it's also true for the intended skill levels of the courses. Has the increase in distance/ skill of the top modern professionals infected course design intended for less skilled players?)

I think these holes do exist, but they are probably rare & difficult to design. They likely require uber-expert distance, or maybe uber-expertise in a specific, non-traditional shot shape like a flex-forehand or roller. I think that Steve's point from earlier in the thread might apply here:

Another subtle difference is that there is more informational value in rare scores than MAD or intuition would tell you.

Examples might be:

Maple Hill Gold #10 (uber-expert distance):

9Qa3Io.gif



Wzcz4T.gif



Hole #2 Northwoods Gold 2019 Layout (uber-expertise in a specific, non-traditional shot shape):

WRIjDW.gif



A70OAn.gif
 
Forgot to include:

In addition to the informational value provided by rare scores handed out by NRIR holes, these scores also seem to be a focal point of post-play player discussion (just as a Hole-In-One might be). I'm not sure if this indicates quantifiable hole value, but the rare scores produced by NRIR holes seem to be irresistibly intriguing.
 
At the core of this discussion is how a player's distance should be viewed in terms of being a skill versus a maxed out physical trait that's a result of genetics, age and optimizing your mechanics. If you believe any person is capable of approximately the same golf distance forehand and backhand then holes of all distances and designs will be "fair" by default because everyone would have the potential to reach those max distances with enough training and practice in the same way most people have the ability to move chess pieces, even those "taxing" diagonal moves up to seven squares.

At the basic level, consider your max golf distance in the open (level ground, not rollable). If you want the occasional chance to birdie a hole, the basket can't be more than say 20 feet longer than your max D. If you want the chance to ace, the basket needs to be at least 20 feet shorter than your max D so your disc's flight can still be high enough off the ground to get in the basket.

Thinking in terms of elite level play, if you want X number of players to potentially be contenders primarily based on demonstrating a mix of other skills and not distance, then the maximum distance for a birdieable par 3 is 20-30 feet more than the shortest distance throwers in field X. Even having just one or two holes longer than that distance discriminates against the bottom level in field X. Even if the longer throwers don't birdie those holes every round, they are the only ones who realistically can do so. And it only takes one more birdie in 3-4 rounds to win.

Is it any surprise that the 1040+ guys are usually the top finishers on ball golf layouts with their finish order sometimes "sorted" as much by the number of OBs/rollaways as other skills? I think a player's "greens in regulation" (GIR) stat is the one that correlates closet with finish position. Although GIR in theory should be based on accuracy, if you can't reach the green in the first place, your accuracy or lack of still lowers your GIR stat.
 
At the core of this discussion is how a player's distance should be viewed in terms of being a skill versus a maxed out physical trait that's a result of genetics, age and optimizing your mechanics. If you believe any person is capable of approximately the same golf distance forehand and backhand then holes of all distances and designs will be "fair" by default because everyone would have the potential to reach those max distances with enough training and practice in the same way most people have the ability to move chess pieces, even those "taxing" diagonal moves up to seven squares.

Although it's not obvious to me what the exact physical traits are that result in enhanced distance potential, it seems that, generally, a leaner frame with longer limbs certainly doesn't hurt.

That said, Pierce seems like a good counter example. And before we had the current crushers at the the top of the tour, Paul McBeth is only 5'8", with a fairly standard frame. Emerson Keith seems like an even further outlier (not that his distance is top of tour, but it's not as if he can't get to long holes.)

Whether that means they just hit some other genetic lottery, or that unlocking the code to form is the real secret sauce, I don't know. Maybe if it was something like ratio of hip to reach width Pierce would at least score high?

But there are also a couple of related questions, which is whether one wants raw distance to be the "one big stat" that dominates other aspects of the game? And then, is it really even possible to remove distance as an advantage if you are going to have holes that are designed to require more than one shot to get to the basket?
 
I don't think we'll ever be able to do much get the best players to be drawn from any population other than the longest throwers.
 
Some summary stats for scoring distribution on a new layout. 950 +- 35 ratings point target group. ~ 95 observations. ~ 50% sanctioned rounds.

BKS21-Gold-Hole-Summary.jpg



The course is Brookside Park: https://www.dgcoursereview.com/course.php?id=349

Course characteristics are City Park Disc Golf - mature trees skirted w/ thick, low branches, grassy fairways, OB park roads & pathways, man-made elevation etc. etc.

#3 is a downhill island green.

#8 has a sloped green w/ OB long.

#18 has an OB road boundary on the left that's in play for about ~600 feet. Fairway is pretty wide/ generous. Right side is thick woods surrounding a city creek. There's an OB path on the right side of the fairway that starts at ~700 feet and continues past the green. (Long approaches can find this, but missed putts do not.) The basket is set on a ridge. Score frequencies:
3 1.05%
4 26.32%
5 48.42%
6 17.89%
7 3.16%
8 3.16%


#5 has a pretty severe left-to-right movement up a small hill.

#16
#17
#20
All in ~ 370 ft. range; based on spread, these seem to be just out of the comfortable distance range for the players in this sample. (#17 has an OB pathway on the right-side that comes into play because the fairway has a slight left-to-right movement, so it gives out more 4s).
 
Top