Hole Enjoyment Survey - Creating a measure

[ChrisWoj]

Good morning to anyone interested. We started a brief discussion on this project in the Par Talk thread, and it was requested by a few that we break it out into its own thread. What I'm attempting to do is create a measure of hole enjoyment. One of the things we're good at in general with measurement of sensation/experience like this is ranking. What we're bad at is estimating appropriate meaning from the differences in ranking.

This URL takes you to the PDF shared via Google Drive:
https://drive.google.com/file/d/145Uu4vUqHyfcrNPxiYRMnJ7vUfc-edBX/view?usp=sharing

This is a trial instrument. This instrument isn't a final version. My plan is to trial this instrument at a few local courses. The goal is to identify a set of prompts that function to measure a single construct. Once those have been identified - then the prompts that emerge need to be looked at critically to determine if they are, indeed, looking at the trait we want it to. I will be working with Rasch analysis to convert the results, which are in ordinal form, to something with abstract invariance associated with the intervals. The idea is to create a reasonable facsimile that tells us how much better one hole is than another, in relatively certain terms (key word in there: relatively, I am aware that measuring human sensation is fickle - probabilities are at the core of using this as a measure).

Some things have already been brought up in the other thread:

1. Long term I would like to see this used at multiple courses in regions with more diverse terrain than my home course. That would give me the chance to look at its results alongside the DGCR favorited hole statistics. This helps determine if we have convergent validity - if it functions to rank in a manner similar to the favorite hole scores, we can begin to trust that it is measuring some element of enjoyment.

2. I would also like to compare performance by relevant demographics: how do players at one skill level, or who throw with one dominant throw measure the same holes against others.

3. If what results can be established as a consistent tool, I would like to see it used by players on tour to see how the enjoyment factors line up with some of the numbers others like Steve West have produced on how the holes play.

At the moment, the most important thing is that the survey I'm trialing has construct validity: if, at a glance, an item on the survey doesn't appear to measure in the end some element of enjoyment (or, as one thread refers to it: fun factor) - let me know. This same thought process will be applied after the first analysis is done, but if its clear I chose something really dumb - why include it? So let me know.

Thoughts? Questions? Expletives?

 

[DavidSauls]

I'm not sure how you can add it in, but one factor I think of is uniqueness.

Is this hole significantly different, or very similar, to other holes (1) on this course, (2) that I play regularly, or (3) that I have played.

Subjectively, many of my favorite holes stand out because of their unique qualities.

Some of the worst, too, I guess.

 

[PMantle]

The goal is to identify a set of prompts that function to measure a single construct. Once those have been identified - then the prompts that emerge need to be looked at critically to determine if they are, indeed, looking at the trait we want it to. I will be working with Rasch analysis to convert the results, which are in ordinal form, to something with abstract invariance associated with the intervals. Questions?

Yes, what line of work are you in?

 

[ChrisWoj]

Yes, what line of work are you in?

Fair question - at the moment I'm an academic and a grill cook. The former a bit more relevant than the latter, though the latter involves walking in circles a lot - apply that how you may.

In terms of academic background - I'm in a PhD Research & Measurement program - basically researching the nuts and bolts of research - primarily focused on development of best practice. Measurement using Rasch is one of the primary focuses of the department, so I get a lot of it. This is more an opportunity to kind of use that in something I enjoy - in a way I want to develop a way to integrate what I enjoy about my professional life into my favorite hobby.

My own focus is on the use of algorithms for sample matching to minimize confounding factors in cases where experimental research is not possible. An example: identifying the success of educational interventions, where I'm looking at broadening our understanding of the appropriateness of using propensity scores and coarsened exact matching as viable strategies. The Pro Tour is getting close to having enough numbers available to match competitors year over year on how they've performed so that we could really discuss the impact of changes to the courses - though I'd still like additional demos like distance, throw/hand dominance, etc. I may look at that in the future.

Overall I'm still kind of feeling out where my hobby and my expertise are going to fit together. I'm hopeful that this project and really yield something fun and lead to a good little extended line of hobby research.

 

[SonicGuy]

Good morning to anyone interested. What I'm attempting to do is create a measure of hole enjoyment. This is a trial instrument. This instrument isn't a final version. The idea is to create a reasonable facsimile that tells us how much better one hole is than another, in relatively certain terms (key word in there: relatively, I am aware that measuring human sensation is fickle - probabilities are at the core of using this as a measure). I would also like to compare performance by relevant demographics.

Thoughts? Questions? Expletives?

 

[ChrisWoj]

:thmbdown:
Sadly this will prove to be a difficult demo to work with, see attached.

 

[BuzzzChief]

One thing that I don't see addressed is the "visual appeal" of a hole. I think in most cases, the signature hole at a course is one where there's a dramatic appearance of some kind. A big dropoff, a throw over water from a scenic viewpoint, a unique elevated basket, a rock formation or wood carving, something like that.

For example, DeLa #27 and BRP #4. There are plenty of "top of the world" and tunnel shots out there, but those two are consistently mentioned by people as "favorite" holes. It's because they look cool.

 

[elmexdela]

One thing that I don't see addressed is the "visual appeal" of a hole. I think in most cases, the signature hole at a course is one where there's a dramatic appearance of some kind. A big dropoff, a throw over water from a scenic viewpoint, a unique elevated basket, a rock formation or wood carving, something like that.

For example, DeLa #27 and BRP #4. There are plenty of "top of the world" and tunnel shots out there, but those two are consistently mentioned by people as "favorite" holes. It's because they look cool.

and now all of minnesotas copycat brp 4 style holes that now took away from brp uniqueness

 

[Hampstead]

 

[Steve West]

If I were forced to never again play one hole of my choosing for the rest of my life, I would gleefully pick BRP 444.

Yet, if I wanted to try to improve the enjoyability of the local disc golf scene by picking holes in the Twin Cities to be eliminated from existence, BRP 444 would be one of the last survivors.

Good luck resolving those ideas into one number.

 

[ChrisWoj]

I'm not sure how you can add it in, but one factor I think of is uniqueness.

Is this hole significantly different, or very similar, to other holes (1) on this course, (2) that I play regularly, or (3) that I have played.

Subjectively, many of my favorite holes stand out because of their unique qualities.

Some of the worst, too, I guess.

One thing that I don't see addressed is the "visual appeal" of a hole. I think in most cases, the signature hole at a course is one where there's a dramatic appearance of some kind. A big dropoff, a throw over water from a scenic viewpoint, a unique elevated basket, a rock formation or wood carving, something like that.

For example, DeLa #27 and BRP #4. There are plenty of "top of the world" and tunnel shots out there, but those two are consistently mentioned by people as "favorite" holes. It's because they look cool.

Good points both. I'll come up with a few prompts related to this and add them to the trial instrument. I agree - those are valuable parts of enjoyment of a hole.

 

[ChrisWoj]

If I were forced to never again play one hole of my choosing for the rest of my life, I would gleefully pick BRP 444.

Yet, if I wanted to try to improve the enjoyability of the local disc golf scene by picking holes in the Twin Cities to be eliminated from existence, BRP 444 would be one of the last survivors.

Good luck resolving those ideas into one number.

Sounds like a reasonable task. If I include questions related to the uniqueness or visual elements of a hole, as requested - that should help. There are already elements associated with things that I would assume are a part of your enjoyment: appropriateness of fairway shape/width, obstacles, etc. Prompts associated with both visual elements and playability elements can both contribute to the overall construct.

 

[Smigles]

This URL takes you to the PDF shared via Google Drive:
https://drive.google.com/file/d/145Uu4vUqHyfcrNPxiYRMnJ7vUfc-edBX/view?usp=sharing

Just a quick thought after reading trough your pdf document, you would profit on some questions from changing the answers.

Like the first question on page two: "The rough appropriately punishes errant shots on this hole." You could go from "not punishing enugh" to "just right" to "too punishing" instead of the "Agree/disagree" answers.

 

[ChrisWoj]

Just a quick thought after reading trough your pdf document, you would profit on some questions from changing the answers.

Like the first question on page two: "The rough appropriately punishes errant shots on this hole." You could go from "not punishing enugh" to "just right" to "too punishing" instead of the "Agree/disagree" answers.

The problem with the set of answers is that for a survey to function as an instrument for measurement you want what is called 'monotonicity' with an ascending or descending nature to the scale. The inclusion of a middle category can often be bad, but I think you're okay with that one.

Another way I could do that may be the inclusion of a pair of prompts:
The rough on this hole punishes errant shots too much.
The rough on this hole does not punish errant shots enough.

I would plan on using the same set of scale options that I do on the rest of the questions.

Does this seem like a reasonable alternative?

 

[chevis]

maybe a question involving risk/reward - easy(par) route vs. hard(birdie) route... but less questions, the more likely to be filled out

 

[biscoe]

This reminds me a bit of the pdga's (failed) attempt at developing a course rating system.

It also reminds me of Mark Twain's quote comparing analyzing humor to dissecting a frog...

 

[ToddL]

I would reword one of the questions from:
"This hole does not make appropriate use of available obstacles"
to:
"This hole makes appropriate use of available obstacles"

All the other questions in the survey are written in a positive tense. This is the only one that's in a negative tense, forcing the reader to use a double negative to answer the question.

 

[ChrisWoj]

maybe a question involving risk/reward - easy(par) route vs. hard(birdie) route... but less questions, the more likely to be filled out

Hmm.. have to think about wording.. "This hole provides appropriate risk for the reward of a birdie." (eagle as a possibility for longer courses?) As to the number of questions - one thing that will happen after this is trialed: questions that do not appear to contribute to the measurement of the construct will be removed.

This reminds me a bit of the pdga's (failed) attempt at developing a course rating system.

It also reminds me of Mark Twain's quote comparing analyzing humor to dissecting a frog...

The Mark Twain quote is strikingly close to a saying in measurement: The moment a measure becomes a goal, it ceases to be a good measure. (comes from a more technical "Law" by Goodhart regarding economics: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.)

I would reword one of the questions from:
"This hole does not make appropriate use of available obstacles"
to:
"This hole makes appropriate use of available obstacles"

All the other questions in the survey are written in a positive tense. This is the only one that's in a negative tense, forcing the reader to use a double negative to answer the question.

Thank you! I missed that and definitely appreciate you noting it.

 

[IHearChains]

My opinions of a single hole are inextricably tied to the rest of the holes on the course.

I love BRP#4 but if I'm on an entire course that plays 18 holes exactly like BRP#4, I would say "this hole sucks".

Part of an epic signature hole is the anticipation that builds while working your way through the course to get to that hole, and that anticipation factor is going to depend on what the rest of the course is like.

 

[lowracks]

Much to most of my tinder dates dismay, I always provide one the next day after I've had time to reflect.