2D to 3D AI-driven real-time form biomechanics app

[Brychanus]

There's apparently a group in Japan developing an app for this. It looks outstanding (at least in this demo). I asked Chris Taylor if he knows anything about timeline or availability.

There's already plenty to study from this first video.

I scraped the video and made a gif of Paul for us. Notice that you get nice color-coded data about the relative speed and lag from his spine to the hand:

Would be nice to get the curves all the way to the rear wing of the disc into the release and for the rest of the body, but otherwise these data should appear familiar:

 

[seedlings]

That's pretty awesome. I'll be curious to see the statistical average throw after a few dozen pros are mapped.

 

[twistedraven]

I like videos and data that show you just how important it is to accelerate late.

 

[Dochammer]

This is cool! Similar products keep popping up and disappearing, maybe we'll start seeing some of them come to market soon.

That's pretty awesome. I'll be curious to see the statistical average throw after a few dozen pros are mapped.

Some what related: I came across this the other day, a study looking at the biomechanics between Ams vs Pros at Waterloo. Setup looks similar to the one Chris Taylor used to record Ezra and some other pros with motion capture, but they've also include high speed camera to measure the disc. I think they mentioned a force plate too in a comment, but I can't find it atm.

Starting with this post, they describe the setup:
https://www.instagram.com/p/CfuuwDApJus/?hl=en

Gilbert 360
https://www.instagram.com/p/Cg1gdivlgsy/?hl=en

 

[Rastnav]

So, not sure that this is useful, but it looks like Josh at Overthrow is using software called OnForm to do reviews which has some built in stuff for doing the skeleton capture. Although it seems like he might not find that feature useful.

You can see him turn it on here:
https://youtu.be/6t4wvulzaC4?t=1125

 

[Sheep]

So, not sure that this is useful, but it looks like Josh at Overthrow is using software called OnForm to do reviews which has some built in stuff for doing the skeleton capture. Although it seems like he might not find that feature useful.

You can see him turn it on here:
https://youtu.be/6t4wvulzaC4?t=1125

It's an Ipad app that he uses that's for tennis/golf and such.

I think he had it already, that stuff is expensive.
The software that SW uses is pretty cheap, but still somewhat expensive. Its.. uhh...
Oh i gotit here somehwere..

Motion Pro.
Super old style software. should just be freeware at this point its so old. haha

 

[seedlings]

 

[dekuscrub]

That's a really interesting idea!

I tried making something similar this weekend and had the most success with AlphaPose. It wasn't too much trouble to get working, but could still use some tuning.

Now, what to do this? Use a loss function to compare amateur throws to pro throws? Try to develop a model for the average pro throw, as seedlings suggested? There are potentially many fun possibilities

 

[Brychanus]

That's a really interesting idea!

I tried making something similar this weekend and had the most success with AlphaPose. It wasn't too much trouble to get working, but could still use some tuning.

Now, what to do this? Use a loss function to compare amateur throws to pro throws? Try to develop a model for the average pro throw, as seedlings suggested? There are potentially many fun possibilities

Holy ****. I didn't know this existed. Now my hobby is going to compete even more with my day job. Are you able to extract information other than the image itself (which is rad and helpful in any case)? There's a little imprecision in the skeleton but it looks pretty good overall.

This is not to nitpick your words but it did make me think: if someone were to get serious about modeling, I would caution the idea of an "average" on its face (I specialize in subject-specific data analysis in other contexts). There is an enormous difference between extracting fundamental principles and truisms that apply to all cases rather than averages over dynamic time processes. So a dynamic mechanical model that is trained and tested for fit to each form skeleton is potentially where to start. Identifying and extracting summary parameters over the dynamic motion also would probably be worthwhile. I would be very interested in getting something that quantifies smash factor in particular (disc speed greater than hand speed).

Minimally I want to see if I can get this booted up.

 

[bryantlikes]

Even shows the proper form for spectating!

 

[Brychanus]

Notice the variation in form:

Spectator in black moves quickly over his feet in dynamic balance downhill with a small buttwipe into a small weightshift to establish a relaxed ready position for viewing.

Spectator in red has a comfortable and slightly psychologically defensive arms crossed posture with a closed shoulder balanced relaxed on front leg following a Hershyzer transition (not shown).

Happy Monday everyone

 

[dekuscrub]

Holy ****. I didn't know this existed. Now my hobby is going to compete even more with my day job. Are you able to extract information other than the image itself (which is rad and helpful in any case)? There's a little imprecision in the skeleton but it looks pretty good overall.

Glad you found this interesting! I was able also able to get OpenPose and VideoPose3D running without too much trouble, but found AlphaPose to be the most flexible. OpenPose was definitely the easiest to get working, though, as the team distributes an executable version of it.

As for the data, all three programs output the pose tracking data in a human/machine readable format. (I believe JSON, but could also be YAML.) Definitely a lot to look at in there.

This is not to nitpick your words but it did make me think: if someone were to get serious about modeling, I would caution the idea of an "average" on its face (I specialize in subject-specific data analysis in other contexts). There is an enormous difference between extracting fundamental principles and truisms that apply to all cases rather than averages over dynamic time processes. So a dynamic mechanical model that is trained and tested for fit to each form skeleton is potentially where to start. Identifying and extracting summary parameters over the dynamic motion also would probably be worthwhile. I would be very interested in getting something that quantifies smash factor in particular (disc speed greater than hand speed).

Haha, I knew I was going to regret writing "average".

I 100% agree with what you're saying here about the approach. Unfortunately, I'm probably not the best person to take make a serious effort at modeling. I'm not a data science and haven't tinkered with dynamical systems in years. (My background is in computational neuroscience and machine learning, but I'm very out of practice.) However, I would be happy to build a dataset and otherwise assist as I'm able.

 

[SocraDeez]

I would be very interested in getting something that quantifies smash factor in particular (disc speed greater than hand speed).

Disc speed > hand speed = probably not possible.

We don't hit or kick discs. We throw them.

 

[Brychanus]

Disc speed > hand speed = probably not possible.

We don't hit or kick discs. We throw them.

Referring to whatever we want to call the disc leveraging out off of the last point of contact, which should ideally result in the disc moving faster than the hand (though curious if you disagree, or what else we'd call it to not confound it with ball elasticity at contact):

 

[SocraDeez]

Referring to whatever we want to call the disc leveraging out off of the last point of contact, which should ideally result in the disc moving faster than the hand (though curious if you disagree, or what else we'd call it to not confound it with ball elasticity at contact):

No, I get what you are saying now: one part of the disc moves faster than the part of the disc we are holding, or i.e. the disc is a lever, too.

It's hard not to think "collision" when you hear "smash" - I wonder how the phrase came about? But, on the significance of "smash factor" in disc golf relative to how it is revered in golf golf: The disc is a relatively rigid, short lever compared to a golf club.

 

[Brychanus]

No, I get what you are saying now: one part of the disc moves faster than the part of the disc we are holding, or i.e. the disc is a lever, too.

It's hard not to think "collision" when you hear "smash" - I wonder how the phrase came about? But, on the significance of "smash factor" in disc golf relative to how it is revered in golf golf: The disc is a relatively rigid, short lever compared to a golf club.

Yeah, it's also slightly weird in golf golf now that I think about it. Also amusing: I am reminded of the time Simon said...

I'm trying to think of something else to call it and I'm verbally challenged.

Tip-o'-the-whippersnapper
Snap factor
Crack effect
Compound pendulum efficiency factor
MAXIMUM LEVERAGE

all bother me conceptually or don't quite roll off the tongue.

 

[foxdawg10]

Disc speed > hand speed = probably not possible.

We don't hit or kick discs. We throw them.

Yes agreed, the discs don't compress like a golf ball or baseball. Although I am surprised by the amount of flex they have in slow motion.

 

[dekuscrub]

I put in a little more this week. Some improvements:

• Added a rough velocity plot with arbitrary joint tracking
• Interpolated missing joint location data
• Smoothed the skeleton animation

This method struggles a bit when something obscures a joint from the camera's view. With multiple perspectives of the same throw (like Overthrow's footage), it's possible create a full 3D approximation using more advanced pose estimation machine learning models. But that would take a lot more thinking and work.

For now, I like the 2D method because it should theoretically work with standard form videos, as are often posted to DGCR. Additionally, now that code is extracting some velocity data, that opens up some fun options for modeling and maybe even form reviews.

Anyway, I'm curious what you all think of all this. The code I wrote is a mess right now. But I'd at least like to clean it up and upload to GitHub.

 

[Brychanus]

I put in a little more this week. Some improvements:

• Added a rough velocity plot with arbitrary joint tracking
• Interpolated missing joint location data
• Smoothed the skeleton animation

This method struggles a bit when something obscures a joint from the camera's view. With multiple perspectives of the same throw (like Overthrow's footage), it's possible create a full 3D approximation using more advanced pose estimation machine learning models. But that would take a lot more thinking and work.

For now, I like the 2D method because it should theoretically work with standard form videos, as are often posted to DGCR. Additionally, now that code is extracting some velocity data, that opens up some fun options for modeling and maybe even form reviews.

Anyway, I'm curious what you all think of all this. The code I wrote is a mess right now. But I'd at least like to clean it up and upload to GitHub.

Awesome work & cool to see GG in there. Since you're out of the gates on this I had a couple thoughts about this to take it from outstanding to truly exceptional and you've got yourself a side hobby or career in the form industry.

Related: I've been sharing some lively discussions with SocraDeez and wanted to orient you to this article in the Hardball times. It alerts us to some of the potential advantages to modeling, what to model, and some of the ways it can challenge common misconceptions. I haven't yet caught up on how things are looking on the pitching modeling front compared to these arguments in 2022, but this was a pivotal piece. I do think there are a few things to debate about in the pendulum conception of the swing.

In these video-extracted models & in general, we really ideally would include the nose of the disc ripping around as the end of the compound pendulum.

Alternatively or in addition, if some inference about release speed/velocity could be extracted from the images, you've really got something top notch there.

Even if not, these timecourses alone are already fascinating and could teach us alot. If you're getting extractions and can post any I'd be interested in breaking it down at some point.

 

[Brychanus]

*Also some potential 2D disadvantage there unless I'm mistaken - hand peak speed should be high following the elbow, but I think because we've only got one plane it's peak appears to start and die before the elbow