Fair point.
Ideally, it's the designer. Ideally, the designer does it right, and nothing ever changes.
The TD becomes the de facto designer with course changes, time changes, putting a tournament on a course with ill-designated pars (or perhaps pars designated for other skill levels), added temp holes, and because the TD has an interest in the missed-hole penalty being as equitable as possible.
There is also a question as to whether "as determined by the designer", or whatever it says (I'm too lazy to quote verbatim), means that the designer is the arbiter of par, which should be set according to the rest of the definition, or whether the designer has absolute authority, regardless of the rest of the definition.