I've observed that faster discs go farther because they can be thrown harder, so they have greater velocity to begin with. In order to do this, (again, all things being equal), faster discs must "behave" with greater stability, to allow them to thrown at greater velocities without crashing and burning.
I agree with some of your observations, but not your conclusion. Wizards, BB Aviars, Challengers, Rocs and a bunch of other slow discs can be thrown just as hard as any high speed disc and, as I've stated in other posts, actually have a higher initial velocity. They don't go as far because they slow down more during the flight than faster discs.
What it sounds like you're observing is that fast discs tend to fade out early if not thrown fast enough, which is true. If you don't get a disc to it's "cruise speed" (the speed it needs to go to get it to "glide") it will fade out early and act more HSS than designed. The flight will be more projectile-like rather than flying disc-like. So if you get a disc up to speed you'll not only get the advantage of the extra speed you put on the disc, but you'll get the advantage of it actually gliding. This is easy to observe in a headwind. Many times people will find they can throw their super fast discs farther into a headwind than they can in no wind. It's because they're getting the discs to actually fly.
You don't observe this with slower discs becasue their cruise speed is so low. If you toss a Roc 50' you'll see it easily, but if you're always throwing it 200'+ you'll never see it. A 50' Roc flight looks way different than a 300' Roc flight. However a 250' Roc flight and a 300' Roc flight look similar but might be thrown different heights. The 50' flight never gets to the cruise speed, but the 250' and 300' (or even 350') ones, do.
You'll probably never see it with a putter because everything you throw that's under its cruise speed is either supposed to fly like a projectile or can be compensated for by adding extra spin, which you can't do enough with high speed discs to overcome the lack of speed.
One consequence of this is that if a fast disc is enough high speed understable then it will quickly go from overstable and fading out early to high speed understable once you hit that cruise speed. That can make predicting where it will land difficult. I had this problem with Illusions. They'd act pretty overstable unless you misread the wind a tiny bit and/or got a particularly good rip. Then instead of seeing the hyzer you wanted, you'd get a big turnover. In my case that usually meant losing the disc.
On the flip side if you exceed a certain speed for any given disc the disc will get more and more high speed understable (pun intended). Some do this gradually (Eagle, OLF, Cyclone) but some do it rather abruptly (e.g. Sidewinder, Archangel, most lids). The former are much easier to control and tend to be more "predictable." Some discs have a super wide cruise speed range. Most putters, mids and overstable drivers that are commonly recommended are like this. The Teebird is the classic example because it flies straight for anyone throwing more than like 280'. I'm not sure what the upper end of it is, but it's farther than most people, even top pros, throw for golf shots.
I like to use and recommend discs that have a large cruise speed range. They're just so much more forgiving and are good indicators of when you did something more right. A better throw will go farther. With some discs a better throw might turn over more and look like a worse throw.
This is also a good time to define "squirrely." When I (and many others) say a disc is "squirrely" what we really mean is that the disc has a very narrow band for its cruise speed and has abrupt high speed stability change when you meet or exceed the cruise speed. They can also be extra sensitive to nose angle which seems to have a similar effect to being sensitive to speed.
