Well I guess I should note that a Soft Challenger is nothing like most soft Wizards, Pro Aviars, D Challengers, etc. Good versions of those putters have a pretty good combination of durability, stiffness, stability, and glide. I can understand not enjoying the feel of a Soft Challenger, but you can get baseline putters in much nicer feeling plastic that make a good compliment to a more durable premium version of the same disc.
The Cryztal Challenger is a funky disc. Discraft seems to make a lot of them but I've never seen anyone use it long enough to actually beat it in to see if it picked up any glide or retained stability. I don't think the HPP Wizards are anymore overstable than the Cryztal Challengers. They're probably more nose angle sensitive than baseline wizards and might be prone to lifting and stalling if you don't get them pretty flat or nose down.