Northern Utah here, so we have a pretty good mix of dirt and chunk. During the spring transitional period where I'm unsure if running skinny MTB tires will damage the trails, I still ride my fat bike since it doesn't leave ruts. But inevitably there isn't enough snow at that time to cover the boulders and gnar, so I get ping ponged around on my bike during those sections of trail. Makes me thankful when the dirt sections dry out and I can transition back to my full suspension bike. I'm sure I could live with a hardtail if I needed to, but not a fully rigid during the summer months. That, or I would completely switch which trails I ride.
As for the single speed aspect...yikes. You're more adventurous than me! Give me a giant granny gear for those long sustained climbs, and give me some extra torque for technical climbing. I would hate having just a single gear for all climbing. That'd be like having to decide on one tool for all repairs. Sure, you can hit a nail with the handle of a screwdriver, but why would you do that when you can use a hammer instead?