Dressing warm definitely keeps your hands warm. It's all cumulative, as the blood flows in and out constantly. I notice a difference in extremity warmth depending upon how I'm layered up. For example let's say its below freezing out, say 25 degrees. If I'm in snow pants and long underwear, I can often get away with wearing just regular GTX Terrex shoes as long as there isn't a ton of snow down. But if I don't have the snow pants on and just have regular athletic wear pants, I notice my feet will get cold and I'll have to wear insulated winter boots.
Same goes for my hands too, depending upon how dressed up I am. Your hands and feet are the canary in the coal mine for how you're dressed. They're the first to get cold, and if they start to it tells me I'm not layered up appropriately.
I've played this week in single digit real feels and will play in much colder this coming January, and I do not use hand warmers at all, and a lot of the time I have to leave my throwing glove stuffed away in my bag for extended periods of time because my throwing hand will get too hot and sweaty.
You're definitely right about gloves getting easily lost. That's an issue no doubt, but so is your discs, phone, beer, birdie bag, and whatever else. I've tried to have a good system in place where I keep things and try to keep the libations to a reasonable level to keep that at bay.