When you no longer have confidence in the disc. A disc leaves my bag if I can't think of the last time I threw it, or if there's a similar disc that does the job better. If you can't pick up the disc and say "think disc is perfect for ______" then there's no use for it in your bag.
Sometimes I take things out of my bag and put new things in, and sometimes I revisit old discs that I haven't thrown in awhile.
As for putters.... There's likely nothing wrong with your putter, but more likely your putt. That said, you also need to have confidence in the putter you hold. The most important thing about a putter is the way it feels, as you won't see a huge difference in flight within close range between different putters. Throwing putters is a different story.
This really describes my philosophy as well. Almost exactly, actually.
If a disc has multiple catastrophic throws and I can't rely on it, I will often pull it and try something else to see if it improves. I've been adding and pulling understable mids for years trying to find the one that "clicks" with me. As my form changes and (hopefully) improves, I've put some of these discs back into rotation in order to see if things are different. Sometimes they are, sometimes not so much.
Sometimes I add discs and see if I like them. If I find myself throwing them a fair bit or like the flights, they get to stay. If I throw it and don't care for it, it comes out.
Sometimes I find discs overlap or I've "outgrown" one. If I realize I hardly ever reach for a disc, I pull it out. If I notice it's missing while it's out, it can find its way back in. Had a big purge along these lines when I realized I could get my Buzzz out to 300' but couldn't get my distance drivers any farther than my fairways. Yeah, my game is in an awkward spot.
Sometimes discs get pulled for aesthetics. I am a sucker for bursts and swirls, but sometimes I just like a color or plastic type better.
I actually just came off of a long hiatus from putting with my Pure. Couldn't rely on it I felt like so I switched to using a Zone for maybe 6 months. I cleaned up my form and realized I was missing a neutral straight disc and was struggling with some shots due to it being out. Now both Zone and Pure get putting duty depending on what I need at the time. My confidence in the disc was a big factor in it coming out and in again.