You shouldn't.
Although, I'd argue they often don't behave when we are covered. I've been cat-called with no makeup on and while wearing the bulkiest winter coat known to man.
At this point in my life, I refuse to modify my life choices, including how I dress, for men. I haven't worn a bra in years now, and I've never been more comfortable.