I'd hate to be the guy after or next to you at the gym.
Just kidding, of course, but the thought of stank sweats and coffee breath...ewww.
As for me, I do it whenever my schedule permits. I go to the gym every day, but usually at different times as I run my own business and have to work around that.