the hardest part of being an adult is dealing with contradictions. this comes up with associating with people who do things you feel are wrong in some way. For instance you're friends with Bob. Bob is an okay guy when you hang out, but you notice he's verbally abusive to other people - strangers mostly. It would reflect poorly on Bob and might lead you to spend less time hanging out with Bob.
Oh here's a good one. Alan is trusted friend who has been dating Alice for 9 months. Alan has no trouble with the ladies and he tells you from time to time that he slept with so-and-so recently. You feel that being unfaithful to the S.O. in a committed relationship is a pretty big fuck-up. Yet you're not directly involved.
Why are these contradictions? Because I feel you are involved in a very small way just by being aware of it. Knowing that my friends act that way does have an impact on my life. So normally one takes steps to address a negative impact on one's life. I suppose a balanced response is to comment with the same length and intensity that it was presented to you with. If Alan makes an off hand comment about scoring with Jill, and this news bothered you, you could make a off hand comment about why you think thats uncool.