Let me just start by saying it's never okay to have an affair. It's lazy, it's selfish, and really, if you want to hook up with someone other than your partner, just break up with them! Seriously. If the thrill is gone, or the thrill is there but you still want to cheat and you know know they'd be pissed if you cheated, just end it. You are no longer on the same page.
That said, people have affairs all the time, so if you're going to be stupid enough to have an affair, pick someone who doesn't take care of your kids, your eyes, or your human body parts. The odds that they person you have an affair with will cook your pet rabbit in a pot of boiling water or try to kill your children may be slim, but stranger things have happened.