From a lot of people, mostly women, many of them ex-wives, he was estranged. That was the legal jargon used: Estranged, & paramour; used when he shacked up with a married woman.