I can relate. I lived with a remorseless spouse for about a month before things took a drastic change. She tried to keep in contact and maintain a "just friends" relationship right under my nose while I was trying to figure out if our (at the time) almost 27 year marriage was going to survive. I put up with that and listened to her defend him when id say anything derogatory about him and not care what it was doing to me. It was pretty awful.
I eventually snooped through her messages, uncovered some more lies, and discovered her saying unflattering things about me. I know what you mean about taking a shot to your self esteem. I understand the pain you're going through.
What helped me with that was getting angry and fed up. I confronted her about her messages, watched her try to squirm out of it even knowing I had read them, then told her if she wanted him then she could have him. Just not while remaining married to me.
Then I called lawyers right in front of her and started setting up consultations. I immediately felt a sense of relief and started feeling better about myself because I was taking my agency back. I didn't want a divorce, but what I wanted less was to continue living in infidelity and uncertainty with a remorseless spouse. It was very liberating and I no longer felt like everything was out of my control.
It turned out to be the best thing I could have done, because one way or another I was getting out of a bad situation. It just so happened that my wife discovered that losing me was far worse than losing her AP. I don't think she thought I'd really do it, but id had enough.
I don't know what your husband might do if you follow my tack, but there's a saying here. Sometimes if you want to save your marriage the best thing you can do is let it go. Cut them free and let them choose. If they don't choose you, then do you really want to stay in a marriage with someone who doesn't want to be there?
I didn't.
You have to make up your mind how much you're willing to tolerate. I hit my limit, and while I really didn't want a divorce making those phone calls, finally deciding to do something, came with a great sense of relief. I felt much better about myself, regardless of the outcome.
I'm so sorry you're going through this hell. Most of us have been there. I think when you decide you've had enough and start taking action you'll begin feeling better about yourself, too. Set your boundaries. Tell him what you expect to see if he wants this marriage, then enforce those boundaries if you need to.