We are all born sexual. Sex is definitely
essential to a holistic living in a marriage, but on the other
hand, we live in webs of suppressive conditioning which work to
restrain the flow of sexual energy, the recognition of impulse and
need and the fulfilling sexual interaction with others.
Your higher consciousness relates about your wild past and that you
know it isn't something you will seek ultimately. That creates the
suppressing condition because circumstances is waging war against
this higher consciousness, in which you knew for sure that if you
succumb to your past behaviour, you will face torment and
suffering. At the greatest cost, you could also lose your
marriage.
I believe you are not nymphomania; just that sexual frustration
will definitely lead to emotional frustration. The combination of
two - devoid of romance and sexual fulfillment often blurs our
vision in love and that we began to question ourselves: if we have
make the right decision, why are we feeling empty?
You must understand that we are unique individuals, with separate
sets of needs. Therefore, if these needs are not met, we seek to
fulfill them.
Even in a
seemingly impeccable marriage/relationship, there are invisible
imperfection we must seek to amend. If you allow
this frustration to grow, (and with critical planetary crisis) you
will implode from within and lose your position to control your
situation.
Your emotion is 24 times stronger than your mind. Should there be a
day when you lose your mind - you lose yourself.
Not possible? Nothing is absolute - I give you one possible
scenario: when alcohol is involved (Under intoxicated mode, Neptune
influence, Moon takes over our consciousness and we become another
person). 
***
I believe that we must constantly preview and renew
our relationship. Even we have probably the best relationship ever,
if we do not seek to work on it, all will crumble in time to
come.
We must humble ourselves, with maturity and wisdom, and recognize
that there are indeed issues we have to resolve. These are
definitely touchy issues that could trigger him into defensive
mode, but it is always better to discuss mutually and be open with
what's wrong, than to pretend that all is going well until all hell
break loose. When frustration breeds unconsciously, we resolved
them through communication - this is the only proven-to-work method
(which is widely known, yet not commonly practiced).
Currently, you may have the strength to reason with yourself and
resist infidelity, but how sure are you to know that this strength
will shield you forever?
When you talk to him, please note you probably need many... many...
many... many of such session before things get better.
Remember:
I) It's a sharing session, not a trashing one.
II) DON'T EVER GET PERSONAL.
III) If he is triggered into aggressive mode, just listen to his
views. Speak in summary and keep things short. Work on it the next
round.
IV) If he triggered into defensive mode, speak objectively than
emotionally. Tone to be gentle and avoid sounding like you are
admonishing/picking on him.
V) Seek for possible solutions mutually.
VI) Never forget that you love him and he loves you. Knowing is not
enough, people likes to be told that they are loved. If things are
tricky, always remember this point.
I wish you well. 
Cheers