I seriously hope you
are not someone I know.
/me count fingers.
Nah, I know you are not 
***
These are two
separate issues coming in conjunction to fuse into one ultimate
emotional annihilation operation. The factors are such that:
I) You are currently being neglected in your current
relationship
II) Falling for a married man.
By observing (I), it seemed to me that it's more about having
to-fill-up-the-void context. Say one fine day your boyfriend treats
you better, would you be liking the other man less? Or say if the
married man chase after you, would you accept him and dump your
relationship? Would you be able to answer the questions without
hestitation? 
Not likely.
Confusion swarm around you.
Flipping through Yunnie's Book of CloUdiSm, there is this paragraph
about the Mirror Law. It's pretty long, but to sum up: here it
states about the psychological/emotional state of a person being
mirrored when he/she falls for another person, sharing similar
status as him/her.
To explain your case in simple words: when we talk about people who
cannot commit themselves into relationship for any reason, they
often find themselves facing opportunity/encountering uncommitted
relationship. (E.g. you are attached, but your relationship is
weakening from negligence. You are falling for a man who cannot be
committed into a relationship with you). I have my best friend,
always lamenting about not being able to find the right mate. She
have everything a guy would seek and when potentials floods in -
everyone were eliminated mentally.
Deep down inside, it is her who cannot commit
and she knows it. Her past relationship wound her
deeply and she prefer to keep things this way.
Next thing? An affair with a married man.
***
Learn to remove
obstacles in a manner where it wouldn't leave regrets and marks on
your life forever. Since you are in relationship, until the day it
fails you, you ought to upkeep what is neccessary in your
relationship as a girlfriend. You can pursue for the kind of
relationship you want, even if he is married, but terminate
whatsoever bonds you have first.
Or if you have persisted in your relationship and earn insight in
how to make them better - you would have level up your relationship
to the next level.
P.S: Never choose alternatives by leaping ships. Always attempt to
repair damage first. Of course, unless you chose to abandon ship.

Cheers