He might have lots of other affair, that is why so sensitive
about phones nos on display, if he got nothing, why do not dare to
answer a call? He is keeping something very big from you, to me,
this look like a marriage of convenenice for him.
End up, marrying for the sake of being married, buy house, and
you become the children production dept for him, after 2 to 3 kids,
he might not even answer any phone call from you.
One of the sign of unfaithful partner is when partner are wary
about phone calls.