huh? If you feel attachment for the department, you cannot leave
in peace because...
You feel you need to help those nice people to cope without
you?
You feel guilty about making life difficult for them?
If that is the case, well, in life, you can NEVER have the cake and
eat it.
What is/are the reason/s you want to leave? Is it valid?