Generally speaking, you should not swim in your contact lenses.
This is for two main reasons – Firstly it is very easy for the
lenses to be washed out of the eye by a small wave or if you place
your head under the water. Secondly the lenses, especially the soft
variety, will absorb any chemicals or germs in the water. They will
then stay in or on the lens for several hours, irritating the eyes
and possibly causing infection. You can get ready-made prescription
swimming goggles from opticians which are not too expensive and
would be safer for swimming.