Common sense applies - the Swimming Test is to ensure individuals are capable of stepping off a ship, swimming to a liferaft fully clothed & climbing in. Generally when ships sink, swimming goggles aren't always available unless the NAAFI is open or they happen to be in your pocket.
I wear bausch and lombe extended wear but I have had ones that fall out easily before. Saying that, if you were having to jump ship then I think your contact lenses would be the last thing on your mind. Suppose it depends how bad your eyesite is!