From Pirates of the Caribbean Online Wiki
Any area of water between the islands is considered High Seas. At one time, a pirate could swim between the islands. However, once a pirate reaches the shallow water boundary - he/she cannot enter the high seas, except on a ship.