To the north of the island you can find the Chinkana ruis, also known as the "laberynth". These ruins have several rooms, passages which are built on a slope very close one to the other. Some distance beyond this place there are two natural sunken formation on the rocks which are similar to giant steps which the local natives attribute to the Sun god or Thunapa, who was a son of the great Sun god or Huiracocha who sent him to profess compassion, but he was hit and badly treated and tied to a raft which was left to drift and eventually formed the desaguadero river.
The boats which go to the north part of the island arrive to Challapampa, from there until Chinkana you must walk around 40 minutes, following the north parth and going to the left.
You can find more information here:
Lonely Planet Bolivia (Country Travel Guide)