There were never any elephants in Bali (until tourist attractions changed that); ancient Goa Gajah probably takes its name from the nearby Sungai Petanu, which at one time was known as Elephant River, or perhaps because the face over the cave entrance might resemble an elephant. It's located some 2km southeast of Ubud on the road to Bedulu.
The origins of the cave are uncertain; one tale relates that it was created by the fingernail of the legendary giant Kebo Iwa. It probably dates to the 11th century, and was certainly in existence during the Majapahit takeover of Bali. The cave was rediscovered by Dutch archaeologists in 1923, but the fountains and pool were not found until 1954.
The cave is carved into a rock face and you enter through the cavernous mouth of a demon. Inside the T-shaped cave you can see fragmentary remains of the lingam, the phallic symbol of the Hindu god Shiva, and its female counterpart the yoni, plus a statue of Shiva's son, the elephant-headed god Ganesha. In the courtyard in front of the cave are two square bathing pools with water trickling into them from waterspouts held by six female figures.
From Goa Gajah you can clamber down through the rice paddies to Sungai Petanu, where there are crumbling rock carvings of stupas (domes for housing Buddhist relics) on a cliff face, and a small cave.
Try to get here before 10am, which is when the big tourist buses begin lumbering into the large souvenir-stall-filled parking lot like, well, elephants.
