A kilometre or so north of Muang Khong, in some hills more or less behind the mayor's office, a trail leads to Tham Phu Khiaw. The cave – actually more of an overhanging ledge – contains some old Buddha images and is the object of local pilgrimages during Lao New Year in April. To find it, head north from Muang Khong for 1.5km and take a track to the left, through a banana plantation. It's only a 15-minute walk (mostly uphill) to the cave entrance, but the track isn't always obvious so it's best to find a local to guide you (15,000K from the tourism office).
