The Fernandina Tower was the result of alterations undertaken in the 14th century to the second Muslim wall. It served as a jail from the end of the 15th century. These days, you can climb the spiral staircase to the top for wonderful views. Note: the lighting is poor and it's a steep and winding climb up three levels. There's also an excellent video on Elvas and its fortifications.
