The Ter Kleef Castle was built sometime during the 13th century. It is not known who the original owner was, and the castle has changed names several times. During the early 14th century it was known as the ’Huis te Schoten’, because of its location near a hamlet called Schoten. The castle was handed over to Margaretha van Kleef in the early 15th century, giving the castle its present name.