In 1662 the castle was burned down and then rebuilt, only to burn down again in 1792. All we can see today is the remains of one wing dating from the 17th century, as well as the large gardens that extended around the castle.
In the 17th and 18th centuries, Lindholmen had an important role in Swedish history. The castle was a grand baroque palace, with towers, wings and an impressive park containing outbuildings and statues. The ruins were restored in 1931–32.