At its peak, Tantallon was one of the strongest fortifications in Scotland. It is still impressive and awe-inspiring even though large portions of it have been severely damaged. It is located in East-Lothian, 2 miles east of North Berwick, with high cliffs on three sides looking out to the North Sea.

A 12-foot curtain wall stretching across the landward side was protected by a deep ditch or dry moat, crossed by a drawbridge. At the northwest end of the wall stood the 6-story circular Douglas Tower, with quarters for the Douglas family, a pit prison (dungeon), and latrines. The East Tower at the southeast end was 5 stories and D-shaped. Little remains of either of these two towers.

Near the centre of the wall was the 5-story Mid tower behind a barbican. It is the most intact tower today, and contained quarters for the keeper or constable as well as the gate house. The hall block or close, which included the Great Hall, a bake house, and a well, stood near the Mid Tower. On the sea side of the close, a sea gate allowed the castle to get supplies or receive visitors from the sea when needed.

The outer ward (bailey), where functional buildings would have been, is mostly ruins; only the dovecot is still standing. The outer ward was defended by another deep ditch. Beyond that, an artillery ravelin (a triangular outbuilding containing guns that could fire upon troops attacking at the curtain wall) was added later.