Kalpitiya Fort was built by the Dutch between 1667 and 1676. Kalpitiya was important as it commands the entrance to the adjacent bay, Puttalam Lagoon. The surrounding Puttalam area was one of the major cinnamon cultivation areas in Sri Lanka. The Dutch even constructed a canal from Puttalam via Negombo to Colombo to transport cinnamon from the area.

The fort is nearly square in shape, with walls about 4 m (13 ft) high, constructed out of coral and limestone from the surrounding area. It has a single entrance, which fronts the lagoon, which has a pediment, with a belfry above and looks like the entrance to a church.

The yellow bricks that comprise the entrance arch were reportedly brought specially from Holland. The story is that the King of Kandy had granted permission only to build a church here and that the Dutch had built the arch to mislead the King into believing that this was simply a fortified church.

Kalpitiya Fort has four bastions on each corner, each with its own guard post, with two smaller bastions facing the lagoon. Inside, the buildings are located around the periphery creating an empty space in the middle of the fort. The walls of a chapel, barracks, dining hall, commander’s residence, and prison are still evident, though the roofs have been replaced. There were two tunnels leading away from the fort — one led to the sea and the other to the Dutch Reformed Church approximately 400 m (1,300 ft) outside the fort. These tunnels are blocked and are inaccessible.