Penality in Scandinavia has been seen as somewhat of an outlier, a redoubt against the punitive turn witnessed in other parts of Western Europe and the United States. This article examines contemporary discourses of penality in Norway following the entry into government of the populist-right Progress Party. The analysis describes how government representatives frame themselves as protecting individual security and prioritising victims whilst pursuing a bordered version of penal populism directed against non-citizens. These non-Norwegians are shown to be the focus of a range of penal populist policies, including fast-track justice, a warehousing prison regime, targeting of petty offenders and the double punishment of imprisonment and deportation. In a context where the penal welfare consensus with respect to Norwegian citizens remains relatively strong, it is easier for the Progress Party to ‘do populism’ by focusing on non-citizens.