How should we measure parties' position on the unidimensional left - right axis? There are several answers provided by the literature, clustering around three central sources: Mass surveys of voters, expert judgements and content analysis of party manifestos. In this thesis, I conduct a construct validation of 10 different measures from these three sources, using out-of-sample predictive power as a benchmark for measurement validity. Specifically, I use the measures to replicate three studies from renowned journals in political science. As a preliminary analysis, I compare the substantial conclusions given by the different measures when replicating the original models. In the main analysis, I compare the predictive power of the replicated models across the different measures, using the 5-fold cross validation method. The empirical results suggests that when conducted in typical statistical analysis, the measures differ very little. For most quantitative purposes, scarce data will be a much bigger threat to erroneous conclusions than wrong measurements. I argue that this speaks in favour of automated content analysis as a method for measuring policy positions, because it is drastically cheaper and has fewer limitations for temporal and geographical scope.