Self-esteem development across adulthood has been in the center of interest for some time now. However, not much is known about factors that shape self-esteem and its development in the second half of life and whether the factors differ with age and gender. To examine these questions, this study uses 2-wave data from the population-based NorLAG study in Norway (N = 5,555; Mage = 58 years; 51% women) and combines self-report data on self-esteem and personality with registry-based information on socioeconomic status (education, income, unemployment), health problems (sick leave, lifetime history of disability), and social relationships (cohabiting partner, lifetime history of divorce and widowhood). Results from latent change score models revealed that self-esteem peaked at around age 50 and declined thereafter. More importantly, lower socioeconomic status, not having a cohabiting partner, unemployment, and disability were each uniquely associated with lower levels of self-esteem and/or steeper declines in self-esteem over the 5-year study period. Over and above registry-based information, personality characteristics were relevant, with a more mature personality being associated with higher self-esteem level. Emotionally stable participants also showed less pronounced declines in self-esteem. Moreover, associations of disability and of emotional stability with self-esteem level were weaker with advancing age. Among women, self-esteem level was more strongly associated with emotional stability and less strongly with openness, compared to men. Our findings demonstrate the utility of registry-based information and suggest that physical health, social relationships, and personality factors are in manifold ways uniquely associated with self-esteem and its development later in life. (PsycINFO Database Record.