The e4 allele of the apolipoprotein E (APOE) gene is a known risk factor for Alzheimer’s disease and may also affect cognitive performance in normal aging. The data presented in the current study represent the first two time points of an ongoing longitudinal study examining cognitive changes in genetically at-risk individuals with much emphasis on test of episodic recall as measured by the California Verbal Learning Test. A total of 110 of 139 people who participated in the first phase of the study were used in the final analysis. Results indicate that there was no significant decline in cognition within participants over time. However, there was significant mean differences based on apoe4 status (participants with positive apoe4 allele have lower average mean scores than participants with negative apoe4 allele (both on short and long delay recall). In addition, there were significant interactive effects between apoe4 genotype and age (only on short delay recall) with the difference between the positive and negative larger for the older adults group. Gender also failed to produce any significant differences. It is concluded that, more complete understanding of the relationship between genes and memory requires consideration of a variety of factors. Several genes in interaction might account for a considerable portion of memory performance. After several candidate genes are known, this may be a fruitful approach.