Type 2 diabetes mellitus (T2DM) is a heterogeneous metabolic disorder characterized by reduced insulin sensitivity and relative insulin deficiency. Long-term T2DM can cause lesions in multiple organs and tissues, including the central nervous system, potentially inducing cognitive dysfunction, and resulting in increased mortality, which has emerged as a significant public health concern and economic burden in the 21st century. Regular physical exercise plays a crucial role in alleviating the development of cognitive impairment in type 2 diabetes. However, the underlying mechanisms remain unclear. This article reviews the pathogenesis of cognitive impairment in type 2 diabetes, and summarizes the mechanisms by which exercise exerts its beneficial effects, thereby providing new insights for future strategies to alleviate cognitive impairment in diabetes.