For continuous carbon fiber reinforced aluminum matrix composites (CF/Al composites), the thermal shrinkage behavior and thermal residual stress distribution in the vacuum pressure infiltration preparation process were studied by combining the numerical simulation with the thermal performance test. The results show that the transverse thermal shrinkage strain of the composite is much larger than that the axial thermal shrinkage strain, and it is transversely isotropic. The radom fiber arrangement RVE model can accurately predict the axial and transverse thermal shrinkage behavior curves of the composite. In the prepared composite, the fiber and matrix alloy are in compressive stress state and tensile stress state, respectively. The transverse residual stresses of the matrix alloy and fiber is less than their axial residual stresses, and both show the transverse isotropic. The matrix alloy will appear different degrees of damage under the action of axial residual tensile stress. Especially, the high residual stress at the small fiber spacing will cause local interfacial failure, which is not conducive to the bearing capacity of the composite. It is an important technical approach to reduce the local segregation of fibers in order to improve the mechanical properties of the composite.