In this work, Inconel 625 alloy is used as a reference object, based on the alloy design ideas of interface control, a model alloy with lower stacking fault energy is designed by Thermodynamic software. The influence of stacking fault energy on the evolution mechanism and grain refinement mechanism of nickel-based alloy with gradient structure was studied by surface mechanical grinding treatment (SMGT) technology. The results show that a 450 μm, 300 μm and 250 μm thick gradient structure layers were prepared in Inconel 625 alloy, M1 and M2 alloys by means of SMGT, respectively. Moreover, the evolution mechanism of the gradient structure of different alloys is also different. For the Inconel 625 alloy, with the gradual increase of strain, its microstructure refinement mechanism gradually changed from dislocation segmentation mechanism to shear band and/or twin segmentation mechanism. However, for the M alloy, the deformation twin splitting mechanism is dominant. Therefore, based on the design concept and the corresponding microstructure evolution mechanism, it can provide a reference for the composition design of low stacking fault energy nickel base wrought superalloy.