The different Mg alloys before and after Cu and/or Zn addition were used as the experimental material. The effects of Cu and/or Zn addition on the microstructures and grain sizes of Mg alloys were firstly discussed and the mechanism of grain refinement after Cu and/or Zn addition was explained. What’more, the phase compositions of different Mg alloys before and after Cu and/or Zn addition were revealed. The results show that after Zn, Cu and Cu+Zn addition in the pure Mg, the grain sizes of the alloys decreased from 1270μm (pure Mg) to 470μm, 120μm and 85μm, respectively and the crystal morphology changed from columnar grain to equiaxed grain. The Cu and Zn addition had the best grain refinement effect on pure Mg, then the Cu addition followed and the Zn addition was the worst. After Zn addition, Mg-3Zn alloy was composed of α-Mg and MgZn phases. The secondary phases were mainly in the morphology of tiny particles. After Cu addition, Mg-5Cu alloy was composed of α-Mg and CuMg2 phases. The secondary phases were mainly in the morphology of discontinuous herringbone structure. After Cu and Zn addition, Mg-5Cu-3Zn alloy was composed of α-Mg, CuMg2 and CuMgZn phases. It was found that CuMg2 phases were in the shape of continuous bulk and the CuMgZn phases were in the in the morphology of discontinuous herringbone structure. The grain refinement of Zn addition on pure Mg was mainly due to the effect of solute Zn element. And the mechanism of the grain refinement of Cu addition was the effect of solute Cu element and pinning effect of CuMg2 phases on the grain boundaries of α-Mg. However, after Cu and Zn addition, the grain size was finer than Mg-3Zn and Mg-5Cu alloys. This was mainly due to composite solute effect of Cu and Zn element and stronger pinning effect of the secondary phases.