The molecular dynamics simulations were used to study the effect of grain size and twin density on the plastic deformation of nano-polycrystalline aluminum alloy. The results show that the dislocation density after relaxation is crucial to the microstructure evolution and the inverse Hall-Petch relation of the nano-polycrystalline Al. The staggered tetrahedrons and complex staggered structures are formed in the fine grains, which is attributed to the restriction of grain size. Thus, the auxiliary deformation of grain boundary is activated. The Shockley partial dislocations nucleate and multiply at the grain boundaries when the twin boundary spacing (TBS) is relatively large. However, with decreasing the TBS, the twin boundary becomes the source of the Shockley partial dislocations. A large number of partial dislocation nucleations at the twin boundary will cause the twin boundary to migrate or even disappear. The deformed nano-twins can also be observed during the plastic deformation process. This research provides theoretical basis for the development of advanced nano-polycrystalline Al alloy with adjustable mechanical properties.