The second phase particles in aluminum alloy are produced in the casting process and have a significant effect on the physical and chemical properties of the material. At present, the commonly used quantitative characterization methods of the second phase have the problems of too much manual work and time consumption. In this paper, a fast extraction and quantitative statistical characterization method of the second phase in large-scale aluminum alloy based on deep learning is proposed. This method can achieve multi angle refined quantitative statistics by fast and intelligent extraction of the second phase features in the image. The results show that the image processing time of this method is the same as that of software batch processing, which is only 0.4s/sheet, but the image segmentation accuracy is improved from 42.74% to 91.12%. In order to meet the requirements of fine characterization, 110000 full field sem images of 7B05 aluminum alloy, including four types, were segmented and extracted. The new characterization parameters of the second phase, such as aspect ratio, shortest distance, surface distribution and line distribution, were calculated. The results of line distribution show that the full field multi angle method used in this paper is better than the traditional statistical results of randomly selected field of view. This method shows that the error of statistical representation is smaller and the feature information is more comprehensive. The quantitative statistical results show that the average area of the second phase is the smallest at the nearest upper and lower surface of the cast rolled 7b05-t4 aluminum alloy section with a thickness of 6 mm, and there is a trough value near the thickness of 3 mm; The average area of the second phase is 1.99 μ m2, 1.84 μ m2 and 2.18 μ m2, the average aspect ratio is 1.89, 1.95 and 1.84, and the number of the second phase is 33574, 33207 and 42035, respectively. The above results show that the rapid extraction and quantitative statistical characterization method of the second phase in aluminum alloy based on deep learning can carry out multi angle data analysis and mining, and provide the basis for the study of microstructure and properties.