The phase, texture, and anisotropy in microstructure and impact toughness of 316L stainless steel were observed and analyzed by means of optical microscope (OM), X-ray diffraction (XRD), scanning electron microscope (SEM), electron back scatter diffraction (EBSD) and metal pendulum tester. Results show that the microstructure and impact toughness of SLM processed 316L stainless steel reveal an obvious anisotropy. The material phase of the samples in the transversal and the vertical direction were equal to γ-Fe phase, and their structure perpendicular to the building direction has a checkerboard morphology. Most of the grains were equiaxed and the grains has been refined (d_mean=9.177 μm), in addition grains in the overlapped area of the molten pool were observed to be finer (below 6 μm), contemporary, the number of grain boundaries with small angle parallel to building direction is large, whereas the morphology parallel to the building direction resembles a fish-scale pattern, most of which comprises columnar grains with a large grain diameter (d_mean= 21.247 μm). Meanwhile, the texture perpendicular to the building direction exhibits a strong fiber texture <110> //rolling direction (RD) (strength of 7.83) and a weak plate texture {112} <110>, whereas the texture parallel to the building direction exhibits a strong fiber texture < 110 >//RD (strength of 12.23). Under a similar density condition, the impact toughness values of transversal direction and vertical direction were 62.8±3.2 and 38.6±4.5, respectively. The toughness of transversal direction was significantly better than the vertical direction and increased by 62.69%. In addition, grain size, size angle grain boundary and texture type has significant effected on impact toughness anisotropy. the grains perpendicular to the building direction were refineed, with an increase the grain boundaries with large angles and an enhancement in the grain toughness, the fiber texture <110> //RD has a low impact toughness resistance, while plate texture {112}<110> has a good impact toughness resistance, the{112}<110> texture perpendicular has low strength and weak {112}<110> texture to the building direction ,therefore, the toughness of transversal direction was preferred.