This research work examined series of dental composite materials containing nano-fillers (nanoalumina, nanosilica, and nanozirconia) and micro-filler (gypsum) to determine their effect on the physical, mechanical, and wear properties. The proportions of the silanized fillers varied from 0 to 3 wt.% in an acrylate based (50 wt.% BisGMA, 49 wt.% TEGDMA, 0.2 wt.% Camphorquinone, and 0.8 wt.% ethyl 4-dimethylaminobenzoate by weight) matrix. The physical, chemical, and mechanical tests such as void content, polymerization shrinkage, water sorption, hardness, compressive strength, and flexural strength (FS) were performed. Three body abrasive wear tests were performed in a dental wear simulator machine under varying load, speed, and temperature conditions. The finding of research indicated that the dental composite filled with 3 wt.% silanized nanozirconia exhibited maximum hardness, maximum FS, and maximum wear resistance whereas, dental composite filled with 2 wt.% silanized nanoalumina filler indicated maximum compressive strength. Gray relational analysis method was applied to rank the dental composite using several performance defining attributes. These results indicated that the best combination of mechanical and wear properties were exhibited by dental composite filled with 3 wt.% silanized nanozirconia.