Moses himself did not go out to battle the Midyanites; he sent Pinhas the priest at their head. The Sages suggest a reason for this.

“Moses sent them” (Numbers 31:6). The Holy One, Blessed be He, told Moses: “Avenge the vengeance” (31:2) – you yourself. And yet, he is sending others? Because Moses had lived in the land of Midyan, he said: It is inappropriate for me to harm those who treated me well. The parable says: Do not cast a stone into a well from which you drank. (Bemidbar Rabba 22)