Your question is very logical but unfortunately Gandhi ji, for whatever other good qualities he may have had was not a logical man. Or rather his logic operated within a rigid, narrow framework.
Of course one can always justify anything, the mind keeps doing that. Surely he would have his own justification. But symbolically it is interesting because Sri Krishna grew up on cow’s milk and as far as I know, cow milk is not only closest to human mother’s milk but also the one widely consumed in all the Ashrams including the Sri Aurobindo Ashram. Most importantly cows are very affectionate by nature and have a soothing healing atmosphere around them. Indians in earlier days consumed cow milk as a prasad and used curd made of cow milk as a base for panchamrit. The fact that Gandhi ji (and now certain modern researchers) directly or indirectly opposed its consumption is itself an indication of the forces working behind him. Incidentally cow is a symbol of light and goat of greed and lust. But these things are beyond the ken of modern logicians who see only the surfaces of things.
Affectionately,
Alok Da