I do it in Age of Fantasy. I use Lord of the Rings figures, and mix wood and high elves together, as well as orcs and goblins.
Because of the point system, it should be that things are balanced by points. The only thing you could lose is having different armies have different strengths and limitations. But you might not lose that, either. And I want to play a game with the figures I have, not try to find figures to match someone's design of the army. I agree that your suggested grouping of figures makes sense.