To me special units are the extra flavour of an army (and the special characters could be the cherry at the top). The actual feel of an army relies in its special rules. Units without special rules are just stats ratios for their points, and every army can play every type of unit, but what defines an army as a gaming mechanism is the rules that set it apart from another one, especially when they are army-specific rules and not generic.
The time spent with the game is assembling, painting and then playing with the army. I think a unique unit is appreciated at each step. Rules are (for me) only appreciated at the last step. And units are bigger than characters so take up more time generally at all three steps.
Also, just the look of the army on the table is important. If it is just rules we could just use numbered counters, yes? (I know it is not that simple but it is kind of my gut feeling on the subject.)