My problem with calling everything an ultralight is that whenever there is a crash (172, lsa, basically anything short of a DH8) then it is referred to in the press as an Ultralight.
That term in the public eye then becomes synonymous with being unsafe. Not only does that sometimes cause concern for potential passengers you might have, but I am sick of getting the statement from people "ultralights, that's pretty dangerous isn't it. They crash all the time".