Everybody I know who wears them just calls them skirts. Not making a big deal about them in ways to point up the "ooh, man wearing woman's clothing" thing but just taking it for granted that anyone can wear anything they like is a large part of the point.
As Iggy Pop says "I don't wear 'women's dresses'. I wear my dresses."