It is written,
Deuteronomy 22:5 (KJV)
The woman shall not wear that which pertaineth unto a man, neither shall a man put on a woman's garment: for all that do so are abomination unto the Lord thy God.
On the flip side, both of these articles of clothing pertain to a man. Any woman would be in violation of the above text if she were to wear either one.
It's common sense. 