Currently, the only way to remove items from your wardrobe is to click the item so you avvie is wearing it, check the little checkbox, and hit Checked To Inventory.
There really should be a Remove All Items From Wardrobe button, but it seems like something that's too useful for them not to end up doing it at some point. It might be a bit of a server load, though, if your wardrobe has thousands of items in it like mine does. :
If you all watch the latest Keith Answers on Youtube, they are working on this issue. I believe a new design is being worked on. :)
I believe that THIS is the video explaining it. :)
~~~Selling my literal soul for this~~~