Thank you for letting me know. I've made a report to the programmers. :)
Is this still an issue? If so, what browser are you using? Have you tried the beta wardrobe?
When you started to make this new HA did you use the "Remove All" option by chance?
If I recall correctly from when I first was testing out this bug (waybackwhen), it had to do with the "Remove All" button. For whatever reason using it wouldn't update the layer counter (removing the numbers).
So, instead of using the remove all, hitting the x next to the item to remove it individually seemed to work. Not sure if this is actually (still) the case or not. Just something I noticed before.