How do you do this? Didn't see it mentioned in either battle guide.
both removing weapons from your battle set and moving them back to your inventory is done in the armory tab of the battle coloseum.
You can remove weapons from your battle set by pressing the edit set button and unticking the boxes of the weapons you want to remove from the set and then pressing the edit set button at the bottom of the page. You can also remove the entire set at once by clicking the delete set button.
If you want to move items from your armory to your inventory you simply press the remove button under that item on the armory page.
Oh, but I have nothing in my armory, no sets. When I go to Swatch and Mementos' profiles though it shows items.
The page to remove potions (and directly equipped weapons from before the armory became a thing) is unnecessarily difficult to locate so I'm just going to link to the correct pages for you.
To find the link for yourself, go to the Personal tab at the top, hover over Pets to pull up that menu, and click the word Pet under the name of the pet you want. On the page that pulls up, you will find a link named Edit Battle Weapons. Click it and ta-da. (It really should be linked elsewhere, since it is regularly used by battlers for potions and shards, but that's literally the only place to find it. I'm pretty sure they don't even mention it in any of the guides.)