does it apply to all pets or just the ones you want, or just one?

You have to continually feed it to your pet(s) for it to work. It doesn't just keep them fed automatically (if you're looking for something like that, try the Resort or the Inn on Omen Islands). But because of that, you can feed it to as many pets as you want

so do you have ti buy multiple, or does it just stay in your inventory?

You only need one, and you can keep it in your inventory or vault or wherever you want to keep it when you're not using it. But it won't keep your pets from getting hungry just by sitting there, they'll still get hungry and you have to actually feed them the berry to fill them up. It just won't disappear after feeding like other food items do, so you can feed it to all of your pets over and over without losing it.

You have to choose "Feed to [pet]" to use it on a pet. But once you do, unlike other food items, the Infinity Berry will automatically make their hunger go to 10/10. The Infinity Berry doesn't disappear after you feed it to a pet, so you can use it as many times as you want.

Source: I have one locked in my inventory (so that I don't lose it), and I use it when I need to battle with my pets. It always fills them up to 10/10.

Thanks for clarifying for me! :)