👋 Hello everyone! Wild that we're almost in March... 🙃

I'm tracking multiple problems on the site that seem to come from the same place, and one that will only continue to get worse as time progresses.

We let players create 100+ pets which is great, and allows for creativity and characterization. Unfortunately, we're seeing the N+1 problem in a lot of these queries. In simple terms, that means that we're fetching an increasing and increasing amount of data on pages (think: training center, pet treasure, anywhere that we list your pets out for you, especially in a dropdown).

An example: Fetch all pets to display in "use item" dropdown → fetch minion information for pet → fetch books read information for pet → fetch battle status for pet → fetch treasure count for pet to see if they're over limit → fetching treasure count automatically fetches item information for each treasure item → etc

We cache as many separate pieces of these queries, and the total response, to the best of our ability but the query still has to run at least once when you first go to the page. We "warm up" the cache with your information as you take your first trip around the site for the day, and so those initial queries are going to be painful and slow, especially as you get more and more pets.

Obviously players with 100+ pets are some of our most passionate and engaged players, so we want to make sure that any changes doesn't hurt y'all, but makes the site actually playable.

So I'm proposing a pet farm (or some similar idea / name!). You can have 100 active pets at a time, that show up in dropdowns around the site, and any additional pets are on your farm, and you can move pets on / off the farm with a simple click, and no time limit. Think of it like "favorite pets", but just as large of a category as it seems like we can make without slowing down or crashing the site for folks with lots of pets.

Let me know what you think about this idea, or if you've got any other ideas that could lead to the same optimizations.

Update I posted this in another post here to clarify!

For those of you with less than 100 pets, but still want a cleaner "option" choice when seeing pets, yes you could move all but one to the farm theoretically!

Ohhhhh, that is really helpful, thank you! I'm tracking all of the pages on the site that make the ridiculous queries and found that most of the time 100 pets was the sweet spot where it'd still load. I'll probably ping you and ask if it's okay for me to take a benchmark of your account loading the battle stats page, since I imagine that is a HORRIFIC use of N+1 queries.

My (vague, since I don't want to propose anything too far down the road anymore) idea is that on any page that has a list of pets (so the stats page, for example) would just limit your "team" of pets, which you choose yourself and put any ones you don't need / want loading on those pages into the 'farm'.

You'd still be able to see all of your pets in the farm and take action on them there, basically the same things you'd do from your pet page (description, treasure, etc) without them all loading around the entire site.

Places where you need to decide on a pet (typically in a dropdown, or a list) would only show from your non-farm pets.

Ohhh I love the idea of the farm also being a place where the pets there are taken care of (fed & played with) and maybe even down the line letting you choose how those things happen. That way you don't need to worry about adding them to the farm.

I don't mind losing the sP sink from keeping your pet in the inn tbh, and like this much better from a friendlyness point of view!

Maybe somewhere down the road, but I get nervous even thinking of the complexity 💀

Hmmm (and this is me spitballing here, trying to be very conscious of saying that!) what if pets in the farm/den (love that name !) still trained automatically in the background but at a much slower pace than putting them into the training center? Maybe instead of earning stats, they'd earn a slow but steady amount of XP over time. That way folks who don't want their pets getting stats can choose (via XP) what to get, and there isn't wasted time for any pets.

Or do you put all of your pets into training each time it's available?

From a writing perspective, imagining the pets running drills with each other or something ;)

Do you just run down the jobs page and collect the income from everyone? Is there anything else that you've got to do to 'keep them active' or anything?

If not, I wonder if it just... automatically collecting the pay while they're in the den, or something similar to that?

I want to basically be a place that you put pets that you want out of sight / out of mind for most of your browsing around the site, but to not be.... dead.

So, you're talking about the exact problem this is attempting to resolve! "Something is broken for me where I can't battle or do anything through the job agency" - That is the site attempting (and failing) to get all of the information about your pets to display.

It's complicated, and I hope I can do a good enough job of explaining, but it's a combination of the number of pets, and the amount of data associated with those pets.

On the account search page for example, it's searching all of those locations, and if you have more data there, it'll take longer (or just fail) to load. Examples there are it needs to validate if the item is in your wardrobe (so searches all of your wardrobe items), in a collection, etc. As you have more and more things in those places, it takes longer for the database to search for your specific data.

So if you have a lot of treasure items, minions, books read, training, battle history, etc for 40 pets, you're going to have a harder time loading those pages than someone who has only 120 pets but barely any data associated with them.

I hope that makes sense, and it's true that a lot of this hinges on moving these queries over to the site new site where they're optimized more from day 1, but also making choices like this one to reduce the amount of overall data coming back regardless.

🤞 We can make some improvements for you here, it is really frustrating for me to see that the people who've put the most into the site are the ones most slowed down and lagged by it :(

Nice try getting that into this thread ;) I've got that on my list, and we're going to get those items out. You'll be able to complete the collection, I promise!