I'm trying to enter the new cash shop items and SubetaLodge is being extremely laggy. Not talking waiting an extra few seconds here, but actual minutes.

-- yup, you're right. I see what the problem is but not what caused it. I've restarted the server in hopes that'll fix it. If not, I'll look into it further.

Hmm. It appears to be a user's sidebar widget that counts the number of items in their collections -- they have a number of custom collections, and having to do the custom counting is slowing things down. I've got an idea for this, but it might have to wait for a day or so -- I'll be travelling all day tomorrow.

yeah not a great time of year for dealing with fiddly stuff - not just personal stuff but bunches of things going on here too.

Stay safe on your travels!

-- for the time being, I've taken the widget offline. Let's leave it offline for a few hours and see if that helps any. If it does, then that's definitely the source of the problem (and if not, I need to keep investigating!)

Its much improved, thank you! Not waiting minutes between loading, but its still no where near snappy.

I'm experiencing alternate periods of lag still on the site (random and less frequent though).

Also I'm updating the subscription items with a note that they were added to the CS for the BF sale and giving them a price and restock location. I know we need to deal with some additional updates and changes but I have (sadly, like all I'm sure) limited time this week. I'm working every day except Thursday and I've also got my best friend temporarily moving into my apartment after he was illegally intimidated out of his despite being on rent relief so everything is a hot mess haha.

If anyone knows what needs updating vs what is in need of big changes etc and can point it out, I can help when I have time.

[edit] Updated the Little Devil Horns, Goat Hoofs, Taurhoofs, and Wildcat Manes

Thanks for those updates - I did make note of them when I was adding the cash shop items, but completely forgot about them after dealing with the lag issues.

, : I've updated the news banner to acknowledge the lag and mention the work we're doing on it.

There are 251 items in the database which are marked as unobtainable (mostly NPC clothes and weapons). We filter those items out of every collection query. I'm thinking of moving those items out of the main database so that we don't have to filter them every time; we can put them on a separate page like we do with hexes and zapper skins for people who are interested in them.

Do either of you see a potential problem with doing this?

I don't see a problem. Some of those items have been around for eons...if Subeta ever gets around to adding them and making them available to users, we can also add them back to our database.

I don't even really see a need to make a separate page for them, but I don't suppose it wouldn't hurt, and might be good for the sake of completion.

Okay: I've removed the unobtainable staff-only and NPC-only items from the main items table. This is helping. The next batch of slow queries are custom checklists based on item notes (such as Morostide items or Free Gifts). Not sure that we can do much about those, though.

In case anyone else is interested, here's the last seven days of slow-query logs (the first number is the size of the log.) Fireside started on the 21st, and then Black Friday started yesterday, and the log was more than double the size of the day before.

16380055 Nov 24 mysql-slow.log.1.gz 7781889 Nov 23 mysql-slow.log.2.gz 5774616 Nov 22 mysql-slow.log.3.gz 4743886 Nov 21 mysql-slow.log.4.gz 126573 Nov 20 mysql-slow.log.5.gz 89395 Nov 19 mysql-slow.log.6.gz 2578844 Nov 18 mysql-slow.log.7.gz

Not sure I understand what's going on.

What I think: When we add a new item, the database adds the item, then goes over every single checklist on the site and adds items as appropriate to existing checklists?

The checklists we make - like plushies, beanbags, etc goes fast because it only needs to check one thing. Except that it used to make sure the item wasn't 300 - so removing the unobtainable items means it doesn't need to do that check. So faster still.

But custom checklists can have multiple checks so that takes longer, especially when its text based like notes.

Is there anything we can do when we add items to make it smoother? For example when I was entering the new Cash Shop items, most of the items were wearable, so I checked the clothing box in the mass add. Once it was entered I went back and edited the non-clothing items to remove the clothing check and put in the book/plushie/whatever checkmark. So the question is when we are adding a mass of items that have different uses, should we leave it blank? Only add the checkmark when all the items we enter are the same item use type?

: When we add a new item to the database, it doesn't immediately add it to everyone's checklist. It adds it to a person's checklist the first time that checklist is accessed (either when that person logs on, or someone else views the public version of that person's checklist).

That way, if a user logs on infrequently, we can do all the updating for their checklists at once, without having to update it one at a time.

Assigning all the new Cash Shop items to Clothing and then changing the few items that needs to be changed to something else doesn't cause much problem at all -- it'll trigger an update for people who are logged on while you're editing, but not for everyone else.

The lag right now (and I really do think we've improved it over the past couple of days) is caused by users logging on and having every one of their checklists update. The standard checklists don't take too long, because that just requires checking the item type field (now that we've taken away the check for whether it's unobtainable or not), but there are food and book checklists for multiple pets, and custom checklists that depend on several fields, and those are what's slowing us down.

ah..okay that makes more sense :D.

Oh its definitely behaving better. Yeah, I don't really see any way to fix that without making custom checklists useless.

I am noticing a massive improvement in the lag, even with folks being even more active. So I think the changes made so far are really helping a lot.

Not a problem. I was online and happened to have the time. I wasn't sure if there was anything beyond those that I missed but if not then yay haha.

Okay -- one more tweak on the lag front.

I've taken these collections:

• Bobblehead
• Pastry
• Pumpkin
• Tile
• Trading Card

There are still some things we can do, but they'll require taking the site offline for a while, and I don't want to do that during an event. I think we've got the lag back down to a tolerable level for now, but if it gets worse again, we'll have to schedule some downtime to tackle it.

That seems to have helped a lot, at least in my experience so far. No lag at all tonight.

For some reason the image for isn't showing up on SubetaLodge. I checked it here and the image seems fine. I edited the address in case they made a subtle change, but it hasn't helped.

Also someone has another issue with mass add. It seems to be working, but the items aren't being added even though it says they are. I tested and the same thing is happening to me. https://www.subetalodge.us/forums/index.php?topic=2634.140

-- I can't get an image for Abalone Banded Comb. Even in your post above (and you say the image seems fine), the image doesn't show up. When I right-click on it and try to view it directly, I get this error:

404 Not Found Code: NoSuchKey Message: The specified key does not exist.

I reported this problem a while back.

Gah. Wardrobe again. When will Subeta ever stop tinkering with it? I'll look at it tonight.

oh...wow that is so weird. It shows up on Subeta just fine for me, I can see it in my post, I can see it shop search, account search, etc. Right click, view image also looks fine.

This is what it looks like in case you were wondering: