Replies

Mar 26, 2020 5 years ago Official
Keith
is sweet
User Avatar
Eradication

Hey everyone! Not a post I want to be writing, but here I am!

: We had a bug that deleted a swath of items from the wardrobe. There isn't a very good reason for this, except that it passed our automated tests because the item did get deleted, just, not once. I could get into n+1 problems and a runaway query, but that wouldn't be very interesting.

: Unfortunately, the data isn't recoverable from our backups. There also isn't a very good reason for this, other than "our backup service provider stopped making backups and we didn't notice because we never need them", but again, not that interesting. We're resolving this problem today (and have manually saved a copy of the database as it is right now) by doubling up on backups and setting a monitor to check them.

These are good things to learn and fix, terrible things to have happen together. We're currently looking for a way to piece together data from other tables (logs, avatars being saved with items that you don't have, outfits saved, etc) to figure out how to make this right, and we'll be providing some CSC to everyone as well. If we can't get the information in that way, we'll just operate on the honor system as best as we can and figure out a way to take care of everyone.

We're working hard to resolve this in a way that leaves everyone off better than they started as quickly as possible, it's a rough time right now generally and we want to make sure this is a happy place you can escape to. Please keep in mind that we're all working on this, and likely have real world things as well, and be kind.

Thank you and please wash your hands Keith :two-hearts:

[edit] (edit from ) Please see this update later in the thread that addresses some of your immediate concerns. We will have another update soon and will properly notify everyone when that is posted.

💖 ✨ 🤗

Mar 27, 2020 5 years ago Official
Bug
User Avatar
Segfault

I'm popping in quickly to answer what concerns I can, because I understand this is a very stressful situation, and I don't want to leave you hanging for long.

  1. The issue that caused this is fixed and no longer happening.

  2. We very much want to restore items to accounts; we are working on figuring out how to do this. We understand many items such as CWs are not replaceable and we are absolutely going to do our best to restore these and all the other items. The CSC mentioned will be given out on top of all that as extra compensation.

  3. We own up to this. Failing to notice that backups stopped working was a serious, unacceptable oversight. As Keith mentioned, we have now fixed our backup service and added monitoring to prevent this from happening again. That said, we understand that that does not excuse our error and does not make the current situation better.

  4. We didn't choose to not ping siteupdate; we forgot. However, we will be making a more detailed post about this tomorrow, and we'll make sure to draw attention to that post.

This is all I can answer for now. There will be more details posted tomorrow as well as a chance to talk to us about this further; we want to hear your concerns and make sure they are all properly addressed.

🐝 ☕ bug (he/him) | your friendly neighborhood code wrangler. stay in the loop! join and check out the latest admin post highlights

Mar 27, 2020 5 years ago Official
Bug
User Avatar
Segfault

We 100% agree with you, and we will draw attention to our post tomorrow since it will have more useful details for everyone.

🐝 ☕ bug (he/him) | your friendly neighborhood code wrangler. stay in the loop! join and check out the latest admin post highlights

Mar 27, 2020 5 years ago Official
Bug
User Avatar
Segfault

I'd also like to note that we're aware the wardrobe is showing broken images instead of items right now. This is an unrelated issue and we're actively working on a fix. The broken images are a display error and do not affect your item data.

🐝 ☕ bug (he/him) | your friendly neighborhood code wrangler. stay in the loop! join and check out the latest admin post highlights

Mar 27, 2020 5 years ago Official
Rah
is magical
User Avatar
Rah

Good morning everyone - just wanted to drop in and say the programmers are having a brief rest after working on this last night. We are going to do our best to get things back in your wardrobes, we know that this is a big deal, and once we have some more solid information about what's happening we'll be updating you.

Until then, posting items that you know are missing is helpful.


Rah image drawn by the dear !

Mar 27, 2020 5 years ago Official
Bug
User Avatar
Segfault

We've just made our official announcement about this issue. Please give it a read here: Missing Wardrobe Items Glitch

We are now also taking outstanding questions and concerns on that topic. We'll be doing our best to address everything that comes up in a prompt and satisfactory manner.

🐝 ☕ bug (he/him) | your friendly neighborhood code wrangler. stay in the loop! join and check out the latest admin post highlights

Please log in to reply to this topic.