Hello all, Alex here.

First off, let me say happy new year to everyone! I really enjoyed my holidays this year, and I hope you did too.

That said, I'd like to make a new years resolution of sorts. Just prior to Christmas, our "recode" version of wardrobe was opened to Gold Account holders for beta testing, and will probably officially replace the old wardrobe before the end of January.

The Recode is a total replacement of Subeta's programming and is absolutely vital to ensuring that the site can continue to grow. The problem is that we're already feeling these growing pains. Recode will eventually fix these, but only when a substantial part of the site is running recode programming. I don't want to wait that long, and I'm sure you don't either.So I'm beginning a "stability project" for Subeta, of which recode is only a part. The stability project will address issues in Subeta's code, servers, database, and handling of downtime. The goal: 99.9% uptime/month by Dec 31st, 2009. I think we can make it!I won't go into much detail about the project here, but I will list some key points:

* Automatic recovery from SQL lockups: This is actually already working. You may have noticed (I hope you have) that the site doesn't go down for hours at a time now. The lockups are now detected and the database reset automatically, within minutes of it happening. No more waiting for Keith or myself to notice the problem and correct it.

* Tune-up of Subeta's old code to reduce the load on the database and eliminate "bottleneck" points that cause lockups.

* Changes to old code necessary to support running the site on a MySQL master/slave system instead of on just one database server.

* Downtime tracking and reporting! We'll need to keep track of when and how long the site is down to measure progress.

* And many more small changes that I hope will make a big difference!


None of this can happen overnight, but expect to see things gradually improve over the course of 2009, starting soon! Thanks so much for being patient with us in the meantime!I hope that was simple and understandable enough.<br />

I'll address a couple of FAQs:

Will this delay the recode (much)?

Yes, but like I said, better to fix the problems now rather than wait for recode to fix them later. It shouldn't delay it by much though, recode is part of the stability project and ultimately the goals are the same.

That sounds nice, how can I help?

I may have need of some testing volunteers eventually, but for now please just be patient with us.

Why haven't you done this sooner?!

Really, it is a work in progress. Up until now though, there really has been no yardstick for progress made, other than "we had a lot of database problems this month," etc. The idea of the stability project is to attain a stability goal (99.9% uptime/month) by a deadline. With the ability to measure downtime, we will be able to focus our efforts on those activities that are having the greatest effect, which often are not the ones you'd think would.

Does this mean you will be fixing the ______ bug?

Most likely, current bugs will be fixed when their area of the site is recoded, unless they are related to stability problems. As it is, it is hard to tell whether bugs are caused by bad coding or simply what I call "server weather conditions" (e.g. high load causes a mysql connection reset in the middle of an attempted trade), which are more difficult to track down.

As always, we really appreciate your patience and support. Without you guys, we wouldn't have anyone to do all this work FOR!

December 31, 1969, 7:00 pm by Kris
Avatar Border
User Avatar

Thank you for your hard work. Happy New Year.

December 31, 1969, 7:00 pm by story_168
User Avatar

Oh, and, no matter what anyone says, SQL will always be pronounced Sequel in my mind. :C

December 31, 1969, 7:00 pm by Innerflame
User Avatar

It is so worth the wait. You and your team do such wonderful work here and this past holiday proved your worth. Thank you for a great place to have fun and find friends.

December 31, 1969, 7:00 pm by jimbogogo
User Avatar

Yes-many thanks for keeping us players informed. This is very reassuring and am sure you and your staff will do a great job.

December 31, 1969, 7:00 pm by PrincessOfDenial
User Avatar

Good luck with all of this mumbo jumbo! I hope the site works better when it's all said and done. :]

December 31, 1969, 7:00 pm by TheCryptKeeper
Avatar Border
User Avatar

Again, a huge 'THANK YOU' & Kudos to all concerned. Without your time and patients the site would not be as awesome as it is.

December 31, 1969, 7:00 pm by Deleted User

right on kieth , what ever all that meant , haha , but yep fix away and i will still be here

December 31, 1969, 7:00 pm by Opa
User Avatar

It's good to hear that there will be some changes soon. I think the site really needs it. Because it seems like there's more down time than up. I've seen the site go down more than 4 times today. It's nice to hear the staff getting together and making the site better.

-Tiffany

December 31, 1969, 7:00 pm by Pato
User Avatar

I Semi-agree with her statement.I mean does seem like alot.I just feel bad for the people who pay for the gold accounts every month, or pay for them yearly and have to deal with all this downtime.

No, im not including myself, i got mine for free. Im just saying for people who pay faithfully.

Thanks for the Update! it's nice to know this will get fixed eventually.

December 31, 1969, 7:00 pm by LizBlackDog
Avatar Border
User Avatar

Pshaw. Call this downtime? When I were a lass we had to complete Saggitarius' quests uphill in the snow both ways. And downtime lasted all winter.

Alex, you rock. love

December 31, 1969, 7:00 pm by Naughty
User Avatar

To be honest ( no one shoot me, get the Damn squirrel!! ) when a site grows it has an AWKWARD period this is what we are enduring as Subeta works to adapt to that said growth! Also, me not liking it mo than anyone else but this is how it is!!

December 31, 1969, 7:00 pm by PUNK
User Avatar

I'm not sure what you were trying to start, Collage - but you appeared rude and ungrateful. Shame, these people work so hard to give us a great site to use, and you do nothing but complain

December 31, 1969, 7:00 pm by alx
User Avatar

We really only have one reason for unscheduled downtime: when the database server locks up (I seem to recall mentioning this in this post).

The database server really only locks up for one reason: when it gets overloaded processing queries, and can't get them done faster than they're coming in.

If you're looking at Subeta as a big black box (like most users) I can see why you'd assume that changes have to be responsible for problems. This is a logical fallacy though - correlation doesn't imply causation.

Think of it as loading a bunch of feathers onto a board. Eventually, one feather is going to cause the board to break, but is it that feather's fault? Of course not; there were simply too many feathers on the board at one time!

December 31, 1969, 7:00 pm by SING_746
User Avatar

I love it how you care enough to let us know what's going on. It's really appreciated.

December 31, 1969, 7:00 pm by NAT
Avatar Border
User Avatar

LOL @ the squirrel collection.

Tank you Keef and Ayleex. It's always really reassuring to read these news posts about the recode.

December 31, 1969, 7:00 pm by paranoia
User Avatar

x__X

I wasn't trying to start a debate, and I'm sure I wasn't the only one thinking it - I just wanted to get it off my chest.

December 31, 1969, 7:00 pm by Etnik
User Avatar

Cool! I can't wait for the new wardrobe!

December 31, 1969, 7:00 pm by Sly
Avatar Border
User Avatar

I just got a MuSQL page, and in minutes the site was back up. You guys really are awesome. Now I don't ever have to find something else to do with my time ever again.

December 31, 1969, 7:00 pm by Music_188
User Avatar

Thanks for the information, Alex.

December 31, 1969, 7:00 pm by Cee
User Avatar

Quote:

; border: 1px solid ; font-family: georgia; font-size: 10;">Honestly, I understand that there are problems with the site but the continuous downtime recently started up within the last month - which means that something you guys did within the last month screwed up the code and is making this downtime occur.

You mean like the giveaway? The huge week long giveaway which 1000 people probably refreshed 10 times a minute?

BAD KEITH. DOING A GIVEAWAY WHICH RESULTED IN DOWNTIME Dun do eet no more. That shud fix.

December 31, 1969, 7:00 pm by JESSYTA
User Avatar

Quote:

; border: 1px solid ; font-family: georgia; font-size: 10;">I'm just saying, I don't think high traffic can be to blame for all of the downtime.

I am going to go out on a limb and say that Keith and Alx are both more knowledgeable about what causes a site to go down... And if Keith says it is due to higher then normal traffic, I will believe him.

December 31, 1969, 7:00 pm by lewysgal
User Avatar

Dear Alex, You are what a programmer SHOULD be like! I didn't understand any of the technical parts of that but thank you for the update! I was wondering what was going on! Do your best and don't forget to stop and get some sleep now and then!

December 31, 1969, 7:00 pm by Sopheroo
Avatar Border
User Avatar

Quote:

; border: 1px solid ; font-family: georgia; font-size: 10;">I am glad that there is now an automated thing to make the site go up when it breaks down, but it has gone down at least 5 times in the past 3 hours.

Without the script, it might have gone down for three hours, with no comeback. One hit KO.

You were informed. I don't know of any petsites that will give you as much information on the current situation.

Again guys, I thank you personally for your patience concerning our site issue. It means the world to us, and your loyalty is what make working for YOU, the users, worth it.

December 31, 1969, 7:00 pm by Naughty
User Avatar

OMG I luff you Alex!! ( The squirrels suck though!! ~

December 31, 1969, 7:00 pm by GryffindorRUFC1
User Avatar

Came here to post that I think it is great and got a Mysql error....too funny lol!

Good job though. Stable site = Happy subetans!!

December 31, 1969, 7:00 pm by paranoia
User Avatar

And there it goes again. :|

I'm just saying, I don't think high traffic can be to blame for all of the downtime.

And - If a good company is having problems in real life, they'll inform their customers if they don't want to lose business. Sure, they don't have to - but it is the right thing to do, and yes, Alex has just done that for Subeta.

December 31, 1969, 7:00 pm by Chazezmom
Avatar Border
User Avatar

Squirrels everywhere! thanks for making a logical, well thought out plan...and informing us. You guys do a great job. and I still want squirrel stew

December 31, 1969, 7:00 pm by Miazaki
Avatar Border
User Avatar

Quote By collage:

; border: 1px solid ; font-family: georgia; font-size: 10;"> When are we actually going to be informed? :

No one has to inform you of diddly squat. They could just update the pets, and take the site down with no explanation. This is a service to you.

December 31, 1969, 7:00 pm by Tris
User Avatar

GO KEEF!! XD

I would have thought it was obvious why the downtime was happening. Giveaway, winter break, advent calendar. December is the busiest time on Subeta. It doesn't take a genius to figure out why we always have problems now. Or perhaps it does...

Anyways, thank you for this post. Looking forward to the new Subeta when it's all done.

December 31, 1969, 7:00 pm by Chazezmom
Avatar Border
User Avatar

Squirrels everywhere! thanks for making a logical, well thought out plan...and informing us. You guys do a great job. and I still want squirrel stew

December 31, 1969, 7:00 pm by Keith
Avatar Border
User Avatar

And I am trying to force myself not to respond to the rest of your comment.

Quote:

; border: 1px solid ; font-family: georgia; font-size: 10;">I don't think that type of measure should ever have to be put in place, or needed - but it has been needed numerous times just today.

You're right -- it's not something that should have to be put in place. I wonder if that's why we've made this post with the plan that we're putting into motion, with the goal of 99.9% uptime -- which would mean that this measure wouldn't need to be taken anymore.

December 31, 1969, 7:00 pm by Miazaki
Avatar Border
User Avatar

Quote By collage:

; border: 1px solid ; font-family: georgia; font-size: 10;"> When are we actually going to be informed? :

No one has to inform you of diddly squat. They could just update the pets, and take the site down with no explanation. This is a service to you.

December 31, 1969, 7:00 pm by PUNK
User Avatar

hehe it looks great, Carol

December 31, 1969, 7:00 pm by Midnight_916
User Avatar

Thank you for the update!

December 31, 1969, 7:00 pm by Carol
User Avatar

Sorry for the formatting issues. I was trying to pretty it up, and I ended up breaking it, LOL. It should be easier to read now.

December 31, 1969, 7:00 pm by Deleted User

you guys work so hard to help make this site better. you deserve a plethora of thanks and appreciation.

December 31, 1969, 7:00 pm by SexyFart
User Avatar

I really appreciate these updates! Thank you. I'm really excited to have the new wardrobe and to see Subeta continue to grow and change in the coming year!

December 31, 1969, 7:00 pm by Deleted User

Happy new year, and thanks for the updates! It's always nice to know what's going on, and if you ever need a volunteer, I'm always up for volunteer work :3 At least volunteering online is more interesting than a food bank xD

December 31, 1969, 7:00 pm by Deleted User

lmfao yes the squirrels have been running rampant today, its good to know it's all part of the big plan. thanks for keeping us informed.

December 31, 1969, 7:00 pm by Faun_171
User Avatar

Thanks and cheers C;

December 31, 1969, 7:00 pm by fab
User Avatar

YES! You guys are awesome! I've been waiting for this! -smush- I love you Subeta staff! And I hope we can reach our goal of 99.9%! Good luck to Alex, Keith, and everyone else who's working on this!

December 31, 1969, 7:00 pm by hourglass
User Avatar

awww, thanks so much for working out the kinks for us!!! good luck

December 31, 1969, 7:00 pm by hnook
User Avatar

Thanks very much Alex. Sub is a terrific site...and i would presume a lot of the downtime of late has been because of the great numbers of people loving the joint! Its usually when we hit the 1100 mark of members online that Sub starts to sink. IT IS just a matter of being patient and trying again in a minute or two!

Thanks again for the explanation.

December 31, 1969, 7:00 pm by Purity
Avatar Border
User Avatar

Thanks for explaining everything to us users.

Happy New Year to you too!

And I would be happy to help anytime.

December 31, 1969, 7:00 pm by Keith
Avatar Border
User Avatar

Quote:

; border: 1px solid ; font-family: georgia; font-size: 10;"> Honestly, I understand that there are problems with the site but the continuous downtime recently started up within the last month - which means that something you guys did within the last month screwed up the code and is making this downtime occur.

Or... the month of December when everyone is off school is a high-traffic month, that we get more users coming on than we're used to?

NO, WAIT. LOGIC. THAT CAN'T BE IT.

IT'S SOMETHING WE DID.

What do you call this post?. This is addressing it. We waited until we had information, and a plan. If you want a post when something happens in the future, sure. I'll make a one line news post that says "Sorry we're having a lot of time Not sure what's going on ".

Now we know what is wrong, and we're working on a fix already.

December 31, 1969, 7:00 pm by theloveofGreed
User Avatar

eeeek! Help me I'm lost in the words! -HIGHLY dyslexic- XD Good luck and yay? I just thought it was funny as hell that the site started acting up during an event. I say the boxes should be a bit easier to find due to the site messing up instead of finding 1 in like.. every 4 hours.

December 31, 1969, 7:00 pm by Nebulae
User Avatar

Thanks for letting us know. =]

December 31, 1969, 7:00 pm by kittiocity
User Avatar

Whoa. o: Thanks for telling us. And I would help with the testing, if anyone can, not just GAs. |D

December 31, 1969, 7:00 pm by paranoia
User Avatar

Anyways thank you for this update that doesn't really tell us anything about the recent downtime - and happy new year to you too. :

December 31, 1969, 7:00 pm by Deleted User

Yay! I just want to say thanks for being so great guys! We really appreciate you working so hard! Subeta is such a fantastic site and the wonderful staff contributes SO much to that. We love you guys!

And happy new year to you too!

Leave a Comment

Use @username to mention someone. Supports markdown formatting.