Asherons Call: Extended Downtime for SC, LC, TD, and FF

SC, LC, TD, FF Extended Downtime
Early Monday morning, the database cluster that supports Solclaim, Leafcull, Thistledown, and Frostfell suffered a hardware failure. Operations was able to recover data from 7AM EDT Monday and start all four worlds back up early Monday evening. The only reported data loss were the items stored in one housing community on Solclaim.

However late Monday night, the database cluster suffered another failure. Multiple players were unable to login and got error messages. It is necessary to bring all four worlds back down.

This is the current status:

Operations is currently working to recover data from Leafcull, Thistledown, and Frostfell. This should be finished by 2AM EDT Tuesday (that's 11PM PDT / 6AM UTC). When this is done, we will know whether or not there is going to be a rollback on these three worlds.

Solclaim will be rolled back to the backup done at 7AM Sunday morning. We're not willing to risk additional incidents like the already reported housing item loss.

The data from all four worlds is going to be migrated to a new database cluster as we are no longer confident in the integrity of the original. Each world in turn will be migrated and then brought back up.

The entire process is expected to take 24-48 hours, but not all worlds will be down for the entire time. As we migrate the data on each world we will bring it back up.

We'll have the order in which the worlds will be brought back up shortly.

This is essentially the same database migration that we performed for Wintersebb, Darktide, and Harvestgain earlier this year. Unfortunately in this case it is necessary to perform the migrations immediately and for all four worlds.

We apologize again for this unplanned downtime and thank you for your patience.

__________________
Alex "Ibn" Beckers
AC1 Online Community Relations
[/blockquote][/i]