2008-02-25 16:57:55
Improvements to backups
For the past weeks, if you are an early bird at least, you may have seen these messages in-game:
Server backup processing… You can keep on playing, but access to the server is temporarily blocked. You will be able to access the server within a few moments. Thank you for your patience.
And a few minutes later...
Server backup completed. You can now access the server. Thank you for your understanding.
We have finally introduced a new system which allow us to backup data much more securely than before.
Historically, backups done during the day did not take into account ‘in-game’ elements which were being modified at the time. Only data outside the game, that is data of disconnected players, were saved in their final form.
To illustrate this, let us take the example of inventory backup. Inventories are backed up every X minutes and again when players disconnect. As a result, in the event of a server crash, certain data may not be up to date in the database. For example, you lend your Bora to a friend, you disconnect, you are saved without the Bora. The server then crashes before your friend’s details have been backed up. Your friend will also be saved without the Bora and so the Bora will disappear. Conversely, if you lend your Bora to a friend and stay in game and your friend disconnects, both of you will be saved with the Bora in the event of a server crash, and so the item is duplicated.
The only backups which were 100% secure were those done during the total shutdown on Tuesdays and Fridays, since server activity was nil.
All this has since changed. Backup now includes in-game elements, characters are frozen for several tenths of a second, in order to prepare the backup, and everything is correctly saved.
The cost of this procedure is minimal. Connections to the server are refused for its duration. In fact, to ensure that data is not changed along the way, the database is frozen for reading as well as writing.
Very soon we plan to back data up daily, probably around 6am. We are trying to establish other ideal times to back data up. Ideally this would occur around 11am and again around 4pm. The goal is to have regular backup events which will ensure that any rollback that occurs due to a crash (which we would rather avoid altogether) will be of a few hours at the most. So players will no longer lose huge amounts of data.
Since blocking connections to the server is disruptive, we must avoid periods of high volume. Peaks in connection, during the week at least, occur around midday and between 6pm and 10pm.
Server backup processing… You can keep on playing, but access to the server is temporarily blocked. You will be able to access the server within a few moments. Thank you for your patience.
And a few minutes later...
Server backup completed. You can now access the server. Thank you for your understanding.
We have finally introduced a new system which allow us to backup data much more securely than before.
Historically, backups done during the day did not take into account ‘in-game’ elements which were being modified at the time. Only data outside the game, that is data of disconnected players, were saved in their final form.
To illustrate this, let us take the example of inventory backup. Inventories are backed up every X minutes and again when players disconnect. As a result, in the event of a server crash, certain data may not be up to date in the database. For example, you lend your Bora to a friend, you disconnect, you are saved without the Bora. The server then crashes before your friend’s details have been backed up. Your friend will also be saved without the Bora and so the Bora will disappear. Conversely, if you lend your Bora to a friend and stay in game and your friend disconnects, both of you will be saved with the Bora in the event of a server crash, and so the item is duplicated.
The only backups which were 100% secure were those done during the total shutdown on Tuesdays and Fridays, since server activity was nil.
All this has since changed. Backup now includes in-game elements, characters are frozen for several tenths of a second, in order to prepare the backup, and everything is correctly saved.
The cost of this procedure is minimal. Connections to the server are refused for its duration. In fact, to ensure that data is not changed along the way, the database is frozen for reading as well as writing.
Very soon we plan to back data up daily, probably around 6am. We are trying to establish other ideal times to back data up. Ideally this would occur around 11am and again around 4pm. The goal is to have regular backup events which will ensure that any rollback that occurs due to a crash (which we would rather avoid altogether) will be of a few hours at the most. So players will no longer lose huge amounts of data.
Since blocking connections to the server is disruptive, we must avoid periods of high volume. Peaks in connection, during the week at least, occur around midday and between 6pm and 10pm.
Add a comment