What an uphill battle...

Many of you have been complaining of the recent lags, so I would like to elaborate on some aspects of this. We are working on several fronts to improve the situation.

First of all: the hosting. Although it’s not the primary cause of lag issues, it limits our capacity to open new servers and it’s always beneficial to have some reserves. We were hoping to do it for the start of the year, it shouldn’t be long now. We’ve been announcing it for two months, but unfortunately it can’t happen overnight. We can’t just decided one day to ‘move house’, the servers must be prepared on the other side, their performance (which is a bit problematic at the moment) must be tested, network settings must be established and I won’t even get started on the administrative issues involved. I still can’t give you a date unfortunately, but it should happen soon.

As for the servers, we are making the most of the migration to refurbish them all. But that won’t change a whole lot as far as game performances are concerned. As it is now, the servers aren’t 100% occupied even at the busiest times during the day.

There are two types of lag, or three if you take all the elements into account.

First of all: the network lag. As for the servers, there’s nothing more to say, all is going well, though the ping speed leaves a lot to be desired. There could be several factors behind this. If it’s happening to everybody, it must be an issue with the server, or it could be due to an incident with their fibres or it could be something more suspect. If it’s not affecting everybody, it could be due to a problematic link with a particular service provider. These providers often share lines, so a problem from one provider can affect people from the others, even foreign ones who transmit information through French systems to access our servers. So if it affects only you, it must be a very narrow, localized issue, either with your computer or on the DSLAM or equivalent you are using.

As for mini-disconnections that happens in every system. Even if your MSN account seems to remain connected throughout, this doesn’t necessarily mean that it’s the fault of our server. The thing is, MSN reconnects in half a second, with DOFUS, you notice it happening, it has to be done by hand. What is more, when you’re navigating between sites, a mini-disconnection has no impact. DOFUS requires a permanent connection, so one disconnection makes a big mess. We are continuing to test a solution which will allow reconnection during combat, this solution being necessary before the launch of the heroic server, but it’s not perfect just yet. Unfortunately DOFUS was designed in this way from the outset, so there are four or five years of code to adapt…

The second type of lag is ‘residual’, which usually happens when the system becomes saturated. In general, this type of lag is minimal and almost unnoticeable because it usually only lasts a few milliseconds. When the duration of this lag (which we see in real time and which lasts for as long as it takes for the instruction from the flash client to be received and to be processed by the server) reaches a certain point, we begin to intervene. We see if there are too many chat messages being sent, and if there are, we limit them. If there are too many players on certain maps, we render them instanced (which means that only one person may occupy each map at a time and you won’t see them nor they you). If it persists, we usually re launch the server to free up its memory and resources.

The corrections introduced on the 4th of January to the friend search function was linked to this residual lag, since searching was taking too long and, with several thousand connected, it was more common to see many searches being carried out at the same time. Everyone suffered.

In fact, it is the last case which is most frustrating, peaks of lag which can go on for as long as tens of seconds. This type of lag occurs relatively infrequently (during times of high use, we can get up to three or four per hour on highly populated server) and does not last too long (usually one to two minutes). We have identified some of the problems, which will be rectified along with the migration to the new server. It is caused by a sufficiently large number of packets being sent by the server. It is not solely an issue with bandwidth, the servers very rarely exceed the bandwidth allocated them even though the connection is 20 times faster, but more concerned with the number of packets which are transmitted. Suffice it to say that sending a packet of 10,000 bytes is quick but sending 10,000 packets of one byte is much slower, above all when it concerns thousands of recipients. The duration of this lag is not excessive, but it is very visible and frustrating.

We have put a sort of a ‘sniffer dog’ in place which specifically analyses the traffic during lag peaks of this kind. We have received very precise data on which areas to improve and step by step the developers are modifying each potential source of the lags.

We therefore expect great improvements to the game on this score. The new hoster will give us a lot of leeway, since the total network structures will have doubled and load-sharing will occur in real time.

In addition, many modifications are about to come on stream, and these will be put on line with each weekly maintenance. You may not notice a thing, but these improvements will group all these small messages into larger, but fewer, messages. The idea is simple, but adapting it to the system is not necessarily so. If a small package needs to be sent, you could say that it’s best to wait until more messages arrive so that they can be grouped together. But this creates a lag for the recipient who’s waiting for the information… We must therefore differentiate between packets according to their importance in order to send urgent elements right away and try to gather other messages into groups.

Las month was more difficult than most since it was holiday season and the normal rhythm of server population (peaks between 6:00 and 8:00 pm) turned into a constant demand. This understandably contributed to the level of latency, but on the bright side, it gave us an opportunity to look over a good number of potential factors linked to the lags. If only one element lay behind the issue, we would have sorted it out long ago, but many distinct and minute factors contribute which, when they occur at the same moment, can lead to a lag peak.

So, a few words before the Tuesday/Friday maintenances. These maintenances, even if they don’t add any elements to the game, will systematically introduce changes on the servers. They also serve to clean up character databases, to wipe the servers’ memory and to carry out work on the disks and the connection databases. Without these maintenances, queues and other annoying little things would occur much more frequently.

It can happen that a bug occurs in the course of a maintenance, even if work was only being carried out on non-game elements. In this case, according to the severity of the bug and the time it takes to correct it, we have to re launch the servers, or if the problem is not too critical, we can wait until the next maintenance. But the intention from the outset, the modifications carried out, would still constitute an over all improvement, be it to performance, stability or security.

Well, there you have it. Please don’t be too harsh on my spelling mistakes, it’s late and I doubt Kaoly will have the time to read over this. I hope I have managed to answer some of your questions and given you some confidence in the next few advances we are preparing to bring to the game.


Concerning connection problems:

For some days now we have been experiencing crashes in the chain of connection, which according to reports are due to an excessive number of simultaneous connections. As much as I’d like to believe it at 6pm, I have more doubts about the origin of the problem when it happens at five o’clock in the morning.

We therefore intend to do two things right away:
• Develop crash-proof servers (well, that doesn’t exist, but we’re going to develop more powerful servers which will be able to take more of a beating)
• Limit the number of IP connections within certain time frames to put an end to improper use.
The entire structure of the game is already in the process of being redeveloped so as to solve the case of the ‘missing server’.

A few weeks ago, we put a series of server optimisations on stream, which we were able to finalize during the holidays. Some will say that it’s coming a bit late, but it’s better late than never, as they say.

We should experience a considerable gain in terms of lags, which should be much less evident in the game.
As you know, the holidays, are over, so the daytime is a more ‘chilled out’ time for the servers, but the peak period between eight and nine o’clock on some days demonstrated that these modifications brought the desired results.

We are not resting on our laurels, though. We are continuing to track the sources of the lags and constantly trying to anticipate what might cause them to manifest themselves.

Happy gaming!