Friday, November 14, 2008

The problems with TCP and how to deal with the lack of UDP in Flash (Part I):

In order to even consider doing network programming in flash, we have to thoroughly understand the limitations, and what that means for your project. In this article, I will be specifically talking about the limitation in Flash CS4 to using TCP only. I will not, however, be covering the consequent lack of peer-to-peer communication in this article, which is in itself another major issue and being stuck with the client/server model is a major source of lag.

We all know that TCP/IP is awesome for its reliability and its guarantees, but the 'faster' UDP has to be the protocol of choice for a fast-action game like a multi-dimensional racing game. But, why exactly, and what will happen if I just go with TCP anyway, since UDP is not permitted in Flash? "So what?". "Whats the big deal?". Well, after my experience with the game I am currently making, I can answer that question quite precicely.

The TCP/IP protocol is really cool because it guarantees that unless the session gets disconnected, if packet B was sent after packet A, then B will, guaranteed, arrive after A. Now, this is very convenient. Firstly, it has an implicit guarantee, that every packet you send WILL arrive. So, there is no danger of 'packet loss'. A packet, simply, cannot, be, lost. Awesome! Furthurmore, suppose in a platformer I give the jump command, and then shoot in the air, I know that on the other side my guy won't shoot first and then jump. That would be bad. Finally we avoid all the nonsense of router business, which although is also a problem with TCP, it would be especially more for UDP, which often people use with peer-to-peer communication. By limiting itself to TCP, and not permitting peer-to-peer, it was much easier for Adobe to provide a solution that prevents the user from having to ever deal with router ports and all that. So, this is awesome. No one wants to deal with mess like packet loss and packet reordering. What could there possibly be not to LOVE about TCP?

Well, the TCP/IP protocol guarantees that, no matter what, no matter the cost, every packet will arrive in the order it was sent. And, unfortunately, there is a cost. There is a cost that can sometimes be in the magnitude of seconds of delay. Packet A will arrive before B and C. Even if B and C got to their destination ages ago, the protocol will tell them, 'guys', 'sorry but you're gonna have to wait'. 'We have a lagger'. Then finally when A arrives, all 3 of them, A, B and C are immediately processed at once together, so you'll get a bunch of packets at once.

There are many reasons why A could have gotten delayed. Perhaps it took the 'long way round' in the network. Perhaps it tried to take the shortest route, but unfortunately, also the most congested route. Perhaps it gost lost in the ether some where? 'Oh oh. Now what?' wonders TCP/IP, 'I know! I'll just get the source to re-send A :)'. So again, TCP makes the source try to send A again, and along goes A through the network eventually arriving at the destination. If it keeps failing, then the protocol will eventually give up and disconnect the entire session.

Ok so now we know how TCP, in theory, can mess up, but how can I make it mess up practically? Well, I have found that, in general, the more packets you try to send simultaneously (e.g, in the same Frame), the longer the delay. There is only a certain number of packets that can be reliably sent in a certain period of time, and this number is not undeterministic. For example, in my game an event is sent for every arrow key you press or let go. So suppose the player let go of the right key and started pressing the up and left key all in the same frame or in close proximity, and if I try to send all 3 events as packets, then similarly my server receives all 3 events and broadcasts them to all other players (this aspect itself is very fast, but it is also trying to send multiple packets to the same player at the same time which comes back to the TCP issue; also note that TCP does not support 'multi-cast', unlike UDP which would have allowed the network card to send the same packet to multiple destinations at once *cringe*). All in all, this can result in a noticable delay, more than the normal round-trip delay caused by lag (which on its own would be up to 300 ms within the US for Flash games).

So this is the problem. And unfortunately, this article is already too long, so I will continue with the solution in a future 'Part 2'.

1 comment:

meran said...

Interesting read but I was wondering if during your experience you came across any issues with socket connection and Nagle's Algorithm. One serious problem that I am currently having is after calling flush() the packets sit in the flush buffer until 200ms time expires and any packet that is within that buffer gets sent to the server. And yes I know for a fact that it sits in the buffer and I can prove it.

The process of the 200ms que of buffer begins again which is killing my project(s).

I have 2 solutions around this issue but none of them are acceptable. If you have come across this issue and have a working solution then please do share. Either way your article was an interesting read.