Mobile XMPP
Thursday, February 14, 2008 by darcoPosted in Jabber, IPv6
From the Android SDK M5 API Changes Overview, Dave Cridland pointed out the following:
The com.google.android.xmppService package has been replaced by the com.google.android.gtalkservice package. This is driven by the fact that the GTalk API is not XMPP compliant, and will be less so going forward. The reason is that XMPP is too verbose and inefficient for mobile network connection, and the GTalk API will be moving to a binary encoding for the protocol between the client and the server.
I'll enumerate the implications and conclusions:
- XMPP is verbose.
- XMPP is inefficient for mobile networking.
- A proprietary binary protocol would be more efficient for mobile devices.
- The former Android xmppService API will diverge away from XMPP.
I think that this direction is unfortunate, and the reasons misguided. Read on, and I'll elaborate...
NOTE: This post seems to no longer be fully inline with my current position—which is that a new (as of yet undefined) UDP-based C2S protocol using EXI would be best suited for mobile devices. Such a thing would however still be XMPP. Edited 2009-07-13.
I'll tackle these implications individually:
XMPP is verbose
I find it hard to argue with this point. In fact, I actually agree: XMPP is verbose by design. It is a truly open protocol—so open in fact, you can (for the most part) learn how the protocol works by just examining the XML stream. This is important because:
- Verbosity makes debugging easier.
- Verbosity makes the protocol quick and easy to learn.
- Verbosity provides an implicit documentation of the protocol, making it truly open.
I do not think that there is anything wrong with verbosity itself. In fact, I seriously doubt XMPP would be as popular as it is now if it were any less so.
XMPP is inefficient for mobile networking
This is a point that I can understand, but fundamentally disagree with. In order to best explain why I disagree, first I should elaborate on why mobile networking is different at all from normal networking. When discussing mobile networking, the following factors are key: Bandwidth, Roaming, and Power Consumption.
Considering bandwidth: This is an understandable concern—especially in light of the "verbosity" of XMPP. A single XMPP stanza can easily be ten times larger than the contained message. Luckily, all of that verbosity is extraordinarily compressible. By using XEP-0138 (Stream Compression), bandwidth consumption can easily be decreased by a factor of ten or more. Lowering bandwidth also has the benefit of decreasing latency.
Concerning roaming: This is another understandable concern primarily because XMPP is a connection-oriented protocol. If you are using TCP/IPv4 as your transport, you are going to be disconnected whenever your mobile device gets a different IP address(which would likely be often). This would be extraordinarily disruptive. From this point of view, I can understand why the traditional transport method between the XMPP server and XMPP client would be undesirable. In the worst case, stanzas could be actually lost without warning. However, TCP/IPv4 is not the only acceptable transport mechanism for XMPP.
The best transport solution for using XMPP on mobile devices would be TCP over Mobile IPv6. Mobile IPv6 allows a device to roam seamlessly across networks while keeping the same IPv6 address, which allows connection-oriented protocols (like TCP) to continue uninterrupted. Combine this with Teredo, and you have a device that can use just about any wi-fi hot spot for connectivity (even if it doesn't support IPv6 natively) and still retain the same IPv6 address. MobileIPv6+Teredo is a powerful combination that can easily be deployed to take full advantage of IPv6, and can be deployed right now.
If you get really creative, you can come up with some less conventional XMPP transports.
Concerning power consumption: This is arguably the most important consideration for mobile networking. The key to conserving power is making sure that the device is asleep as often as possible. If a device has to wake up every few seconds to process random presence stanzas and send TCP ACK packets, this can understandably cause some serious power consumption problems. This is a real concern because normal XMPP traffic between the client and server has a lot of what I can best describe as "background noise" stanzas.
A large percentage of this "background noise" that we receive on our desktop XMPP clients we simply don't (usually) care about on a mobile device. I would much rather have longer battery life than to have my friend's presence always up to date on my phone's roster. Luckily, it turns out to be quite easy to filter out most all of this background noise with existing XEPs, without neutering the XMPP implementation.
This is most easily achieved by using XEP-0016 (Privacy Lists) to block all incoming <presence> stanzas whenever the machine goes to sleep. This way, my phone won't be woken up whenever a random person on my roster goes idle, or changes a song on iTunes. The next time the machine wakes up because of user input, the block on <presence> stanzas would be removed. If I then wanted to view the presence on my roster, the device would then send a <presence> probe.
An unresolved issue is that of <iq> stanzas, which tend to get sent often when someone goes online. Often times these stanzas are useless jabber:iq:version queries which some XMPP clients (coughiChatcough) send to everyone who is 'available' on their roster. Bad, naughty XMPP client.
One possible solution to the <iq> stanza issue is to block them like <presence> stanzas. However, this is undesirable because it would make other users unable to initiate Jingle connections to your device whenever it is asleep. Other than to twist the arm of some of the various naughty XMPP clients to change their implementation, I'm not sure what a good solution would be. It is a problem that I think could be easily resolved, even if it meant a new XEP.
A binary protocol is more efficient for mobile devices than XMPP
In order for a proprietary binary protocol to be more efficient for mobile devices, it would need to be shown that the existing XMPP protocol is not efficient for this purpose. While I would agree that a naïve XMPP implementation would run into some serious problems on a truly mobile device, I hope I have illustrated above why the protocol itself is not the problem. I can think of no viable reason why a proprietary binary protocol would outperform a reasonable mobile XMPP-native implementation.
Trackback from your own site.
Entries (ATOM/RSS)
Friday, February 15, 2008
Great post, very informative.
You brought up the issues XMPP has with roaming between various IPv4 networks. I've been quietly wondering about this problem myself. You mentioned IPv6 as being the solution to this issue and that Teredo might be able to help fill the gap until we see more native IPv6 deployments. I was wondering though; if an XMPP client is connected via IPv6 facilitated by Toredo wouldn't the connection still be dropped when the underlying IPv4 connection is broken while moving between wireless networks? I admit that I'm no networking guru and have no experience with Toredo so I may be missing something (e.g. because Toredo is a UDP protocol and thus connectionless maybe the IPv6 tunnel is left intact when moving from net to net.)
Again thanks for the post.
Monday, February 18, 2008
Learning, debugging, etc. is 0.1% of a protocol's usage. The price is too high for XML's verbosity.
Monday, February 18, 2008
Andrisi: Did you read the blog? You can compress the xml stream, and very well to, as it is based on characters. And with an Jabber-transport, you can filter what be set when to the client.
Adam: The thing is that IPv4 is used as an transport. If you change that, you do not need to change IPv6-address, as long as your new IPv4-address can reach you home server, where you have your real IPv6-address. Thats why it is mobile. You can even do better tricks with IPv6.
And if you use your phones GPRS, GSM whatever, you will have a constant IPv4 (and later IPv6, I hope) address, as long as you have connection to your phone companies GPRS, GSM or whatever. You can even switch between them. It works for me, any way. And it is NOT much data used for XMPP, as I have it on my phone and had checked my data trafic use.
Monday, February 18, 2008
@Adam: The solution isn't just teredo, as teredo is just a way to get IPv6 connectivity on IPv4 networks. The key is mobile IPv6, which allows you to keep the same IP address across networks.
@Andrisi: I'm not sure what "price" you are referring to. With stream compression, XMPP streams become quite compact.
Wednesday, February 27, 2008
I can just agree that choosing a binary protocol is ill-advised.
Compression can be applied to data independent of the protocol chosen and text data are good at getting compressed.
As time goes by mobile devices are getting more power and what seemed like a lot of data to transfer over mobile networks 3 years ago seems a very small amount today. Either that or we would all be using WAP and nobody would be even dreaming about looking up a Google map (what, downloading large images to a mobile phone?!) on an iPhone.
P.S. Would a binary protocol solve the 3rd key issue that you mentioned - power consumption?
Sunday, March 2, 2008
Compression saves bandwidth but adds processing cycles, which is unfortunate in a near real-time domain—I don't want any extra lag there. A binary encoding (pick a ‘standard’, preferably the same as others) is observed to reduce the parsing effort to about 1/10th compared to verbose encoding. This definitely does good for RTT, as opposed to time-consuming decompression PLUS verbose decoding.
So, I'm partly glad that Google went binary, and I hope it either is or will become a standard encoding.
This was also discussed in http://blog.dave.cridland.net/?p=45
(A binary encoding may potentially hurt extensibility, but I'm not sure about that argument—a sane encoding shouldn't weaken the data model.)
Tuesday, March 4, 2008
Use of XMPP over HTTP works well across roaming and address changes. HTTP does need to poll tough - however the polling frequency can be modulated based on known states of the client to further optimize bandwidth and CPU.
Finally, use techniques specially designed for Mobile devices such as Push gateways to signal the device when data is available for it to further reduce traffic and latency associated with polling.
+1 for Binary encoding. Do your development with the "verbose mode" and throw the binary encoding switch at run time.
There are couple of clients in the market that use specially designed proxies made for XMPP for the mobile environment that shift some of the heavy lifting to the server side.
Wednesday, April 2, 2008
From having deployed applications on several major carriers, I can safely say the situation is actually more complicated than this.
Monday, May 5, 2008
Other important aspects to consider are:
Tuesday, July 22, 2008
@opjabber: IPv6 is a solution today. With technologies like 6to4 and Teredo, you can get IPv6 connectivity with zero cooperation of your network provider—assuming they don't block it outright (why would they?).
@Ernest, @Janne, and others: I have nothing against a binary encoding in principal as long as it is a one-to-one transform to and from the verbose XML, and is still able to handle new namespaces. If a binary encoding is flexible enough to handle that, then I don't have a problem with it—as long as the standard is open. For example, one such binary encoding happens to be a zlib-compressed XML stream. My biggest problem with Google's action is that they are not just changing the transport, they are changing the API. They could have an XMPP compliant API and just have a bizarre transport, but this isn't the direction they are apparently going.
Sunday, August 24, 2008
I personally believe that XMPP would be best suited. Mostly because several of the existing instant messaging service providers are exploring XMPP. It is said that Yahoo! is already moving | toward XMPP.
When the world is fnaly but slowly moving toward one standard, it is stupid that Google should go against that.
Sunday, August 16, 2009
One thing to notice is that the TCP connection problem is a "design bug" in Android. Other mobile OSes are quite capable of keeping multiple different network paths online. E.g. I've been usually bombus on my Nokia via GPRS while surfing/emailing via WLAN, and that was fine.
Guess if your car cannot go faster than 30mph, declaring a 30mph limit on all roads sounds like a good idea.