ervin's piece of web

With Fremantle and the N900 almost out the door, it is time that we start a more coordinated effort within the KDE community to support Maemo as an official target system. In the past we had some scattered and not coordinated efforts, results got linked on techbase.

I attended the Maemo Summit 2009 in Amsterdam. Of course a few other KDE people were there. We sat down together, talking about how we could get the KDE platform working well on those devices. And we decided that if we want to see KDE succeed on such devices, we’d have to get more serious and coordinated about it.

That’s why shortly after coming back from the meeting, I asked for the creation of the kde-maemo mailing list, where we’ll be able to address Maemo as one of the official KDE target systems (like FreeBSD, Solaris, Windows and Mac OS X are). It is time for us to make Maemo a first class citizen in our community.

Of course it is not wishful thinking we’re talking about here. The efforts already started, for instance Alexis has been working on a Plasma based shell for the N900. It is still rough, and a first experiment, but it at least shows that our platform is viable on this system. Parallel to that, there’s also efforts on how to make the deployment of a Maemo toolchain more convenient, or how to modularize our platform for such embedded systems.

It is just the beginning of the journey, let’s see where it leads us!

I previously blogged about the new IMAP resource in Akonadi. We were aiminag at getting this resource from the start, but here is the not so secret story: this effort gave birth to another component, namely libkimap.

The journey started because we really wanted for KDE to have a strong and efficient IMAP support. Since we’re not really into reinventing the wheel if we can avoid that the journey started by looking at existing IMAP solutions we could reuse. The natural first contender of course was to reuse our old IMAP ioslave which we currently use in KMail and build the resource on it. But really it is showing its age now, and the interface such ioslaves expose are too limited for our needs (for those in the known of KIO internals: it would require an extensive use of the special() method, or command encoding in urls…). So we have been evaluating a few other contenders coming from various mail clients, but unfortunately most of them are either exposing synchronous API (not convenient for something event driven like an Akonadi resource), or very tied to a MIME implementation (which was a no-no as we needed to use the resource with kdepim’s MIME implementation).

Indeed this journey didn’t start quite well… We had to produce a new IMAP implementation. So a new quest started, one for an adequate API design. At that point we knew we wanted something: fast, memory efficient, asynchronous and easy to extend (as the IMAP RFC has plenty of extensions).

First, let’s examinate the “fast and memory efficient” constraints. Around the same time I got started on libkimap, Andras Mantia (fellow KDABian and hacker extraordinaire) was working on making the Akonadi server protocol handling faster and more memory efficient… And as a bonus, the Akonadi protocol for the lowest level parts, have quite some similarities with IMAP as it was modelled after it. So far so good, I could reuse Andras work on the new IMAP stream parser. It has been a two way collaboration as I also found bugs in there which got fixed. This parser is now the core of libkimap, although it is hidden from the public API, it is for a great part responsible of the good performances of the library.

Then, let’s solve the “asynchronous and easy to extend” constraints, those had a direct impact on the API. For that we rely on the good old “job based” API. You just need to create a Session object which holds a connection to an IMAP server and queue Jobs on it. Jobs are then executed sequentially. We have jobs for a lot of things: mailbox listing, fetching messages or headers, retrieving annotations, quota information, ACLs, etc. And that’s where the job based API gets interesting, we need to extend it? Just add a job, binary compatibility will be kept and so on, it makes it much easier to manage it over time.

Oh! And of course, since we still care about performances, each job is equipped to be able to pipeline several IMAP commands to the server, which dramatically reduces latency. So you guessed it, jobs don’t map 1:1 to IMAP commands, this way we can provide some convenience to developers because they get pre-shipped micro behavior (for instance, listing mailboxes and taking care about the namespace extension results always in the same list of commands, so we wired it in jobs which pipeline commands).

So we indeed solved our initial problem, we made an asynchronous IMAP library which is fast, efficient and easy to extend. It also tries to be clever when that actually makes it more convenient for developers (see the pipelining example above), but not too much, allowing to have a fine control on the higher level logic and strategies you’d need to implement in your application. And thanks to this strong basement, we could tackle the task of building the Akonadi IMAP resource on top of it.

As a post scriptum: is our library tied to a MIME implementation? Well, like others we’re tied to one: libkmime, available in kdepimlibs as well. You can’t really get around that in the end. Indeed, for fetch operations if you try to be independent of any MIME implementation, you end up implementing your own subset of MIME. That said we tried hard, and managed to keep that coupling as minimal as possible, and in fact only the FetchJob really depends on libkmime, everything else is independent of it. So, it wouldn’t be a big cost to implement your own FetchJob variant using another MIME implementation.

And finally, as a post post scriptum, if people out there wants to grab it and play with it, it is in our kdepimlibs module on trunk. You can browse it online here: http://websvn.kde.org/trunk/KDE/kdepimlibs/kimap/.