Still, one of the challenges is also to widen the scope of our KDE Platform. For that, a draft plan was made during Tokamak4, and since then we’ve been progressing carefully on the matter. We tried to get as much feedback as possible on the plan, not rushing things to make sure we weren’t stepping on anyone toes.

Today, I’m happy to announce that the very first corner stone of this plan got delivered. We added support for "profiles" in our platform. The CMake scripts for it got committed this morning, along with some changes to libplasma which effectively becomes our first library supporting those profiles.

By selecting a profile at build time, you get a default setup for our libraries which will enable or disable some extra features and dependencies. For instance, if you choose the "Mobile" profile the feature set coming from kdelibs will be reduced but on the other hand there will be much less internal dependencies in kdelibs, this way an application will only need a reduced subset to be able to run.

This more modular kdelibs depending on the profile chosen is of course only a first ongoing projects, but we have other topics to tackle like the runtime dependencies (namely klauncher and kded) of our platform. On this area we still lack reliable data as it is much harder to track. Still reducing dependencies during build time will be a big leap forward. And I’m truely excited because we’re slowly (but steadily!) getting to a slimer KDE Platform.

So we used the opportunity to have a Solid meeting to summarize the current situation of our infrastructure, and to make plans for 2010. That includes quite a few of clean ups on our stack, but also more ambitious and cool stuff like reporting devices reachable via the network. If you’re interested in details, I sent a mail summarizing the Solid meeting at Tokamak4, and you should probably subscribe to kde-hardware-devel if you’re not there yet.

Today though, I just wanted to let everybody know that I’ve been working on a virtual machine to help KDE developers easily get a Maemo SDK. I added some documentation so now we have a Qt/Maemo SDK VM page on techbase (download URL and installation procedure are there).

It should be relatively easy for anyone to have it working: download, boot, login, run a script, done you can know use Qt 4.6 in a fully setup Maemo SDK. Hopefully that will lower the bar to contributing for quite some people.

That’s it for now, don’t hesitate to drop me a mail if you have issues with it.

I attended the Maemo Summit 2009 in Amsterdam. Of course a few other KDE people were there. We sat down together, talking about how we could get the KDE platform working well on those devices. And we decided that if we want to see KDE succeed on such devices, we’d have to get more serious and coordinated about it.

That’s why shortly after coming back from the meeting, I asked for the creation of the kde-maemo mailing list, where we’ll be able to address Maemo as one of the official KDE target systems (like FreeBSD, Solaris, Windows and Mac OS X are). It is time for us to make Maemo a first class citizen in our community.

Of course it is not wishful thinking we’re talking about here. The efforts already started, for instance Alexis has been working on a Plasma based shell for the N900. It is still rough, and a first experiment, but it at least shows that our platform is viable on this system. Parallel to that, there’s also efforts on how to make the deployment of a Maemo toolchain more convenient, or how to modularize our platform for such embedded systems.

It is just the beginning of the journey, let’s see where it leads us!

The journey started because we really wanted for KDE to have a strong and efficient IMAP support. Since we’re not really into reinventing the wheel if we can avoid that the journey started by looking at existing IMAP solutions we could reuse. The natural first contender of course was to reuse our old IMAP ioslave which we currently use in KMail and build the resource on it. But really it is showing its age now, and the interface such ioslaves expose are too limited for our needs (for those in the known of KIO internals: it would require an extensive use of the special() method, or command encoding in urls…). So we have been evaluating a few other contenders coming from various mail clients, but unfortunately most of them are either exposing synchronous API (not convenient for something event driven like an Akonadi resource), or very tied to a MIME implementation (which was a no-no as we needed to use the resource with kdepim’s MIME implementation).

Indeed this journey didn’t start quite well… We had to produce a new IMAP implementation. So a new quest started, one for an adequate API design. At that point we knew we wanted something: fast, memory efficient, asynchronous and easy to extend (as the IMAP RFC has plenty of extensions).

First, let’s examinate the “fast and memory efficient” constraints. Around the same time I got started on libkimap, Andras Mantia (fellow KDABian and hacker extraordinaire) was working on making the Akonadi server protocol handling faster and more memory efficient… And as a bonus, the Akonadi protocol for the lowest level parts, have quite some similarities with IMAP as it was modelled after it. So far so good, I could reuse Andras work on the new IMAP stream parser. It has been a two way collaboration as I also found bugs in there which got fixed. This parser is now the core of libkimap, although it is hidden from the public API, it is for a great part responsible of the good performances of the library.

Then, let’s solve the “asynchronous and easy to extend” constraints, those had a direct impact on the API. For that we rely on the good old “job based” API. You just need to create a Session object which holds a connection to an IMAP server and queue Jobs on it. Jobs are then executed sequentially. We have jobs for a lot of things: mailbox listing, fetching messages or headers, retrieving annotations, quota information, ACLs, etc. And that’s where the job based API gets interesting, we need to extend it? Just add a job, binary compatibility will be kept and so on, it makes it much easier to manage it over time.

Oh! And of course, since we still care about performances, each job is equipped to be able to pipeline several IMAP commands to the server, which dramatically reduces latency. So you guessed it, jobs don’t map 1:1 to IMAP commands, this way we can provide some convenience to developers because they get pre-shipped micro behavior (for instance, listing mailboxes and taking care about the namespace extension results always in the same list of commands, so we wired it in jobs which pipeline commands).

So we indeed solved our initial problem, we made an asynchronous IMAP library which is fast, efficient and easy to extend. It also tries to be clever when that actually makes it more convenient for developers (see the pipelining example above), but not too much, allowing to have a fine control on the higher level logic and strategies you’d need to implement in your application. And thanks to this strong basement, we could tackle the task of building the Akonadi IMAP resource on top of it.

As a post scriptum: is our library tied to a MIME implementation? Well, like others we’re tied to one: libkmime, available in kdepimlibs as well. You can’t really get around that in the end. Indeed, for fetch operations if you try to be independent of any MIME implementation, you end up implementing your own subset of MIME. That said we tried hard, and managed to keep that coupling as minimal as possible, and in fact only the FetchJob really depends on libkmime, everything else is independent of it. So, it wouldn’t be a big cost to implement your own FetchJob variant using another MIME implementation.

And finally, as a post post scriptum, if people out there wants to grab it and play with it, it is in our kdepimlibs module on trunk. You can browse it online here: http://websvn.kde.org/trunk/KDE/kdepimlibs/kimap/.

Well, recently I’ve also been working on showing up collection statistics we can get from Akonadi, and also I ported KMail message list view to Akonadi (also making it a separate library). In the meantime, the fearless Andras has been porting KMail mail reader view to Akonadi (also making it a separate library).

In fact, it is the current thread in the huge task which is the porting of KMail to Akonadi. We’re ripping KMail apart, each important set of features are factored out in a library and ported to Akonadi. In the end we’ll have a KMail completely based on Akonadi. But also, it will be much more modular, reusable for other mail clients, but also news readers, etc. Which means that the new architecture will be better suited at supporting a wide range of devices and a good base for future works.

Anyway, both Andras and me had a small test application for each of our new frameworks… So I took some time to merge the features of both into a single application: the Akonadi Mail Reader. This new baby is mainly a prototype to toy with ideas and try out the components we’re making out of the monolithic KMail. Still, it weights just under 400 lines of C++ code, and you can completely browse all the mailboxes you configured in Akonadi thanks to it. Of course, here is the obligatory screenshot:

It looks so much like a miniature version of KMail that it is almost scary. But don’t fool yourself, there’s a lot to do to have a full fledged KMail which will be only based on Akonadi. We’re not there yet, still it is nice to see the whole thing taking shape, in particular to reach the point where you can actually read your mail over IMAP using this small prototype and feel almost at home with it thanks to this familiar touch.

From my point of view, the KDE culture of working a lot with components really pays off. KMail was first created at a time where this KDE culture didn’t reach it’s full potential yet, hence why we need to refactor it now. But, following this culture it is really nice to see that we’ll end up with small packages of mail client functionalities, and, that thanks to them and to Akonadi, it will be easy to integrate them in any application. We made a relatively complete mail reader in under 400 lines of code, so simply displaying mail content in your application or a message list becomes a trivial task.

Buuuut… I figured out that I could post something about what I’m doing in KDAB (OK! OK! Sebas has been kicking me for days to make it happen). And if that proves to be enjoyable writing for me, and enjoyable reading for you people I may do that again from time to time.

So what am I doing in KDAB these days? Well, for a while now I’ve been helping the pimsters with the refactoring of our PIM suite toward making it purely Akonadi-based. There’s quite a lot of work going on in this particular space, and I’ve been focusing mainly on the IMAP support.

It has been an interesting journey so far which started with evaluating the IMAP stacks we could reuse. First on the ranks were obviously our old ioslave which served us well, and some IMAP code Tom Albers developed for Mailody. It turned out that we had to abandon the ioslave. It served us well for years, but didn’t work for some of the plans we had (most notably for adding the IDLE support I’ll talk about later). While working on IMAP I had a couple of ideas on how to still use the old ioslave, but they were really nasty and hackish… I think it was a clear sign that the ioslave reached its full potential and keeping it alive was a dead end for the future.

So after this investigation phase it has been decided to write a small library for IMAP. We already had some codecs convenience in the KIMAP library from kdepimlibs so we stepped up to extend it. And for the API we quickly settled on a job based one because we had strong needs for asynchronous operations. We clearly don’t want to be blocking waiting for some IMAP command to be processed. So our dear old KJob friend has been used for that, and it turned out to be a very good choice.

We’ve been relatively quick at having enough useful jobs implemented in the library once I wrote the core bits of the API. I got quite some help from Andras Mantia for that. By the way, I think that just like our almighty David Faure, Andras is having a few clones himself, once he gets on a task he going so fast you’d better not be on his way… something suspect is at play there, I’m sure.

In any case, Andras (and his hundreds of clones) help was welcome, as it helped me move higher in the stack and work on the Akonadi resource itself. I’ll just slow down a minute and explain for those not in the known what an Akonadi resource is (for the ones in the known, you can skip to the next paragraph). So Akonadi is our grand unified layer to access data in your desktop, applications talk to it and then they have to understand only the akonadi concepts (in short: items, and collections of items). Of course you need to get your items and collections from somewhere, and that’s the role of Akonadi resources. They just get data from “somewhere” and put it in the Akonadi cache where the applications can make use of them. Obviously for a future Akonadi based KDEPIM suite we needed an IMAP resource to access your mail on IMAP servers. And if you followed closely, that means that soon applications won’t even need to know where the data comes from, you write just an Akonadi client and it can go over: POP, IMAP, whatever! It all depends on the resources your server loaded. Neat huh?

Work on the IMAP resource started only a few days before the Akonadi Sprint in Berlin which I was attending. And the job based API quickly proved that it was the right choice. I’ve been pretty quick at building the resource on top of it (and Andras was producing new jobs for my own consumption at a furious pace). At this meeting we had kind of an informal contest about who would come up first with an Akonadi resource which can list a mailbox (since at the time POP, IMAP, and maildir support were all developed and started around the same time)… it turned out that I heard about this informal contest when I was putting the finishing touch to the mailbox listing. Sooo… DONE!

I didn’t stop there of course as more features were needed to have a complete resource. And I’ve been working on that almost exclusively during the sprint. Discussing with Volker (aka the Akonadi overlord, the alpha and omega of its design) we came up with nice design ideas for the groupware support (in particular Kolab)… but that’s probably a story for another time…

Some time after the sprint, I flew over to other tasks in PIM land. And left the Akonadi IMAP resource untouched for a while. I came back to it only last week to implement in trunk one long requested feature: the IDLE extension. — And obviously that’s where the name of this blog post come from, just like my favorites novels you get the title only in the last chapter or paragraph. — Thanks to this extension, if it is supported by the server, you get notified from changes by the server itself. For instance, it is not needed anymore to poll every N minutes to notice that a new mail appeared (I think this feature is sometimes called “push mail”). For the user that means, that the IMAP server can now notify the client of new emails, emails “arrive” faster at the user, and you can save a lot of unnecessary mail checking. Really handy!

I committed IDLE support in trunk this week, so it will get released with our 4.4 release. In the meantime you can start playing with the Akonadi resource when our 4.3 release will be out, as we already ship it there (although it’s not used by default, see it more like a pre-version of what’s coming, for instance this one doesn’t support IDLE).

That’s it for today, I hope you enjoyed those pimsters news from the trenches of the IMAP battle. See you later!

Of course, Air being almost out of the door we deserve a new updated LaTeX beamer template. Since I wrote the Oxygen template, I decided to produce a new one based on the great work from Nuno. As usual I’m providing a tarball with the template, and you can take a look at an example presentation

And tomorrow morning, very early, I’ll meet some more gearheads from Toulouse, and we’ll take the plane for Gran Canaria. Looking forward to it! See you all in Las Palmas.

OK, that was really short, I’ll try to blog more during the conference. I swear!

It’s also nice to have everybody on the deck ready for action. And action we had, lots of different topics got covered: from the framework itself, to the appearance of the shell, it’s interaction with the other major part of the desktop (namely kwin), the integration of the features from Qt kinetic, etc.

Personally I tried to focus as much as possible on our service framework, so for that I’m writing a library which will help delegating all the service work to Jolie. It’s not there yet, but we’re definitely seeing progresses. I can currently write a program which loads Jolie’s metaservice, fires up a service description and talks to it. It “just” needs to be wrapped into a nice API now. Jolie is really a pleasant piece of software to work with.

Also on the first day, I talked about my new pet project: Zanshin. A new todo/action management software, I’m using it daily for a couple of weeks already without major issues. Of course it’s still a bit rough, and I have great plans for it in order to help people to integrate it in there workflow. I want something simple and flexible. I’ll probably blog more about that in the coming weeks.

I’ll end this post with a quote I used in my talk about Zanshin:

If your mind is empty, it is always ready for anything; it is open to everything. — Shunryu Suzuki

I expect a 10 page essay about this quote on my desk next week.

So JOLIE is an interpreter for a high level language to interact with services. Services as in service oriented architecture, and yes that includes web services but also much more. And, as you might have noticed, we discussed with the guys working on JOLIE during the Tokamak Mark I and as Danny hinted, I wrote a Qt implementation of SODEP (the protocol used to interact with running instances of JOLIE).

Now you might wonder, what’s the point of all that? Well, it’ll enable KDE, to be a first class citizen in the service oriented world (and seeing the amount of web services out there or the growth of D-Bus usage, that’s an important goal). By “first class citizen”, here I mean making it trivial to interact with those services, today we can interact with them but that still require quite some hand made code, something JOLIE and the facilities we’re planning to add in Plasma will hopefully make obsolete.

That’s mostly post-4.1 material… Except that Fabrizio Montesi one of the humongous JOLIE developers couldn’t wait and wrote some proof of concept code. So I’ll post a few screenshots he made because they’re pretty cool in my opinion. So what he made is a small service named Echoes and driving an amarok instance, and GWT based application providing a gui client to this service. Then users can fight over your playlist.

We tested it, it’s pretty nice all instances are synchronized. Also, if something is changed directly in Amarok you notice it in Echoes GUI. Now, it becomes really cool because you can embed such service clients in your cellphone:

Or even as a Plasma Widget:

Of course, it’s still all a bit experimental and ad hoc at the moment. Our goal post-4.1 is to make this kind of service client GUIs trivial to write and better integrated in KDE. Services are widespread now, let’s make use of them!