Debian proposals in GSoC 2014

The GSoC student application period is over, and the last two days were pretty interesting.

For a few years now, Olly Betts has provided us with a spreadsheet to graph the number of applicants to an organization over time.

Here’s the graph for Debian this year:

Debian GSoC proposals, 2014 edition

(Historical graphs: 2013, 2012. Spreadsheet available from Olly’s blog)

On Wednesday, I was thinking “hmm, 30 applicants, this is a slow year…”. Well, the number of proposals more than doubled in the last two days, to conclude on a whooping 68 applications! The last one was submitted just three seconds before the deadline…

If you want to take a look at the proposals, head over to the Debian wiki.

Time to get on reviewing! The final student acceptances will be published in just less than a month, on April 21st.

Hello from DebCamp

DebConf flag (minus the wind)

Small update as someone was complaining about the lack of pictures from DebCamp on planet.

Not to worry, everything is going fine, and some of the most important stuff is ready…


You can see a few pictures from the gallery. The view from the venue is quite outstanding (it was better when there was some sun on Tuesday, but my camera battery was out…).

On my TODO-list:

  • A lot of mentors.d.n related stuff : prototyping things to make our codebase more bearable by actually using other people’s work, e.g. debianqueued, firehose, …, and building upon Lucas’s idea from his new contributor survey (see the quite vague idea at the bottom)
  • FedMsg stuff (will probably pick up when Simon arrives)
  • Some 3D-printing packaging stuff (actually getting software in NEW, I got started on Printrun, and maybe Slic3r)

See you there!

Bootstrapping fedmsg for Debian

As you might (or might not) know, this summer, I have taken on mentoring of a GSoC project by Simon Chopin (a.k.a. laarmen) which goal is to bring fedmsg, the Fedora Infrastructure message bus, to Debian. Most of the work I’ll be talking about here is Simon’s work, please send all the praise towards him (I can take the blame, though).

What is this about?

As the project proposal states, the idea is to provide Debian with a unified, real-time, and open mechanism of communication between its services. This communication bus would allow anyone, anywhere, to start consuming messages and reacting to events happening in Debian’s infrastructure:

  • trigger a test build on a git push to a source repository
  • trigger automated testing (piuparts, lintian) as soon as an upload hits the archive
  • get a desktop notification when a package you care about gets changed

When we told upstream about our plan of adapting fedmsg to work on Debian, they were thrilled. And they have been very supportive of the project.

How is the project going?

Are you excited? I know I’m excited.

yep, he's excited too

Well, the general idea was easy enough, but the task at hand is a challenge. First of all, fedmsg has a lot of (smallish) dependencies, most of them new to Debian.

Thanks to Simon’s work during the bonding period, and thanks to paultag’s careful reviews, the first batch of packages (the first dependency level, comprising kitchen, bunch, m2ext, grapefruit, txws, txzmq and stomper) is currently sitting in the NEW queue. The four remaining packages (fabulous, moksha.common, moksha.hub and fedmsg proper) are mostly ready, waiting in the Debian Python Module Team SVN repository for a review and sponsorship.

While we’re waiting for the packages to trickle into Debian, Simon is not twiddling his thumbs. Work has taken place on a few fronts:

  • Getting fedmsg integrated to (actually sending the first messages from “Debian”‘s infrastructure)
  • Writing a (stop-gap) wrapper to convert some of our mail “broadcasts” (debian-devel-changes, debian-bugs-dist) into fedmsg messages, until a more proper integration can be done
  • Working on reliability of the message transport, following some concerns raised by DSA.


Package backports was chosen because I’m an admin and could do the integration quickly. That involved backporting the eleven aforementioned packages, plus zeromq3 and python-zmq (that only have TCP_KEEPALIVE on recent versions), to wheezy, as that’s what the mentors.d.n host is running. (Also, python-zmq needs a new-ish cython to build so I had to backport that too). Thankfully, those were no-changes backports, that were easily scripted, using a pbuilder hook to allow the packages to depend on previously built packages.

I have made a wheezy package repository available here. It’s signed with my GnuPG key, ID 0xB8E5087766475AAF, which should be fairly well connected.

Code changes

After Simon’s initial setup of debexpo (which is not an easy task), the code changes have been fairly simple (yes, this is just a proof of concept). You can see them on top of the live branch on debexpo’s sources. I finally had the time to make them live earlier this week, and has been sending messages on Debian’s fedmsg bus ever since.


mentors.d.n sends its messages on five endpoints, tcp:// through tcp:// That is one endpoint per WSGI worker, plus one for the importer process(es). You can tap in directly, by following the instructions below.


Debmessenger is the stop-gap email-to-fedmsg bridge that Simon is developing. The goal is to create some activity on the bus without disrupting or modifying any infrastructure service. It’s written in hy, and it leverages the existing Debian-related python modules to do its work, using inotify to react when a mail gets dropped in a Maildir.

Right now, it’s supposed to understand changes mails (received from debian-devel-changes) and bugs mail (from debian-bugs-dist).

I’ll work on deploying an instance of debmessenger this weekend, to create some more traffic on the bus.

Reliability of the bus

I suggested using fedmsg as this was something that already existed, and that solved a problem identical to the one we wanted to tackle (open interconnection of a distribution’s infrastructure services). Reusing a piece of infrastructure that already works in another distro means that we can share tools, share ideas, and come up with solutions that we might not have considered when working alone. The drawback is that we have to either adapt to the tool’s idiosyncrasies, or to adapt the tool to our way of working.

One of the main points raised by DSA when the idea of using fedmsg was brought up, was that of reliability. Debian’s infrastructure is spread in datacenters (and basements :D) all over the world, and thus faces different challenges than Fedora’s infrastructure, which is more tightly integrated. Therefore, we have to ensure that a critical consumer (say, a buildd) doesn’t miss any message it would need for its operation (say, that a package got accepted).

There has been work upstream, to ensure that fedmsg doesn’t lose messages, but we need to take extra steps to make sure that a given consumer can replay the messages it has missed, should the need arise. Simon has started a discussion on the upstream mailing list, and is working on a prototype replay mechanism. Obviously, we need to test scenarios of endpoints dropping off the grid, hence the work on getting some activity on the bus.

How can I take a look?

a.k.a. “Another one rides the bus”

A parisian bus built in 1932

(Picture © Yves-Laurent Allaert, CC-By-SA v2.5 / GFDL v1.2 license)

So, the bus is pretty quiet right now, as only two kinds of events are triggering messages: a new upload to, and a new comment on a package there. Don’t expect a lot of traffic. However, generating some traffic is easy enough: just login to mentors.d.n, pick a package of mine (not much choice there), or a real package you want to review, and leave a comment. poof, a message appears.

For the lazy

Join #debian-fedmsg on OFTC, and look for messages from the debmsg bot.

Current example output:

01:30:25 <debmsg> debexpo.voms-api-java.upload (unsigned) --
02:03:16 <debmsg> debexpo.ocamlbricks.comment (unsigned) --

(definitely needs some work, but it’s a start)

Listening in by yourself

You need to setup fedmsg. I have a repository of wheezy packages and one of sid packages, signed with my GnuPG key, ID 0xB8E5087766475AAF. You can add them to a file in /etc/apt/sources.list.d like this:

deb<sid|wheezy>/ ./

Then, import my GnuPG key into apt (apt-key add), update your sources (apt-get update), and install fedmsg (apt-get install python-fedmsg). The versions are << to anything real, so you should get the real thing as soon as it hits the archive.

Finally, in /etc/fedmsg.d/, you can comment-out the Fedora entries, and add a Debian entry like this:

    "debian": [
    ], runs a fedmsg gateway connected to the mentors.d.n endpoints, and thus forwards all the mentors messages. It’ll be connected to debmessenger as soon as it’s running too.

To actually see mesages, disable validate_signatures in /etc/fedmsg.d/, setting it to False. The Debian messages aren’t signed yet (it’s on the roadmap), and we don’t ship the Fedora certificates so we can’t authenticate their messages either.

Finally, you can run fedmsg-tail --really-pretty in a terminal. As soon as there’s some activity, you should get that kind of output (color omitted):

  "i": 1, 
  "msg": {
    "version": "2.0.9-1.1", 
    "uploader": "Emmanuel Bourg <>"
  "topic": "", 
  "username": "expo", 
  "timestamp": 1373758221.491809

Enjoy real-time updates from your favorite piece of infrastructure!

What’s next?

While Simon continues working on reliability, and gets started on message signing according to his schedule, I’ll take a look at deploying the debmessenger bridge, and making the pretty-printer outputs useful for our topics. There will likely be some changes to the messages sent by debexpo, as we got some feedback from the upstream developers about making them work in the fedmsg “tool ecosystem” (datanommer and datagrepper come to mind).

You can tune in to Simon’s weekly reports on the soc-coordination list, and look at the discussions with upstream on the fedora messaging-sig list. You can also catch us on IRC, #debian-soc on OFTC. We’re also hanging out on the upstream channel, #fedora-apps on freenode.

Hello world

Or rather, hello Planet!

Here’s a somewhat traditional introductory post.

I’m Nicolas Dandrimont, I’m French, I’m sysadmin in a grande école, where I’m mostly in charge of the GNU/Linux workstations and servers.

In Debian, I’m a DM, currently in the NM queue, so I might become a DD soon-ish. I am (rather inactively) co-maintaining a few packages. In my Debian “career”, I have been involved in OCaml packaging and Python packaging, although lately most of my time has been spent on Google Summer of Code (mentor for two projects in 2012, org admin for Debian in 2013), and on

In other free-software related projects, I own a RepRap 3D printer, and I grew some interest in the related software, e.g. Slic3r and printrun. There have been a lot of action in Fedora about packaging 3D-printing-related software, and it’d be great to get a team together to work on that in Debian during the jessie release cycle. Consider this a call for interested parties 🙂

Unrelatedly, paultag has tricked me into working on hy, which is way too much fun. Blame him if you feel that I have been inactive lately, this has been eating way too much of my free time 😉

Hopefully I’ll be able to make regular updates on the work I do in Debian and free software, so stay tuned!