On behalf of the rest of Planet Ubuntu, I'd like to apologise for Stephan Hermann. Most everyone else there isn't half as obnoxious, so please ignore that entry.

One of the annoying features of reading several planets, is that you end up reading entries from people you otherwise respect, who seem to have various complexes about nonsensical issues. We all saw the month in which half of Debian decided it was cool to rip on Ubuntu, and now apparently it's the new cool thing to do if you work on Fedora.

According to a particularly obnoxious entry, the quality or otherwise of Ubuntu's kernel[0] does 'damage to the free software community'. Now, they've focussed on binary drivers. Which is fair enough: I am personally quite uncomfortable with binary drivers, and extremely unimpressed with Ubuntu's recent move towards binary drivers by default. While I respect the fact that they still encourage users to install free drivers, providing them by default is not a winning move, in my opinion. But I'm not going to discuss this further right now, because if I do, the terrorists win.

The reason the terrorists win, is because some people like to furiously handwave and make strawmen. Apparently now, everyone who's ever touched Ubuntu is rabidly for binary drivers, but the problem here is the handwave from criticism of Fedora's alleged community into yet another ramble about the kernel tree and binary drivers.

I certainly hope the Fedora community is more mature than this, both in general, and in their ability to have a constructive debate about the level of its community involvement. Right now, it is not even disputed that there is effectively none. At first glance, they may appear similar, but if you look further, the similarities vanish. Anyone can come in and get involved in the decision making in Ubuntu, but more important than that, the process is at least completely transparent, so you can see what's going on, on public lists, at every step. Fedora does not have this: decisions magically appear On High from RHEL types. This is not necessarily a bad thing, and does not necessarily result in a lower-quality distribution, but if everyone could just accept this as being true and move on, rather than bitterly baiting a successful community distribution about it, I think everyone would be a lot happier.

The biggest difference is -- yes, basically all the Ubuntu core developers are employed by Canonical. These people were handpicked from the Debian community originally, and now a great deal of them start as Ubuntu community members, and are employed by Canonical because having them work full-time on this stuff would be awesome. The process is still entirely transparent: being employed is a mark of recognition (sorry), and a request to continue working on this even more, rather than a pre-requisite to being able to do anything. Witness Fedora Core vs. Fedora Extras, for example: there are quite a few packages that non-Red Hat employees are prevented from meaningfully contributing to.

So, the summary in a nutshell: binary drivers bad (and Ubuntu moves towards promoting same even worse), Fedora 'community' not at all, handwaves bad, refusal to get involved with a meaningful debate about community governance and structure without engaging in petty mudslinging and pointless handwaves inexcusable.

(Disclaimer: I am not involved with any distributions, nor do I have any professional affiliation with them either. I'm just an irritated observer.)

[0]: I don't claim to have any insider knowledge here; the only point at which its quality or otherwise has become an issue for me lately, is a random hang in gettimeofday() in a corner case recently, which I've been too lazy to isolate and report as a proper bug; I worked around it in the X server anyway.

Wouter asserts that Ubuntu has more than 30 full-time employees. I know it wasn't his number (Scott came up with it and should've been a bit more clear), but it's badly wrong, anyway. Canonical has more than 30 employees, but Ubuntu ... the full-time distribution team hardly breaks double figures. That's pretty comparable to the number of people working on the m68k port (albeit the latter are not funded full-time). But it's an interesting point nonetheless, and thanks to the assumption that Canonical does nothing other than Ubuntu, people seem to assume that Ubuntu was produced by a team of 30, 40, whatever; not even close. That team was barely ten.

(Note: Some of the graphs in here aren't entirely in step with the text -- we didn't really think about blogging it until very late in the game, so the graphs linked to are more a rough idea of where we were at the time, not exhaustive.)

There's nothing like a challenge.

Along with the rest of Canonical and a whole bunch of excellent community guys, Thom May and I have been at the Ubuntu conference at Mataro since last Sunday. While imbibing large amounts of Coke, Thom decided that boot was probably too slow, with an eye on the promised forty-second desktop. He put bootchart on his laptop, and there the madness started.

Originally, we believed our boot was around 20 seconds without readahead, and felt pretty good about ourselves. But once we moved bootchart into the right place (starting at the top of rc2 is not a useful metric), we had a more realistic view -- around 1min for a boot to the gdm login screen -- and set to work. One thing we found was that gdm slept. A lot. After I took to the gdm source with a very large axe, we no longer had a huge gulf in our boot process with time elapsing, but no disk or CPU usage.

We had already replaced large swathes of the massive shell horror that is known as hotplug with grepmap, so it was time to look at other things. cupsys in particular was a huge disk hog, so some fine-tuning from Scott James Remnant soon set that in order. But we had kicked the more obnoxious sleeps out of gdm, and we were still looking at well over fifteen seconds from X starting through to gdm actually prompting you for your login, which sucked. I stopped laughing at Thom when he told me the X server startup really was too slow. So I straced it.

Output of strace -e file (i.e. show all file accesses) when starting X:
5702 xorg-trace-file

In July, I got very frustrated with the old, crufty, and generally horrid MetroLink loader in XFree86/X.Org, which was also used in Debrix (being a fork of X.Org), and threw it away.
daniels@catsby:~/x/xorg/xc/programs/Xserver/hw/xfree86/loader% wc -l *.[ch] | tail -1
13540 total
daniels@catsby:~/x/debrix/debrix--devel/hw/xorg/loader% wc -l *.[ch] | tail -1
897 total

It seemed like some major surgery was needed on the loader, or at least to beat the more obnoxious parts out of it. For instance, stat()ing the Radeon driver forty-one times. I don't even have a Radeon in this machine; it's i855-based. Turns out that the loader was running a regular expression over every single file in /usr/X11R6/lib/modules, and then stat()ing them, for every module load. It is now no longer doing so:
daniels@catsby:~/public_html% wc -l xorg.trace-1040
1904 xorg.trace-1040

With a lazy 3798 file accesses gone, there was no longer a massive disk I/O hit as X started, but it was still not enormously quick. Further beating of gdm ensued, and we discovered that scaling a 1600x1200 pixmap down and then overlaying a transparent PNG really, really hurt. Enabling autologin let us fly through the process, but this was far too much of a security risk to consider, so we went back to measuring gdm proper.

While we were at it, in an inspired move, Scott James Remnant set to work on cupsd, which really was a horrific disk hog, so soon set that in order.

Hotplug was still doing a lot of work (but no disk I/O) for a very long time, so we decided to parallelise hotplug and readahead, so we could have pure CPU grunt work interleaved with smashing the disk. This seems to have worked very effectively, and has shaved quite some time off our accesses. Starting some parts of rc2 in parallel seems to have worked very well, also; gdm starts at 14 (before most services that you don't need for a desktop).

At present, the biggest bottleneck we have is hotplug; Scott is working on replacing the hotplug init script with moving through udevd, and replacing many of our init scripts with hotplug triggers, so we can more effectively parallelise most of our boot process. Thom is, at present, working on moving all the readahead files somewhere where they can be streamed quickly; at the moment, we are getting flayed by seek()s. Beyond this, gdm looks like it needs some serious work, but we believe we are at a very strong position, especially after beating more file accesses out of X (this is our current bootchart, at time of writing).

These tests were done (by both myself and Thom) on IBM ThinkPad X40s, with Pentium M 1.2GHz CPus and low-speed hard drives. One of the larger blockers is i855, which takes forever to initialise through VBE: 'profiling' the X server by throwing in time information with all the logs showed us nothing was usefully slow (in terms of low-hanging fruit), but video initialisation still takes a good four or so seconds. It'll be interesting to measure the results on standard desktop disks (not 15kRPM SCSI or such) and a chipset we have the full video BIOS information for. However, everything we have done thus far is totally applicable to other systems -- hotplug still runs in full, so you can dump the exact same stuff we're using on a totally different system (even PowerPC, if you like), and it will still work perfectly fine, with no modification.

Current statistics: booting through a full, typical startup to GDM login screen, including hotplug -- 42 seconds (warty: 90 or more); file accesses when starting the X server -- 1093 (warty: at least 5700); files read in by readahead: 1000 (total size: 58M).

(Update: X is now down to 538 opens.)

71 S Nov 09 Fabio M. Di Nit ( 0) Accepted xorg 6.8.1-0ubuntu1 (source)

While reading the Planet Debian discussions about whether Canonical is evil because it doesn't have all its employees working on Debian all the time -- my last gig was working on X servers, which is absolutely incidental to Debian. Since then, I've done a heck of a lot more Debian work (both incidental and direct) than I did at my last job. I don't think we've actually decresed anyone's Debian time.

Michael Banck wonders about the Ubuntu development model. Two things I can provide constructive answers to are:

• NMUs: no, we don't have maintainership as such. However, we do have unofficial areas -- for example, myself and Fabio take care of X, I do most of the stuff relating to ppp and also discover1 (including the data list), et al. Doing pppoe without a PPPoE setup has been interesting, but oh well.
• Debian: for most all of us, Debian was an after-work thing anyway (or, at least, a small part of work). Nothing's changed here; the only thing that's changed for me is that I haven't gotten around to setting up a chroot yet (only got DSL bumped up to unlimited recently), so I don't have a Debian machine to build on: I get all my uploads sponsored right now.

Bluetooth support in Ubuntu is also pretty flawless. Put this line in /etc/apt/sources.list:
deb http://people.ubuntu.com/~jdub/warty/ ./
and run:
sudo apt-get update
sudo apt-get install gnome-bluetooth bluez-utils gnome-phone-manager
and you have working Bluetooth support. Brilliant!

The IBM ThinkPad X40, as I've mentioned before, is 1.47kg (less if you get the 4-cell, rather than the 8-cell, battery) of pure love. I couldn't imagine owning another laptop. But, there's something immensely cool about it -- other than its size, its weight, its build quality, its features. It's that, with one boot option and one package, I have working support for ACPI suspend-to-RAM (and proper resuming), DRI, wireless LAN, Bluetooth (more on that in a soon-to-be-written entry), PCMCIA, VGA out, Ethernet, USB, sound, and just about everything else. I don't know if the internal modem works -- but I'll find out tomorrow -- and the SD slot doesn't work, because the SD consortium have only just released the relevant specs, apparently. Other that, we're talking about flawless out-of-the-box support. Other distributions will need to apply the latest patch from acpi.sf.net to their kernel to feel the full love, I suspect.

Two steps ... holy god.

... in which our hero performs an install without a CD-ROM drive, seeded almost entirely from /var/cache/apt/archives.

I last installed my desktop ages ago, and since then I've dist-upgraded almost every package that's installed. Installing my X40 has been fun: our bandwidth is incredibly limited by transfer, so instead of grabbing an ISO (not like it has a CD-ROM drive anyway), I decided to attempt a netboot install from /var/cache/apt/archives.

First, I set up apt-move to move /var/cache/apt/archives to a mirror directory, and to mirror something replicating the standard Ubuntu archive. Secondly, I set up dhcpd3 to netboot the contents of pxeboot.tar.gz (found on http://archive.ubuntu.com/ubuntu/dists/warty/main/daily-installer-i386), to the trusty laptop.

Now, here comes the fun. Once this was done, I needed to wget the main/debian-installer/binary-i386 Packages{,.gz,.bz2} file, and manually drop the sums/sizes into the Release file, which needed to be altered to specify the right component/suite. Once this file was there, a small for script got every single udeb specified in the d-i Packages file, and dropped it into the pool structure. Once all these things were satisfied, I was on my way to a netboot install, with minimal bandwidth consumed; thanks to Colin Watson for patiently putting up with my stupid questions.