udisks and gnome-disk-utility - past, present and future

The past year or so I’ve been working a lot on gnome-disk-utility and DeviceKit-disks^Wudisks and I haven’t really blogged anything about it. Time to fix that!

First of all, given that the main DeviceKit daemon was killed by libudev (if you are using GLib you want GUdev instead!), we (that is, myself and other contributors) decided to rename the project - most people were cool with that; see this message for more reasons. One improvement in this area is that we’re committing to limited ABI stability - hopefully this will make things easier on distributors.

Most of the work in the udisks project been triggered by features we want in the Palimpsest Disk Utility - which is the most prominent part of gnome-disk-utility - the other part includes various desktop shell integration bits that I don’t have room to cover in this post.

On the Disk Utility side, we’re now using an user interface that scales better with many disks - in the old user interface, we used to show all partitions as children of each disk in the tree view. In the new user interface, we’re using a grid to convey the partitioning:

Also see bgo #579211

While this interface isn’t optimal for the laptop or the machine-with-only-a-single-disk cases, it works a lot better for machines with many disks

Bunch of disks

The Disk Utility also have somewhat complete support for MD RAID (aka Software RAID on Linux) - we’ll show you running arrays and also arrays not yet running. There’s a way to add/remove components and currently I’m working on creating the UI for Online Capacity Expansion (e.g. dynamically expand a RAID-5 array from 4 to 8 drives).

The SMART stuff has been revamped and there is now a way to turn off the desktop shell notifications. As it turns out a lot of our current users are like “I’m fine with using a disk that is about to fail” - kinda shocking but there you go. Another change is that we don’t consider a disk to be failing if there’s only a couple of bad sectors - the disk will have to have many bad sectors to be considered failing. This helps with all the false positives the SMART warnings generated in e.g. Fedora 11. Anyway, it seems like the desktop notifications are useful - more than a handful of people have pinged me on IRC thanking me and Lennart for this feature. Woot!

We also now have rudimentary support for showing the SAS topology - this is really useful since machines with many disks typically use dedicated disk enclosure shelves. Currently we only show SAS Expanders but the plan is to peek (and poke!) SES-2 Enclosures (and in the future also SGPIO ones) to get information about bay numbering and to toggle the FAILURE and LOCATE leds. Ideally we want to turn on the FAILURE led if a member of a RAID array is kicked or if SMART status indicates that the disk is about to fail. Ideally, we also want to do (sensible!) things like starting a RAID rebuild if a kicked RAID member is replaced by a uninitialized and same-size disk. As always, we have to be really careful about automatically doing policy things like this - especially on Linux where it is hard to make any assumptions on how the system is used.

The goal of the udisks effort is to be useful to users. For example, if you do video editing or a lot of photography, you care about IO performance. So I added a way to easily benchmark drives and RAID arrays:

RAID-6 FTW!

This feature was inspired by various similar Windows programs (a pastime on big storage forums such as this one involves trading benchmark screenshots) and the zcav program. It’s pretty cute, actually; I learned that my Intel 80GB G1 SSD only gives me 140MB/s on my Lenovo Thinkpad X61 while I get a full 270MB/s on my workstation (connected via SAS). Looks like the X61 is only using SATA 150 MB/s - a search on the tubes confirmed this. Gee! This feature is also useful when experimenting with RAID setups - for example, a 10K 36.7GB SAS disk benchmarks like this but when you put three of them together in Software RAID-5 it turns that writing is really slow. I know it’s supposed to be slow when writing to software RAID… but, really, this slow? Maybe I need to tweak some kernel tunables (which would be bad - we should be fast out of the box etc etc).

Finally, I did a really cute hack a few weeks ago - I made Palimpsest use D-Bus over TCP/IP over SSH to speak to udisks:

Bunch of servers

With the way this works we leverage SSH for authentication and Avahi for Service Discovery. The feature is still a bit rough - it needs some UI additions (such as a disconnect button) and we probably want to replace the stock OpenSSH dialog with something, uhm, less scary. There’s also the question of authorization - right now you have to connect as root in order to modify things (udisks uses polkit for authorization) - we need to figure out how we want this to work. Right now I’m thinking udisks will ship some polkit configuration bits (a .pkla file) and a udisks-admin group that administrators can add users to. We’ll see.

I’m kinda excited about the remote connectivity bits - it means that the Disk Utility is now very useful for headless server setups and said servers won’t have to run any graphical UI in order to be managed through the Disk Utility. We probably want to make the Disk Utility (e.g. Palimpsest) run on Win32 and OS X too (for users not running Linux on the desktop) for remote connectivity to e.g. Linux boxes running udisks. It shouldn’t be too hard to do this as we mainly depend on GTK+ and D-Bus.

There’s a bunch of plans for both udisks and Palimpsest. We want to add support (meaning: udisks API and Palimpsest User Interface bits) for the following things: LVM, iSCSI (both target and initiator), Enclosure Management (as mentioned above), Event logging (e.g. “raid5:md2: read error corrected”), Multi-path, Btrfs and many other useful storage technologies. There’s also a few rough edges and missing features (e.g. file system resizing) but we’ll get to that in due time.

FWIW, it’s important to realize that udisks and the Disk Utility complements, but does not replace, existing command-line tools. This means that you can go ahead and keep using your command line tools and scripts and the udisks/GNOME/Palimpsest/etc. stack will react to changes. For example, you can start/stop RAID arrays using mdadm(8), partition your disks using parted(8), create filesystems using mkfs(8), mount/umount filesystems and so on - the UI is kept up to date. While it’s approximately ten times harder (with the 10x figure being 65% accurate) to write software this way, it’s really the only way to do things on something like Linux where our set of users is as diverse as it is.