Wednesday, March 7, 2012

Simpler, Faster, Better

For the past year or so, I've been working on and off on the next version of udisks and what is now known as the Disks GNOME application (née palimpsest):

Hello Shiny!

This code will ship in the upcoming GNOME 3.4 and Fedora 17 releases.

The motivation for rewriting udisks and palimpsest hadn't really anything to do with the user-experience per se - the main motivation was to port udisks/palimpsest code from dbus-glib to use all the D-Bus work I've been doing the last couple of years, e.g.

Since everything had to be rewritten to use new interfaces, I figured that while I was at it, I'd redesign the UI part as well. A big thanks goes to Jon McCann for helping me design the application and enduring my complicated descriptions of how storage works on Linux.

Cleanups

There's a couple of new user-visible features in the code but before delving into specifics I want to mention that we've also removed a couple of features. Why? Because the old interface was quickly becoming a mess and it certainly wasn't very GNOME3-ish neither in the way it looks nor the simplicity you'd expect. If there is one mantra that has driven this effort, it may very well be "don't try to expose everything in a desktop user interface" and I think this is something that extends to the greater GNOME 3 effort as well.

For example, the extent to which Disks support LVM (and MD RAID for that matter) is now that we only show the mapped devices (if the VG / array is running) and, except for e.g. showing the user-friendly (!) name /dev/vg_x61/lv_root instead of /dev/dm-2, that's pretty much it. You can still, of course, manipulate the device (such as changing the file system label) and its contents as if it was any other device (see below), but we no longer provide any UI to configure LVM or MD RAID itself - you are expected to use the command-line tools to do so yourself.

Another important goal of Disks that hasn't changed is the realization that we're only one tool among many - we don't pretend to own your machine and that we're the only tool you'll ever use. That's why you'd e.g. see device names in the user interface (without being primary identifiers) so you can copy/paste that to the terminal and use command-line tools on the device. Additionally, thanks to the notifications from udev, the Disks application will update its UI in response to changes you do from e.g. the command-line or other applications - for example, if you mkfs(8) a new filesystem, or add a new partition using parted(8).

Common operations

Fundamentally, the Disks end-user interface hasn't changed much (it's still basically a tree-view on the left and volume grid on the right), however instead of lots of buttons, most operations are initiated by accessing a popup-menu either for the volume or drive in question. Common operations like formatting a device, changing the passphrase of an encrypted device, creating a new partitionformatting a disk, checking ATA SMART data are pretty much the same, although the UI has been cleaned up some. Pretty basic stuff, really, no big changes here.

Mount and encryption options

A common theme since I started working on storage in desktop Linux (some eight years ago), is that some people are extreme control-freaks about where a device is mounted and with what options. Most of the time you won't need this (the desktop shell will automount the device somewhere in /media) but there's a couple of important uses for this. One common example, is that people are using their computer to serve media files to their TV or Media Jukebox over the LAN. For this to work you often need to make sure the device is mounted in say, /mnt/foo (because /mnt/foo is what you added to the config file for the media server) and that this happens before the media server is started.

We've been trying all sorts of things over the years (the most complex being the now deprecated gnome-mount program reading mount options from e.g. GConf) and while some of these things have worked great in theory, they were just way too complex; our users just didn't have the time nor the inclination to figure out how to use our software... and I don't blame them... my experience is that if the effort more complex than editing a single text file with emacs(1) or vi(1) you've lost most people.

(In retrospect, realizing when your software is too complex for people to use takes much longer than you think - worse, I suspect a lot of developers don't realize just how over-engineered their software is until it's too late.)

So, the way it has worked the past few years in GNOME is that if you want specific configuration for mounting a specific disk, I've been telling people to just add an entry to /etc/fstab and use one of the symlinks  in /dev/disk. Works great. So it was only natural to actually add some UI for this and the result is this dialog

Editing an entry in the /etc/fstab file

which basically correspond to the fields in the /etc/fstab file.

A nice touch here (I think) is that the "Identify As" combo box allows you to select any of the /dev/disk symlinks that currently point to the device, for example, you can make the given mount options apply to any disk connected to the given USB port (actually, partition 1 from said disk). This is of course nothing new, you've always been able to use any /dev/disk symlink in the /etc/fstab file but I bet most people don't have time or inclination to find out exactly what link to use so most just end up using e.g. /dev/sdb1 because that works right now for them. Again, the good old "if it's more complex than editing a single file" line of thought.

(My secret hope here is that even old Unix neck-beards will want to use such UI dialogs instead of editing the /etc/fstab file manually ... or at the very least realize how such a combo box adds value before they flame the tool to death :-) ...)

As for /etc/fstab, we also have similar UI for the /etc/crypttab file except that this is a bit more complex insofar that we allow managing passphrase-files to e.g. automatically unlock a device on boot-up... you may need encrypted /etc for this to be secure - this of course depends on your threat model but it could be you have provisions for securely shredding all content in /etc/luks-keys when needed etc. etc., I don't know :-)

Anyway, the nice thing about using standardized files such as /etc/fstab and /etc/crypttab for this, is interoperability with the rest of the OS - in particular, you can edit the mount options e.g. the boot file-system from the Disks application. Additionally, things like systemd will actually mount devices referenced in the /etc/fstab file at start-up (unless the noauto option is used) which is something you really want for the media server use-case mentioned above.

Disk images and loop devices

Another new feature has to do with disk images - the Disks application now allows you to create and restore disk images (including showing the speed and estimated time left) which is handy especially e.g. from a rescue live cd (it's basically just a GUI for dd(1)). Additionally, there is UI for attaching disk images and using them and, thanks to my friend Kay Sievers, it even works with partitioned disk images

Loop device, with partitions, oh my!

which I think is pretty handy.

Desktop integration

Another important part of the equation is what the user sees when it comes to storage devices in general (e.g. in general, not only when using the Disks application). This is largely the domain of the Desktop Shell, the Files application (née Nautilus) and the file chooser. These pieces all rely on the GLib Volume Monitor APIs which is a high-level abstraction with a stable API/ABI that applications can use (among other things) to figure out what volumes to present in their user interfaces.

In a nutshell, the volumes to show is a complex mixture of physical storage devices (say, mountable filesystems residing on a plugged-in USB stick), devices backed by GVfs backends (say, digital cameras and iPod that are not block devices), GVfs network backends (e.g. smb:// network mounts) and non-storage mounts such as e.g. NFS mounts, FUSE mounts and fstab entries representing things that can be mounted but may not already be mounted.

Now, the way GVfs works is pretty complex (there are multiple volume monitor daemons and storage backends instances) but the important point is that the volume monitor responsible for device and fstab mounts has been updated to use udisks2, the same storage daemon used by the Disks application. In addition to returning GDrive, GVolume and GMount objects for storage devices, this volume monitor also return GVolume entries for /etc/fstab entries so you can e.g. mount a NFS server by clicking an icon in Files (again, a common user request), like this

Desktop Integration. Bringing it all together

Note that the name, icon and desktop visibility of the GVolume instance representing the NFS mount is controlled by mount options in the /etc/fstab file - see this write-up for more information.

23 comments:

  1. I'm a big fan of palimpsest and its awesome to see this update! Big kudos to you, giving us such a clean interface for such a complex area (storage).

    ReplyDelete
  2. Excellent ! This could be a panel in Control Center, don't you think ? Something like "Hardware > Disk & storage"

    ReplyDelete
  3. on demand NFS mount is a must ! Can't wait for it ! (so i can get rid of samba with my nas)

    @tinram: The control center panel is a great idea.

    ReplyDelete
  4. Gnome, slowly becoming the OSX interface for the OpenSource world.
    "That function requires an IQ greater than a potted plant. So we removed it." also known as "Choice? why would you want that?"

    Good to see your sprinting to match the lowest common denominator.

    ReplyDelete
    Replies
    1. The above post is 10 years too late. ;)

      Delete
    2. What? This version seems to have more features as well as a better design. Did you even read the blog post before commenting?

      Delete
    3. His nick sounds like "insane malintent" so he's probably just a troll.

      Delete
  5. I'm in too minds about this. On the one hand, I think it's a shame that you're back-pedalling on features, so you can't use GNOME to do any serious disk work any more. On the other hand, I've never used GNOME to do this anyway - I doubt I'd trust the tools to be honest (that's not meant as an insult. I don't trust nautilus to reliably move a large dir tree from A to B, simply because it's segfaulted enough times whilst I've been trying it. I still use GNOME for the remaining 99%).

    I think it's worth considering what the future looks like. With any luck, if btrfs becomes prevalent, then the tooling needed for what we currently use MD and LVM for will become a lot simpler, and exposing the new simpler surface in a GUI a lot more practical.

    ReplyDelete
  6. I really love where this is going. I'd love to see benchmarking reappear somewhere, maybe in another app (it's gone, right?). I'm really happy with this, though. Especially the mount options!
    This tool feels much more powerful than before, and I get the impression that it'll solve pretty well any disk related problem I have.

    The icon-only buttons do kind of scare me, though. I don't feel very comfortable clicking those when I'm dealing with the hard drives that have all my stuff on them, and I'm not sure if you should trust those icons to carry meaning across locales.
    First, the gears icon is used in two places, for two different menus, although one menu is similar to the other.
    Second, the Unmount and Delete icons feel really awkward beside each other, as they seem so similar. And the Delete button just scares the heck out of me, because its icon is not far from the Unmount button's and I can't tell from the tooltip if there is a confirmation step. It makes me not want to go near the thing.

    Is there really no room for labels? I think they'd solve a lot of problems :)

    Another thing is I kept trying to click the little emblems near each partition (like the star and the play icon), and it took me a while to figure out they're just conveying information. The emblems light up when I hover over them, and it would probably improve things if they didn't do that, and just stayed the same as the text colour at all times.
    Also, maybe they should be lined up with the rest of the text, instead of off to the side.

    ReplyDelete
  7. I miss benchmarking my disks. Will that reappear?

    Also a button for deleting a partition next to mount/dismount seems out of place.

    ReplyDelete
  8. Something that I miss from udisks (talking about older version) is overriding mount parameters.
    Real example I had:
    NTFS partition (on a USB hard drive) mounted using ntfs-3g. I could not get udisks to mount it with exec option (system: Ubuntu 11.4 I think).

    I know, that maybe exec is not something that's 1005 supported on NTFS (or is it?), but in this case lady kept some scripts there to manipulate files she had, and could not run them. It was not feasible to convert to other fs because it also has to be readable under Windows.

    I instructed her to remount the disk manually, adding exec parameter. Just adding it into /etc/fstab was not a best solution either, because she might have to use it another machine, so it's better that she knows how to mount disks.

    Btw. lated IIRC I found the mount parameters hardcoded in udisks source. I think HAL had parameters to tweak all sorts of things like that.

    ReplyDelete
    Replies
    1. Image entertainment.
      VerifEyed is the world leading technology capable of determining whether digital images are original or modified (e.g., Photoshoped).
      image authentication software
      image verificiation software
      image forensic software

      Delete
  9. No one forked old full featured interface?

    ReplyDelete
    Replies
    1. I wish… This removal enrages me (that post's tagline, even more so).

      I've looked into it a bit, but there's some gdbus-generated code in udisks that won't compile anymore.

      Delete
    2. Thats bad, the new idea to make Gnome unuseful will kill this project.

      Delete
  10. Removing the ability to setup and configure RAIDs and LVs was a shit idea, a major step backward.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. had always been a command line user of mdadm, but using disk-utility to manage the raid interface was an order of magnitude easier, provided a better understanding of disks and arrays, status of arrays, creation of arrays with free space, etc... It is a real shame that this feature has been cut out. It does look like RAID support is coming back in 3.7.x versions though... phew.

      Delete
  11. This comment has been removed by the author.

    ReplyDelete
  12. Will RAID configuration reappear in a future version? I thought that was the entire point of GDU, since that was the only non-trivial thing it did.

    ReplyDelete
  13. So you just drop old interface and no backwards compatibility? Thanks, I dream about rewriting work with udisks every month. Thanks to you on new Ubuntu my code to work with udisks broken. And for what reason, for using shiny new glib functionality?

    ReplyDelete