inactivity log for davidz

New D-Bus features in GLib 2.30

2011-09-22T16:44:00.004-04:00

For the upcoming GLib 2.30 release, there's a couple of new features to make it even easier to use D-Bus.

C Code Generator

GLib 2.30 ships with a command called gdbus-codegen(1) which can be used to generate C code. The command is similar in spirit to the dbus-binding-tool(1) command but it targets the D-Bus implementation added to GLib 2.26 (often informally referred to as GDBus). The command's manual page and migration documentation is the authoritative source of documentation, but from a 50,000 feet view what the tool does is simply to map a D-Bus interface (described by XML) to a couple of GObject-based types (including all D-Bus methods, signals and properties). It of course supports the PropertiesChanged signal added in D-Bus spec 0.14.

Even though the tool is targeting C programmers (most higher-level languages are a lot more dynamic than C so objects can be exported via e.g. language meta-data such as annotations), it's useful to note that the generated code is 100% annotated so it can be used from e.g. JavaScript through GObject Introspection - for example the GNOME Documents application (written in JavaScript) is consuming the GNOME Online Accounts client-library which is 99% generated by the gdbus-codegen(1) command.

In addition to just generating code, the gdbus-codegen(1) command can also generate very nice Docbook documentation (example 1, example 2) for D-Bus interfaces - in this respect it's useful even for non-GNOME applications in the same way a lot of libraries like libblkid and libudev are already using gtk-doc for their C library documentation.

Object Manager

Another part of GLib 2.30 is support for the org.freedesktop.DBus.ObjectManager D-Bus interface - in fact, the implementation in GLib, that is, the newly added GDBusObject{Proxy,Skeleton}and GDBusObjectManager{Client,Server} types, is actually what drove me to propose this as a standard interface instead of just doing a GLib-only thing (with shoutouts to my homeboys smcv and walters for excellent review and feedback).

In a nutshell, the org.freedesktop.DBus.ObjectManager interface is basically a formalization of what each and every non-trivial D-Bus service is already doing: offering some kind of GetAll() method (to return all objects) and signals ::Foo{Added,Removed} (to convey changes) on its Manager interface in its own special way (example 1, example 2). It first sounds weird to standardize such a simple thing as object enumeration and change signals but if you think about all the possible edge cases and race-conditions then using a well-tested implementation just makes everything so much easier. Additionally, with the way the interface is defined and the newly added path_namespace match rule, two method invocations and a single round-trip is all it takes for a client to grab the state from the service - this is a huge win compared to existing services that typically first retrieve a list of object paths and then gets properties for each object path (and only if you are lucky it does the latter in parallel).

When combined with the gdbus-codegen(1) command (which can also generate specialized GDBusObject types for use with GDBusObjectManager) you can start writing service-specific code right away and not worry about horrible implementation details like marshaling, too many round trips or race conditions. It just works out of the box as you'd expect it to.

Writing a C library, intro, conclusion and errata

2011-07-06T12:58:00.008-04:00

This is a series of blog-posts about best practices for writing C libraries. See below for each part and the topics covered.

The entire series about best practices for writing C libraries covered 15 topics and was written over five parts posted over the course of approximately one week. Feel free to hotlink directly to each topic but please keep in mind that the content (like any other content on this blog) is copyrighted by its author and may not be reproduced without his consent (if you are friendly towards free software, like e.g. LWN, just ask and I will probably give you permission):

Part one

Part two

Part three

Part four

Part five

Topics not covered

Some topics relevant for writing a C library isn't (yet?) covered in this series either because I'm not an expert on the topic, the topic is still in development or for other reasons:

Networking
You would think IP networking is easy but it's really not and the low-level APIs that are part of POSIX (e.g. BSD Sockets) are not really that helpful since they only do part of what you need. Difficult things here include name resolution, service resolution, proxy server handling, dual-stack addressing and transport security (including handling certificates for authentication).

If you are using modern GLib networking primitives (such as GSocketClient or GSocketService) all of these problems are taken care of for you without you having to do much work; if not, well, talking to people (or at least, read the blogs) such as Dan Winship, Dan Williams or Lennart Poettering is probably your best bet.
Build systems
This is a topic that continues to make me sad so I decided to not really cover it in the series because the best guidance I can give is to just copy/paste whatever other projects are doing - see e.g. the GLib source tree for how to nicely integrate unit testing (see Makefile.decl) and documentation (see docs/reference sub-directories) into the build system (link).

Ideally we would have a single great IDE for developing Linux libraries and applications (integrating testing, documentation, distribution, package building and so on - see e.g. Sami's libhover video) but even if we did, most existing Linux programmers probably wouldn't use it because they are so used to e.g. emacs or vi (if you build it, they will come?). There's a couple of initiatives in this area including Eclipse CDT, Anjuta, KDevelop and MonoDevelop.
Bundling libraries/resources
The traditional way of distributing applications on Linux is through so-called Linux distributions - the four most well-known being Debian, Fedora, openSUSE and Ubuntu (in alphabetical order!). These guys, basically, take your source code, compile it against some version of other software it depends on (usually a different version than you, the developer, used), and then ship binary packages to users using dynamic linking.

There's a couple of problems with this legacy model of distributing software (this list is not exhaustive): a) it can take up to one or two distribution release cycles (6-12 months) before your software is available to end users; and b) user X can't give a copy of the software to user Y - he can only tell him where to get it (it might not be available on user Y's distro); and c) it's all a hodgepodge of version skew e.g. the final product that your users are using is, most likely, using different versions of different libraries so who knows if it works; and d) the software is sometimes changed in ways that you, the original author, wasn't expecting or does not approve of (for example, by removing credits); and e) the distribution might not forward you bug reports or may forward you bug reports that is caused by downstream patches; and f) there's a peer pressure to not depend on too new libraries because distributions want to ship your software in old versions of their OS - for example, Mozilla wants to be able to run on a system with just GTK+ 2.8 installed (and hence won't use features in GTK+ 2.10 or later except for using dlopen()-techniques), similar for e.g. Google Chrome (maybe with a newer GTK+ version though). These problems are virtually unknown to developers on other platforms such as Microsoft Windows, Mac OS X or even some of the smartphone platforms such as iOS or Android - they all have fancy tools that bundles things up nicely so the developers won't have to worry about such things.

There's a couple of interesting initiatives in this area see e.g. bockbuild, glick and the proposal to add a resource-system to GLib. Note that it's very very hard to do this properly since it depends not only on fixing a lot of libraries so they are relocatable, it also depends on identifying exactly what kind of run-time requirements each library in question has. The latter includes the kernel/udev version, the libc version (unless bundled or statically linked), the X11 server version (and its extensions such as e.g. RENDER) version, the presence of one or more message buses and so on. With modern techniques such as direct rendering this becomes even harder if you want to take advantage of hardware acceleration since you must assume that the host OS is providing recent enough versions of e.g. OpenGL or cairo libraries (since you don't want to bundle hardware drivers). And even after all this, you still need to deal with how each distribution patches core components. In some circumstances it might end up being easier to just ship a kernel+runtime along with the application, virtualized.

The way the series is set up is so it can be extended at a later point - so if there is a demand for one or more popular topics about writing a C library, I might write another blog entry and add it to this page as it's considered the canonical location for the entire series.

Errata

Please send me feedback and I will fix up the section in question and credit you here (I already have a couple of corrections lined up that I will add later).

Writing a C library, part 5

2011-07-05T15:31:00.006-04:00

This is part five in a series of blog-posts about best practices for writing C libraries. Previous installments: part one, part two, part three, part four.

API design

A C library is, almost by definition, something that offers an API that is used in applications. Often an API can't be changed in incompatible ways (it can, however, be extended) so it is usually important to get right the first time because if you don't, you and your users will have to live with your mistakes for a long time.

This section is not a a full-blown guide to API design as there's a lot of literature, courses and presentations available on the subject - see e.g. Designing a library that's easy to use - but we will mention the most important principles and a couple of examples of good and bad API design.

The main goals when it comes to API design is, of course, to make the API easy to use - this include choosing good names for types, functions and constants. Be careful of abbreviations - atof might be quick to type but it's not exactly clear that the function parses a C string and returns a double (no, not a float as the name suggests). Typically nouns are used for types and while verbs are used for methods.

Another thing to keep in mind is the number of function arguments - ideally each function should take only a few arguments so it's easy to remember how to use it. For example, no-one probably ever remembers exactly what arguments to pass to g_spawn_async_with_pipes() so programmers end up looking up the docs, breaking the rhythm. A better approach (which is yet to be implemented in GLib), would be to create a new type, let's call it GProcess, with methods to set what you'd otherwise pass as arguments and then a method to spawn the actual program. Not only is this easier to use, it is also extensible as adding a method to a type doesn't break API while adding an argument to an existing function/method does. An example of such an API is libudev's udev_enumerate API - for example, at the time udev starting dealing with device tags, the udev_enumerate type gained the add_match_tag() method.

If using constants, it is often useful to use the C enum type since the compiler can warn if a switch statement isn't handling all cases. Generally avoid boolean types in functions and use flag enumerations instead - this has two advantages: first of all, it's sometimes easier to read foo_do_stuff(foo, FOO_FLAGS_FROBNICATOR) than foo_do_stuff(foo, TRUE) since the reader does not have to expend mental energy on remembering if TRUE translates into whether the frobnicator is to be used or not. Second, it means that several booleans arguments can be passed in one parameter so hard-to-use functions like e.g. gtk_box_pack_start() can be avoided (most programmers can't remember if the expand or fill boolean comes first). Additionally, this technique allows adding new flags without breaking API.

Often the compiler can help - for example, C functions can be annotated with all kinds of gcc-specific annotations that will cause warnings if the user is not using the function correctly. If using, GLib, some of these annotations are available as macros prefixed with G_GNUC, the most important ones being G_GNUC_CONST, G_GNUC_PURE, G_GNUC_MALLOC, G_GNUC_DEPRECATED_FOR, G_GNUC_PRINTF and G_GNUC_NULL_TERMINATED.

Checklist

Choose good type and function names (favor expressiveness over length).
Keep the number of arguments to functions down (consider introducing helper types).
Use the type system / compiler to your advantage instead of fighting it (enums, flags, compiler annotations).

Documentation

If your library is very simple, the best documentation might just be a nicely formatted C header file with inline comments. Often it's not that simple and people using your library might expect richer and cross-referenced documentation complete with code samples.

Many C libraries, including those in GLib and GNOME itself, use inline documentation tags that can be read by tools such as gtk-doc or Doxygen. Note that gtk-doc works just fine even on low-level non-GLib-using libraries - see e.g. libudev and libblkid API documentation.

If used with a GLib library, gtk-doc uses the GLib type system to draw type hierarchies and show type-specific things like properties and signals. gtk-doc can also easily integrate with any tool producing Docbook documentation such as manual pages or e.g. gdbus-codegen(1) when used to generate docs describing D-Bus interfaces (example with C API docs, D-Bus docs and man pages).

Checklist

Decide what level of documentation is needed (HTML, pdf, man pages, etc.).
Try to use standard tools such as Doxygen or gtk-doc.
If shipping commands/daemons/helpers (e.g. anything showing up in ps(1) output), consider shipping man pages for those as well.

Language bindings

C libraries are increasingly used from higher-level languages such as Python or JavaScript through a so-called language binding - for example, this is what allows the Desktop Shell in GNOME 3 to be written entirely in JavaScript while still using C libraries such as GLib, Clutter and Mutter underneath.

It's outside the scope of this article to go into detail on language bindings (however a lot of the advice given in this series does apply - see also: Writing Bindable APIs) but it's worth pointing out that the goal of the GObject Introspection project (which is what is used in GNOME's Shell) is aiming for 100% coverage of GLib libraries assuming the library is properly annotated. For example, this applies to the GUdev library (a thin wrapper on top of the libudev library) can be used from any language that supports GObject Introspection (JS example).

GObject Intropspection is interesting because if someone adds GObject Introspection support to a new language, X, then the GNOME platform (and a lot of the underlying Linux plumbing as well cf. GUdev) is now suddenly available from that language without any work.

Checklist

Make sure your API is easily bindable (avoid C-isms such as variadic functions).
If using GLib, set up GObject Introspection and ship GIR/typelibs (notes).
If writing a complicated application, consider writing parts of it in C and parts of it in a higher-level language.

ABI, API and versioning

While the API of a library describes how the programmer use it, the ABI describes how the API is mapped onto the target machine the library is running on. Roughly, a (shared) library is said to be compatible with a previous version if a recompile is not needed. The ABI involves a lot of factors including data alignment rules, calling conventions, file formats and other things that are not suitable to cover in this series; the important thing to know about when writing C libraries is how (and if) the ABI changes when the API changes. Specifically, since some changes (such as adding a new function) are backwards compatible, the interesting question is what kind of API changes result in non-backwards-compatible ABI changes.

Assuming all other factors like calling convention are constant, the rule of thumb about compatibility on the ABI level basically boils down to a very short list of allowed API changes:

you may add new C functions; and
you may add parameters to a function only if it doesn't cause a memory/resource leak; and
you may add a return value to a function returning void only if it doesn't cause a memory leak; and
modifiers such as const may be added / removed at will since they are not part of the ABI in C

The latter is an example of a change that breaks the API (causing compiler warnings when compiling existing programs that used to compile without warnings) but preserve the ABI (still allowing any previously compiled program to run) - see e.g. this GLib commit for a concrete example (note that this can't be done in C++ because of how name mangling work).

In general, you may not extend C structs that the user can allocate on the stack or embed in another C structure which is why opaque data types are often used since they can be extended without the user knowing. In case the data type is not opaque, an often used technique is to add padding to structs (example) and use it when adding a new virtual method or signal function pointer (example). Other types, such as enumeration types, may normally be extended with new constants but existing constants may not be changed unless explicitly allowed.

The semantics of a function, e.g. its side effect, is usually considered part of the ABI. For example, if the purpose of a function is to print diagnostics on standard output and it stops doing it in a later version of the library, one could argue it's an ABI break even when existing programs are able to call the function and return to the caller just fine possibly even returning the same value.

On Linux, shared libraries (similar to DLLs on Windows) use the so-called soname to maintain and provide backwards-compatibility as well as allowing having multiple incompatible run-time versions installed at the same time. The latter is achieved by increasing the major version number of a library every time a backwards-incompatible change is made. Additionally, other fields of the soname have other (complex) rules associated (more info).

One solution to managing non-backwards-compatible ABI changes without bumping the so-number is symbol versioning - however, apart from being hard to use, it only applies to functions and not e.g. higher-level run-time data structures like e.g. signals, properties and types registered with the GLib type-system.

It is often desirable to have multiple incompatible versions of libraries and their associated development tools installed at the same time (and in the same prefix) - for example, both version 2 and 3 of GTK+. To easily achieve this, many libraries (including GLib and up) include the major version number (which is what is bumped exactly when non-backwards-compatible changes are made) in the library name as well as names of tools and so on - see the Parallel Installation essay for more information.

Some libraries, especially when they are in their early stages of development, specifically gives no ABI guarantees (and thus, does not manage their soname when incompatible changes are made). Often, to better manage expectations, such unstable libraries require that the user defines a macro acknowledging this (example). Once the library is baked, this requirement is then removed and normal ABI stability rules starts applying (example).

Related to versioning, it's important to mention that in order for your library to be easy to use, it is absolutely crucial that it includes pkg-config files along with the header files and other development files (more information).

Checklist

Decide what ABI guarantees to give if any (and when)
Make sure your users understand the ABI guarantees (being explicit is good)
If possible, make it possible to have multiple incompatible versions of your library and tools installed at the same time (e.g. include the major version number in the library name)

Writing a C library, part 4

2011-07-01T15:19:00.002-04:00

This is part four in a series of blog-posts about best practices for writing C libraries. Previous installments: part one, part two, part three.

Helpers and daemons

Occasionally it's useful for a program or library to call upon an external process to do its bidding. There are many reasons for doing this - for example, the code you want to use

might not be easily used from C - it could be written in say, python or, gosh, bash; or
could mess with signal handlers or other global process state; or
is not thread-safe or leaking or just bloated; or
its error handling is incompatible with how your library does things; or
the code needs elevated privileges; or
you have a bad feeling about the library but it's not worth (or (politically) feasible) to re-implement the functionality yourself.

There are three main ways of doing this.

The first one is to just call fork(2) and start using the new library in the child process - this usually doesn't work because chances are that you are already using libraries that cannot be reliably used after the fork() call as discussed in previously (additionally, a lot of unnecessary COW might be happen if the parent process has a lot of writable pages mapped). If portability to Windows is a concern, this is also a non-starter as Windows does not have fork() or any meaningful equivalent that is as efficient.

The second way is to write a small helper program and distribute the helper along with your library. This also uses fork() but the difference is that one of the exec(3) functions is called immediately in the child process so all previous process state is cleaned up when the process image is replaced (except for file descriptors as they are inherited across exec() so be wary of undesired leaks). If using GLib, there's a couple of (portable) useful utility functions to do this (including support for automatically closing file descriptors).

The third way is to have your process communicate with a long-lived helper process (a socalled daemon or background process). The helper daemon can be launched either by dbus-daemon(1) (if you are using D-Bus as the IPC mechanism), systemd if you are using e.g. Unix domain sockets, an init script (uuidd(8) used to do this - wasteful if your library is not going to get used) or by the library itself.

Helper daemons usually serve multiple instances of library users, however it is sometimes desirable to have a helper daemon instance per library user instance. Note that having a library spawn a long-lived process by itself is usually a bad idea because the environment and other inherited process state might be wrong (or even insecure) - see Rethinking PID 1 for more details on why a good, known, minimal and secure working environment is desirable. Another thing that is horribly difficult to get right (or, rather, horribly easy to get wrong) is uniqueness - e.g. you want at most one instance of your helper daemon - see Colin's notes for details and how D-Bus can be used and note that things like GApplication has built-in support for uniqueness. Also, in a system-level daemon, note that you might need to set things like the loginuid (example of how to do this) so things like auditing work when rendering service for a client (this is related to the Windows concept known as impersonation).

As an example, GLib's libproxy-based GProxy implementation uses a helper daemon because dealing with proxy servers involves a interpreting JavaScript (!) and initializing a JS interpreter from every process wanting to make a connection is too much overhead not to mention the pollution caused (source, D-Bus activation file - also note how the helper daemon is activated by simply creating a D-Bus proxy).

If the helper needs to run with elevated privileges, a framework like PolicyKit is convenient to use (for checking whether the process using your library is authorized) since it nicely integrates with the desktop shell (and also console/ssh logins). If your library is just using a short-lived helper program, it's even simpler: just use the pkexec(1) command to launch your helper (example, policy file).

As an aside (since this write-up is about C libraries, not software architecture), many subsystems in today's Linux desktop are implemented as a system-level daemons (often running privileged) with the primary API being a D-Bus API (example) and a C library to access the functionality either not existing at all (applications then use generic D-Bus libraries or tools like gdbus(1) or dbus-send(1)) or mostly generated from the IDL-like D-Bus XML definition files (example). It's useful to contrast this approach to libraries using helpers since one is more or less upside down compared to the other.

Checklist

Identify when a helper program or helper daemon is needed
If possible, use D-Bus (or similar) for activation / uniqueness of helper daemons.
Communicating with a helper via the D-Bus protocol (instead of using a custom binary protocol) adds a layer of safety because message contents are checked.
Using D-Bus through a message bus router (instead of peer-to-peer connections) adds yet another layer of safety since the two processes are connected through an intermediate router process (a dbus-daemon(1) instance) which will also validate messages and disconnects processes sending garbage.
Hence, if the helper is privileged (meaning that it must a) treat the unprivileged application/library using it as untrusted and potentially compromised; and b) validate all data to it - see Wheeler's Secure Programming notes for details), activating a helper daemon on the D-Bus system bus is often a better idea than using a setuid root helper program spawned yourself.
If possible, in particular if you are writing code that is used on the Linux desktop, use PolicyKit (or similar) in privileged code to check if unprivileged code is authorized to carry out the requested operation.

Testing

A sign of maturity is when a library or application comes with a test suite; a good test suite is also incredible useful for ensuring mostly bug-free releases and, more importantly, ensuring that the maintainer is comfortable putting releases out without loosing too much sleep or sanity. Discussing specifics of testing is out of the scope for a series on writing C libraries, but it's worth pointing to the GLib test framework, how it's used (example, example and example) and how this is used by e.g. the GNOME buildbots.

One metric for measuring how good a test suite is (or at least how extensive it is), is determining how much of the code it covers - for this, the gcov tool can be used - see notes on how this is used in D-Bus. Specifically, if the test suite does not cover some edge case, the code paths for handling said edge case will appear as never being executed. Or if the code base handles OOM but the test suite isn't set up to handle it (for example, by failing each allocation) the code-paths for handling OOM should appear as untested.

Innovative approaches to testing can often help - for example, Mozilla employ a technique known as reftests (see also: notes on GTK+ reftests) while the Dracut test suite employs VMs for both client and server to test that booting from iSCSI work.

Checklist

Start writing a test suite as early as possible.
Use tools like gcov to ascertain how good the test suite is.
Run the test suite often - ideally integrate it into the build system ('make check'), release procedures, version control etc.

Writing a C library, part 3

2011-06-29T18:12:00.003-04:00

This is part three in a series of blog-posts about best practices for writing C libraries. Previous installments: part one, part two.

Modularity and namespaces

The C programming language does not support the concept of namespaces (as used in e.g. C++ or Python) so it is usually emulated simply by using naming conventions. The main reason of namespaces is to avoid naming collisions - consider both libwoot and libkool providing a function called get_all_objects() - which one should be used if a program links to both libraries? Namespacing is an important part of a naming strategy and applies to variables, function names, type names (including structs, unions, enums and typedefs) and macros.

The standard convention is to use a short identifier, e.g. for libnm-glib you will see nm_ and NM being used, for Clutter it's clutter and Clutter and for libpolkit-agent-1, it's polkit_agent and PolkitAgent. For libraries that don't use CamelCase for its types, the same prefix is normally used for functions and types - for example, libudev the prefix used is simply udev.

Code that isn't using namespaces properly is not only hard to integrate into other libraries and applications (the chance of symbol collisions is high), there's also a chance that it will collide with future additions to the standard C library or POSIX standards.

One benefit of using namespaces in C (one that ironically is not present in a language with proper support for namespaces), is that it's a lot easier to pinpoint what the code is doing by just looking at a fragment of the source code - e.g. when you see an item being added to a container, you are usually not in doubt whether the programmer meant to invoke GtkContainer's add() method or ClutterContainer's add() method because of how C namespacing forces the programmer to be explicit, for better or worse.

In addition to choosing a good naming strategy, note that the visibility of what symbols (typically variables and functions) a library export can be fine-tuned, see these notes for why this is desirable.

On the topic of naming, it is usually a good idea to avoid C++ keywords (such as "class") for variable names, at least in header files that you except C++ code to include using e.g. extern "C". Additionally, generally avoid names of functions in the C standard library / POSIX for variable names such as "interface" or "index" because these functions can (and on Linux, actually is) be defined as macros.

Checklist

Choose a naming convention - and stick to it.
Do not export symbols that are not public API.

Error handling

If there's one statement that adequately describes error handling in the C programming language, it's perhaps that it's something that people rarely agree on. Most programmers, however, would agree that errors can be broken down into two categories 1) programmer errors; 2) run-time errors.

A programmer error is when the programmer isn't using a function correctly - e.g. passing a non-UTF-8 string to a function expecting a valid UTF-8 string such as g_variant_new_string() (if unsure, validate with g_utf8_validate() before calling the function) or passing an invalid D-Bus name to g_bus_own_name() (if unsure, validate with g_dbus_is_name() and g_dbus_is_unique_name() before calling).

Most libraries have undefined behavior in the presence of being used incorrectly - in the GLib case the macros g_return_if_fail() / g_return_val_if_fail() are used, see e.g. the checks in g_variant_new_string() and the checks in g_dbus_own_name(). Additionally, for performance, these checks can be disabled by defining the macro G_DISABLE_CHECKS when building either GLib itself or applications using GLib (but usually aren't). Not all parameters may be checked, however, and the check might not cover all cases because checks can be expensive. Combined with the G_DEBUG flag, it's even easy to trap this in debugger by running the program in an environment where G_DEBUG=fatal-warnings.

Having g_return_if_fail()-style checks is usually a trade-off - for example, GLib didn't initially have the UTF-8 check in g_variant_new_string() - it was only added when it became apparent that a considerable amount of users passed non-UTF-8 data which caused errors in unrelated code that was extremely hard to track down - see the commit message for details. If this cost is unacceptable, the programmer can easily use the g_variant_new_from_data() function passing TRUE as the trusted parameter.

Even with a library doing proper parameter validation (to catch programmer errors early on), if you pass garbage to a function you usually end up with undefined behavior and undefined behavior can mean anything including formatting your hard disk or evaporating all booze in a five-mile radius (oh noz). That's why some libraries simply calls abort() instead of carrying on pretending nothing happened. In general, a C library can never guarantee that it won't blow up no matter what data is passed - for example the user may pass a pointer to invalid data and, boom, SIGSEGV is raised when the library tries to accesses it. Of course the library could try to recover, longjmp(3) style, but since it's a library it can't mess around with process-wide state like signal handlers. Unfortunately, even smart people sometime fail to realize that the caller has a responsibility and instead blames the library instead of its user (for the record, libdbus-1 is fine which is why process 1 is able to use it without any problems). In most cases, problems like these are solved by just throwing documentation at the problem.

To conclude, when it comes to programmer errors, one key take away is that it's a good idea to document exactly what kind of input a function accepts. As the saying goes, "trust is good, control is better", it is also a good idea to verify that the programmer gets it right by using g_return_if_fail() style checks (and possibly provide API that does no such checks). Also, if your code does any kinds of checks, make sure that the functions used for checking (if non-trivial) are public so e.g. language bindings have a chance to validate input before calling the function (see also: notes on errors in libdbus).

A run-time error is e.g. if fopen(3) returns NULL (for example the file to be opened does not exist or the calling process is not privileged to open it), g_socket_client_connect() returns FALSE (the network might not be up) or g_try_malloc() returns NULL (might not have enough address space for a 8GiB array). By definition, run-time errors are recoverable although the code you are using might treat some (like malloc(3) failing) as irrecoverable because handling some run-time errors (such as OOM) would complicate the API not only on the function level (possibly by taking an error parameter), but also by requiring transactional semantics (e.g. rollback) on most data types (see also: write-up on why handling OOM is hard and a good explanation of Linux's overcommit feature).

For simple libraries just using libc's errno is often simplest approach to handling run-time errors (since it's thread-safe and every C programmer knows it) but note that some functions including asprintf(3) does not set errno to ENOMEM if e.g. failing to allocate memory. If you are basing your code on a library like GLib, use its native error type, e.g. GError, for run-time errors. An interesting approach to handling errors is the one used by the cairo 2D graphics library where (non-trivial) object instances track the error state (see e.g. cairo_status() and cairo_device_status()). There are many many other ways to convey run-time errors - as always, the important thing when writing a C library is to be consistent.

Checklist

Document valid and invalid value ranges for parameters (if any) and provide facilities to validate parameters (unless trivial) for programmers and language bindings
Try to validate incoming parameters at public API boundaries
Establish a policy on how to deal with programmer errors (e.g. undefined behavior or abort()).
Establish a policy on how to deal with run-time errors (e.g. use errno or GError)
Ensure the way you handle run-time errors map to common exception handling systems.

Encapsulation and OO design

While C as programming language does not have built-in support for object-oriented programming lots of C programmers use C that way - in many ways it's almost hard not to. In fact, many C programmers regard the simplicity of C (compared to, say, C++) as a feature insofar that you are not bound to any one object model - for example, the kernel uses various OO techniques and the GLib/GTK+ stack has its own dynamic type system called GType on which the GObject base class (that many classes are derived from) is built.

There's of course a price to pay for defining your own object model - it typically involves more typing (identifiers are longer) and, especially for GObject, involves actual function calls to register properties, add private instance data and so on (example). On the other hand, such a dynamic type system often offer some level of type introspection so it's possible to easy link the property for whether a check-button widget is active with whether an text-entry widget should use password mode using the g_object_bind_property() function (screenshot). Polymorphism in GObject is provided by embedding a virtual method table in the class struct (example) and providing a C functions that uses the function pointer (example) - note that derived types can access the class struct to chain up (example).

One important feature of object-oriented design in C is that it usually promotes encapsulation and data hiding through the use of opaque data types - this is desirable as it allows extending the data type (e.g. adding more properties or methods) without breaking or requiring a recompile existing programs using the library (a future installment will discuss API and ABI and what it means wrt. API design). In an opaque data type, fields that would usually be in the C struct are hidden from the user and instead are made available via a getter (example) and/or setter (example) - additionally, if the object model support properties, the member may also be made available as a property (example) - for example, this is useful for notifying when the property changes.

Of course, not every single data structure need to be a full-blown GObject - for example, in some cases data hiding might not be desirable (sometimes it's awkward to use a C getter function) or maybe it's too slow to do from an inner loop (direct struct access is without a doubt faster). Also, for simple data structures it is sometimes desirable to initialize struct instances directly in the code.

Even when a full-blown object model (like GType and GObject) isn't used, it's never a bad idea to use opaque data structures and getters/setters. As an interesting alternative to this, note that some libraries explicitly allows extending a C structure without considering it an ABI change - while there's no easy way to enforce this (the user may allocate the structure on the stack), at least the library author can always tell the programmer that he shouldn't have done so (which may or may not be useful).

Checklist

Establish an object model for your library (if applicable).
Hide as many implementation details as is practical without impacting performance
Ensure that you can extend your library and types without breaking API or ABI.
If possible, build on top of an established and well-understood object system (such as the GLib one)

Writing a C library, part 2

2011-06-28T16:15:00.003-04:00

This is part two in a series of blog-posts about best practices for writing C libraries. Previous installments: part one.

Event handling and the main loop

Event-driven applications, especially GUI applications, are often built around the idea of a "main loop" that intercepts and dispatches all sorts of events such as key/button presses, incoming IPC calls, mouse movements, timers and file/socket I/O and so on. The main loop typically "call back" into the application whenever an event happens.

For example, the GLib/Gtk+ library stack is built around the GMainContext, GMainLoop and GSource types and other library stacks provide similar abstractions. Many kernel and systems-level programmers often look funny at GUI programmers when they utter the word "main loop" - much the same way GUI programmers stare confused at kernel programmers when they say put() or get() something. The truth is that a main-loop is really a well-known concept with a different name: it's basically an abstraction of OS primitives such as select(2), poll(2) or equivalent on e.g. Windows.

It is important to note that a multi-threaded application may run different main loops in different threads to ensure that callbacks happen in the right threads - in GLib this is achieved by using the g_main_context_push_thread_default() function which records the main loop for the current thread in thread-local storage. This variable is in turn read when starting an asynchronous operation (such as g_dbus_connection_call() or g_input_stream_read_async()) to ensure that the passed callback function is invoked in a thread running a main loop for the context set with g_main_context_push_thread_default() earlier.

Some main loops, for example the GLib one, allows creating recursive main loops and this is used to implement GtkDialog's run() method. While this indeed appears to block the calling thread, it is important to note that events are still being processed (to e.g. process input events and redraw animations). Specifically, this means that the functions (plural since applies to everything in the call stack) that brought up the dialog might end getting called again (from a callback). Thus, when using functions like gtk_dialog_run() you need to ensure that your functions are either re-entrant or that they are guaranteed to not get called when the dialog is showing (typically achieved by making the dialog modal so the UI action triggering the display of the dialog can't be accessed). Because of pitfalls like this, you must clearly document if a function is using a recursive main loop.

Note that main loops are not a GUI-only concept - a lot of daemons (e.g. background process without any GUI) are built around this concept since it nicely integrates events from any source whether they are file descriptor based or synthetic such as timers or logging events. In fact, a considerable part of the system-level software on a modern Linux system is built on top of GLib and uses its main event loop abstraction to dispatch events - most of the time such daemons sit idle in one or more main loops and wait for D-Bus messages to arrive (to service a client), a timeout to fire (maybe to kick off periodic house-keeping tasks) or a child process to terminate (when using a helper program or process to do work).

Now that we've explained what a main loop is (or rather, what the idea of a main loop is), let's look at why this matters if you a writing a C library. First of all, if your library doesn't need to deliver events to users, you don't need to worry about main loops. Most libraries, however, are not that simple - for example, libudev delivers events when devices are plugged or changed, NetworkManager wants to inform of changes in networking and so on.

If your library is using GLib, it is often suitable to just require that the user runs the GLib main loop (if the application is using another main loop, it can either integrate the GLib main loop (like Qt does) or run it in a separate thread) and use g_main_context_get_thread_default() when setting up a callback. This is the way many GObject-based libraries, such as libpolkit-gobject-1, libnm-glib or libgudev work - for example, callbacks connected to the GUdevClient::uevent signal will be called in what was the thread-default main loop when the object was constructed. For a shared resource, such as a message bus connection, a good policy is that callbacks happen in what was the the thread-default main loop when the method was called (see e.g. g_dbus_connection_signal_subscribe() where this is the case) since applications or libraries have no absolute control of when the shared object was created. In any case, functions dealing with callbacks must always document in what context the callback happens in.

On the other hand, if your library is not using GLib, a good way to provide notification is simply to a) provide a file descriptor that e.g. turns readable when there are events to process; and b) provide a function that processes events (and possibly invoke callback functions registered by the user). A good example of this is libudev's udev_monitor_get_fd() and udev_monitor_receive_device() functions. This way the application (or the library using your library) can easily control what thread the event is handled in. As an example of how libudev is integrated into the GLib main loop, see here and here. In the libudev case, the returned file descriptor is the underlying netlink socket used to receive the event from udevd (via the kernel); in the case that there are no natural file descriptor (could be the event is only happening in response to a certain entry in, say, a log file), your library could use pipe(2) (or eventfd(2) if on Linux) and use a private worker thread to signal the other end.

If your library provide callback functions, make sure they take user_data arguments so the user can easily associate callbacks with other objects and data. If the scope of your callback is undefined (e.g. may fire more than once or if there is no way to disconnect the callback), also provide a way to free the user_data pointer when it is no longer needed - otherwise the application will leak data that needs to be freed later. See g_bus_watch_name() and for an example.

Checklist

Provide APIs for main loop integration
Make sure callback functions take user_data arguments (possibly with free an accompanying free function)

Synchronous and Asynchronous I/O

It is important for users of a library to know if calling a function involves doing synchronous I/O (also called blocking I/O). For example, an application with an user interface need to be responsive to user input and may even need to update the user interface every frame for smooth animations (e.g. 60 times a second). To avoid unresponsive applications and jerky animations, its UI thread must never call any functions that does any synchronous I/O.

Note that even loading a file from local disk may block for a very long amount of time - sometimes tens of seconds. For example the file may not be in the page cache and the hard disk with the file system may be powered down - or the file may be in the users home directory which could be on a network filesystem such as NFS. Other examples of blocking IO includes local IPC such as D-Bus or Unix domain sockets.

If an operation is known to take a long time to complete (synchronous or otherwise), it is often nice if it is possible to easily cancel the it (perhaps from another thread). For example, see the GCancellable type in the GLib stack. Another nicety (although easily implemented via the GCancellable type) is a way to set a timeout for potentially long-running operations - see e.g. g_dbus_connection_send_message_with_reply() and g_dbus_proxy_call() and note how the latter has an object-wide timeout so the timeout only has to be set once.

Some libraries provide both synchronous and asynchronous versions of a function where the former blocks the calling thread and the latter doesn't. Typically asynchronous I/O is implemented using worker threads (where the worker thread is doing synchronous I/O) but it could also involve communicating with another process via IPC (e.g. D-Bus) or even TCP/IP. For example, in the libgio-2.0 case asynchronous file I/O is implemented via synchronous IO (e.g. basically read(2) and write(2) calls) in worker threads (using a GThreadPool) simply because the Linux kernel does not (yet?) provide an adequate way to do asynchronous I/O suitable for libraries (see also: colorful notes about Asynchronous I/O). On the upside, this is mostly an implementation detail and the libgio-2.0 implementation can migrate to a non-threaded approach should such a mechanism be made available in the future.

Asynchronous I/O typically involves callbacks (or at least some kind of event notification) and thus involves a main loop. If a library provides functions for this, it should clearly state what thread the callback will happen in, and whether it requires the application to run a (specific kind of) main loop - see the previous section about main loops for details.

If a library is thread-safe, it is often easier for the application itself to just use the synchronous version of a function in a worker-thread - if using GLib, g_io_scheduler_push_job() is the right way to do that.

In some cases synchronous I/O is implemented by using a recursive main loop (typically by using the asynchronous form of the function) - this should be avoided as it typically causes all kinds of problems because of reentrancy and events being processed while waiting for the supposedly synchronous operation to complete. As always, clearly document what your code is doing.

Some libraries, such as those in the GLib stack, use a consistent pattern for asynchronous I/O for all of its functions involving the GAsyncResult / GSimpleAsyncResult, GAsyncReadyCallback and GCancellable types - this makes it a lot easier both for programmers and for higher-level language bindings especially since important things like life-cycles are part of this model (for example, you are guaranteed that the callback will always happen, even on cancellation, timeout or error).

Checklist

Clearly document if a function does any synchronous I/O
Ideally suffix synchronous functions it with _sync() so it's easy to inspect large code-trees using e.g. grep(1)
Consider if an operation needs to be available in synchronous or asychronous form or both.
Point to both synchronous and asynchronous functions in your API documentation.
If possible, use an established model (such as the GIO model) for I/O instead of rolling your own

Writing a C library, part 1

2011-06-27T15:49:00.008-04:00

This is part one in a series of blog-posts about best practices for writing C libraries.

Base libraries

Since libc is a fairly low-level set of libraries, there exists higher-level libraries to make C programming a more pleasant experience including libraries in the GLib and and GTK+ stack. Even while the following is going to be somewhat GLib- and GTK+-centric, these notes are written to be useful for any C code whether it's based on libc, GLib or other libraries such as NSPR, APR or some of the Samba libraries.

Most programmers would agree that it's usually a bad idea to implement basic data-types such as string handling, memory allocation, lists, arrays, hash-tables or queues yourself just because you can - it only makes code harder to read and harder to maintain by others. This is where C libraries such as GLib and GTK+ come into play - these libraries provides much of this out of the box. Plus, when you end up needing non-trivial utility functions (and chances are you will) for, say, Unicode manipulation, rendering complex scripts, D-Bus support or calculating checksums, ask yourself (or worse: wait until your manager or peers ask you) if the decision to avoid a well-tested and well-maintained library was a good decision.

In particular, for things like cryptography, it is usually a bad idea to implement it yourself (however inventing your own algorithm is worse); instead, it is better to use an existing well-tested library such as NSS (and even if you do, be careful of using the library correctly). Specifically, said library may even be FIPS-140 certified which is a requirement if you want to do business with the US government.

Similarly, while it’s more efficient to use e.g. epoll than poll for event notification, maybe it doesn't matter if your application or library is only handling on the order of ten file descriptors. On the other hand, if you know that you are going to handle thousands of file descriptors, you can still use e.g. GLib for the bulk of your library or application - just use epoll from dedicated threads. Ditto, if you need O(1) removal from a list, maybe don’t use a GList - use an embedded list instead.

Above all, no matter what libraries or code you end up using, make sure you have at least a rudimentary understanding of the relevant data-types, concepts and implementation details. For example, with GLib it is very easy to use high-level constructs such as GHashTable, g_timeout_add() or g_file_set_contents() without knowing how things are implemented or what a file descriptor really is. For example, when saving data, you want to do so atomically (to avoid data-loss) and just knowing that g_file_set_contents() Does The Right Thing(tm) is often enough (often just reading the API docs will tell you what you need to know). Additionally make sure you understand both the algorithmic complexity of the data-types you end up using and how they work on modern hardware.

Finally, try not to get caught up in religious discussions about “bloated” libraries with random people on the Internet - it’s usually not a good a use of time and resources.

Checklist

Don’t reinvent basic data-types (unless performance is a concern).
Don’t avoid standard libraries just because they are portable.
Be wary of using multiple libraries with overlapping functionality.
To the extent where it’s possible, keep library usage as a private implementation detail.
Use the right tool for the right job - don’t waste time on religious discussions.

Library initialization and shutdown

Some libraries requires that an function, typically called foo_init(), is called before other functions in the library is called - this function typically initializes global variables and data structures used by the library. Additionally, libraries may also offer a shutdown function, typically called foo_shutdown() (forms such as foo_cleanup(), foo_fini(), foo_exit() and the grammatically dubious foo_deinit() have also been observed in the wild), to release all resources used by the library. The main reason for having a shutdown() function is to play nicer with Valgrind (for finding memory leaks) or to release all resources when using dlopen() and friends.

In general, library initialization and shutdown routines should be avoided since they might cause interference between two unrelated libraries in the dependency chain of an application; e.g. if you don’t call them from where they are used, you are possible forcing the application to call a init() function in main(), just because some library deep down in the dependency chain is using the library without initializing it.

However, without a library initialization routine, every function in the library would have to call the (internal) initialization routine which is not always practical and may also be a performance concern. In reality, the check only has to be done in a couple of functions since most functions in a library depends on an object or struct obtained from e.g. other functions in the library. So in reality, the check only has to be done in _new() functions and functions not operating on an object.

For example, every program using the GLib type system has to call g_type_init() and this includes libraries based on libgobject-2.0 such as libpolkit-gobject-1 - e.g. if you don’t call g_type_init() prior to calling polkit_authority_get_sync() then your program will probably segfault. Naturally this is something most people new to the GLib stack gets wrong and you can’t really blame them - if anything, g_type_init() is a great poster-child of why init() functions should be avoided if possible.

One reason for library initialization routine has to do with library configuration, either app-specific configuration (e.g. the application using the library might want to force a specific behavior) or end-user specific (by manipulating argc and argv) - for example, see gtk_init(). The best solution to this problem is of course to avoid configuration, but in the cases where it’s not possible it is often better to use e.g. environment variables to control behavior - see e.g. the environment variables supported by libgtk-3.0 and the environment variables supported by libgio-2.0 for examples.

If your library does have an initialization routine, do make sure that it is idempotent and thread-safe, e.g. that it can be called multiple times and from multiple threads at the same time. If your library also has a shutdown routine, make sure that some kind of “initialization count” is used so the library is only shutdown once all users of it have called its shutdown() routine. Also, if possible, ensure that your library init/shutdown routines calls the init/shutdown routines for libraries that it depends on.

Often, a library's init() and shutdown() functions can be removed by introducing a context object - this also fixes the problem of global state (which is undesirable and often break multiple library users in the same process), locking (which can then be per context instance) and callbacks / notification (which can call back / post events to separate threads). For example, see libudev's struct udev_monitor.

Checklist

Avoid init() / shutdown() routines - if you can’t avoid them, do make sure they are idempotent, thread-safe and reference-counted.
Use environment variables for library initialization parameters, not argc and argv.
You can easily have two unrelated library users in the same process - often without the main application knowing about the library at all. Make sure your library can handle that.
Avoid unsafe API like atexit(3) and, if portability is a concern, unportable constructs like library constructors and destructors (e.g. gcc’s __attribute__ ((constructor)) and __attribute__ ((destructor))).

Memory management

It is good practice to provide a matching free() function or each kind of allocated object that your API returns. If your library uses reference counting, it is often more appropriate to use the suffix _unref instead of _free. An example of this in the GLib/GTK+ stack the functions used are g_object_new(), g_object_ref() and g_object_unref() that operate on instances of the GObject type (including derived types). Similarly, for the GtkTextIter type, the relevant functions are gtk_text_iter_copy() and gtk_text_iter_free(). Also, note that some objects may be stack-allocated (such as GtkTextIter) while others (such as GObject) can only be heap-allocated.

Note that some object-oriented libraries with the concept of derived types may require the app to use the unref() method from a base type - for example, an instance of a GtkButton must be released with g_object_unref() because GtkButton is also a GObject. Additionally, some libraries have the concept of floating references (see e.g. GInitiallyUnowned, GtkWidget and GVariant) - this can make it more more convenient to use the type system from C since it e.g. allows using the g_variant_new() constructor in place of a parameter like in the example code for g_dbus_proxy_call_sync() without leaking any references.

Unless it’s self-evident, all functions should have documentation explaining how parameters are managed. It is often a good idea to try to force some kind of consistency on the API. For example, in the GLib stack the general rule is that the caller owns parameters passed to a function (so the function need to take a reference or make a copy if the parameter is used after the function returns) and that the callee owns the returned parameters (so the caller needs to make a copy or increase the reference count) unless the function can be called from multiple threads (in which case the caller needs to free the returned object).

Note that thread-safety often dictates what the API looks like - for example, for a thread-safe object pool, the lookup() function (returning an object) must return a reference (that the caller must unref()) because the returned object could be removed from another thread just after lookup() returns - one such example is g_dbus_object_manager_get_object().

If you implement reference counting for an object or struct, make sure it is using atomic operations or otherwise protect the reference count from being modified simultaneously by multiple threads.

If a function is returning a pointer to memory that the caller isn’t supposed to free or unref, it is often necessary to document for how long the pointer is valid - for example the documentation for the getenv() C library function says “The string pointed to by the return value of getenv() may be statically allocated, and can be modified by a subsequent call to getenv(), putenv(3), setenv(3), or unsetenv(3).”. This is useful information because it shows that care should be taken if the result from getenv() is used by multiple threads; also this kind of API can never work in a multi-threaded application and the only reason it works is that applications or libraries normally don’t modify the environment.

It is often advantageous for an application to not worry about out-of-memory conditions and instead just call abort() if the underlying allocator signals an out-of-memory condition. This holds true for most libraries as well since it allows a simpler and better API and huge code-footprint reductions. If you do decide to worry about OOM in your library, do make sure that you test all code-paths or your effort will very likely have been in vain. On the other hand, if you know your library is going to be used in e.g. process 1 (the init process) or other critical processes, then not handling OOM is not an option.

Checklist

Provide a free() or unref() function for each type your library introduces.
Ensure that memory handling consistent across your library.
Note that multi-threading may impose certain kinds of API.
Make sure the documentation is clear on how memory is managed.
Abort on OOM unless there are very good reasons for handling OOM.

Multiple Threads and Processes

A library should clearly document if and how it can be used from multiple threads. There are often multiple levels of thread-safety involved - if the library has a concept of objects and a pool of objects (as most libraries do), the enumeration and management of the pool might be thread safe while applications are supposed to provide their own locking when operating on a single object from multiple threads, concurrently.

If you are providing a function performing synchronous I/O, it is often a good idea to make it thread-safe so an application can safely use it from a helper thread

If your library is using threads internally, be wary of manipulating process-wide state, such as the current directory, locale, etc. Doing so from your private worker thread will have unexpected consequences for the application using your library.

A library should always use thread-safe functions (e.g. getpwnam_r() rather than getpwnam()) and avoid libraries and code that is not thread-safe. If you can’t do this, clearly state that your library isn’t thread-safe so applications can use it from a dedicated helper process instead if they need thread-safety.

It is also important to document if your library is using threads internally, e.g. for a pool of worker threads. Even though you think of the thread as a private implementation detail, its existence can affect users of your library; e.g. Unix signals might need to be handled differently in the the presence of threads, and there are extra complications when forking a threaded application.

If your library has interfaces involving resources that can be inherited over fork(), such as file descriptors, locks, memory obtained from mmap(), etc, you should try to establish a clear policy for how an application can use your library before/after a fork. Often, the simplest policy is the best: start using nontrivial libraries only after the fork, or offer a way to reinitialize the library in the forked process. For file descriptors, using FD_CLOEXEC is a good idea. In reality most libraries have undefined behavior after the fork() call, so the only safe thing to do is to call the exec() function.

Checklist

Document if and how the library can be used from multiple threads.
Document what steps need to be taken after fork() or if the library is now unusable.
Document if the library is creating private worker threads.

GNOME Online Accounts

2011-04-27T11:52:00.001-04:00

The past couple of weeks I've been looking into how to make GNOME work with online accounts. I sent some notes about to d-d-l about this last week. I'm now at a point where I have working code. And with working code, I of course mean a working screencast (Youtube Video):OK, I mentioned working code and I wasn't kidding — it's stashed in this git repo for now. I expect to move it to git.gnome.org assuming we decide to use this in GNOME. More importantly, I've uploaded the docs here. Oh, and to build this, you will need GLib from the gdbus-codegen branch — fear not, this will be merged to mainline soon!

GNOME 3.0 is out

2011-04-06T17:46:00.000-04:00

Yay, yay, yay, GNOME 3.0 is out!

I'm especially proud of being part of this release, as I was involved in coding up two very user-visible pieces - the Authorization Dialog and the Calendar Drop-down. Not only is the shell super-duper hot looking (making GNOME 2.x vintage desktops look like Windows 95 more than ever), it is a true pleasure to use. One that grows on you. One that makes you never go back.

The best part about this release is probably how re-energized everyone working on GNOME 3 is. It is important to realize that 3.0 is only the beginning and that we have tons of stuff lined up for 3.0.x, 3.2 and beyond. I'm especially psyched about the prospect of making the desktop shell integrate nicely with the many Web Applications that are in use today. Myself? I'm already busy working on improvements for 3.2 - for the authorization dialog we want to present better strings to the user (e.g. including name of app in the dialog and what object the authorization is about) and for the calendar we want Locations (and possibly weather too) in the drop-down. And ideally an easier way to make the shell show e.g. your Google Calendars if that is your calendar application of choice.

PolicyKit textual authentication

2011-02-22T12:52:00.000-05:00

Related to yesterday's bling post, I wanted to mention that PolicyKit also has a textual authentication agent (video):

This is useful in PolicyKit applications running in a non-graphical environment. It's simple to use: simply use PolkitAgentTextListener if the response from PolicyKit indicates that authorization can be obtained if more information is provided. Here are the changes that was needed for pkexec(1) command. An obvious candidate for this would be the pkcon(1) command.

GNOME 3 Authorization

2011-02-21T15:20:00.001-05:00

The past couple of days I've been working on implementing the GNOME 3 Authorization Dialog. The implementation uses PolicyKit to achieve this and is conceptually very simple: the Shell process itself just implements the org.freedesktop.PolicyKit1.AuthenticationAgent interface to handle authentication requests from the system-wide authority (video).

For authentication, the Shell is using the existing code that in turn uses PAM. This means that the Authorization Dialog will work out-of-the-box with e.g. fingerprint readers, provided, of course, that the PAM stack has been set up correctly (video).

Additionally, the code has been tested with pam_rps - a PAM module whereby you authenticate by proving that you are smart enough to win in Rock-paper-scissors. The idea is that if your application works with pam_rps, then there's a good chance that it will work with any PAM module (video).

One major difference - and something I've been wanting to do for a while - is that the dialogs are now system-modal. In addition to fixing a lot of focus problems, this also makes the dialogs look more trustworthy since they are part of the actual shell. They're also way prettier.

(Bonus chatter: Of course, there's an emphasis on look in the above paragraph because any random app can still go full-screen to fool the user to give them their password. To do this in a more secure way, we need to use the Secure Attention Key and run the dialog in another security context etc. It's an open question whether it's even worth it to do this. But I digress.)

You can find the current code drop in this bug.

GNOME 3 Calendar

2011-01-31T13:01:00.000-05:00

From design through review to code, I am proud to present the new GNOME calendar panel:

Thanks to everyone involved!

GNOME 3 Calendar Integration

2010-11-30T15:03:00.003-05:00

I'm actually not supposed to be working this week (the first screenshot contains a subtle hint why), but here's a couple of screenshots of the upcoming calendar integration for the next major GNOME release:

Click to embiggen

Working in Javascript is really weird compared to C. But so far I think it's weird in a good way.

Click to embiggen

Here's the git branch with this code and here's the bug. There's still some work left to make the code look and act like the mockups. I'm really hoping we can land the bulk of this code before Christmas - the biggest blocker right now is getting events from Evolution which is stressing the capabilities of the gjs runtime and GObject Introspection. To make forward progress on the UI side, I had to switch to a in-process event source with fake events.

gdbus(1) bash completion

2010-08-18T17:58:00.000-04:00

It seems to be a well-kept secret that the new gdbus(1) command available in the latest GLib releases does bash completion. So here's a quick screencast demonstrating this very useful feature!

Enjoy!

This blog has moved...

2010-05-19T00:07:00.001-04:00

I'm no longer running my own server (good riddance!) so http://blog.fubar.dk does not work anymore (except it being a CNAME to here but I'm not sure how well that works). If you're seeing this, you're in the right place. The new setup has comments enabled (an often requested feature) and it looks like it's easier to use and stuff. Hopefully it'll make me blog some more. We'll see!

udisks and gnome-disk-utility - past, present and future

2009-12-16T11:16:00.000-05:00

The past year or so I’ve been working a lot on gnome-disk-utility and DeviceKit-disks^Wudisks and I haven’t really blogged anything about it. Time to fix that!

First of all, given that the main DeviceKit daemon was killed by libudev (if you are using GLib you want GUdev instead!), we (that is, myself and other contributors) decided to rename the project - most people were cool with that; see this message for more reasons. One improvement in this area is that we’re committing to limited ABI stability - hopefully this will make things easier on distributors.

Most of the work in the udisks project been triggered by features we want in the Palimpsest Disk Utility - which is the most prominent part of gnome-disk-utility - the other part includes various desktop shell integration bits that I don’t have room to cover in this post.

On the Disk Utility side, we’re now using an user interface that scales better with many disks - in the old user interface, we used to show all partitions as children of each disk in the tree view. In the new user interface, we’re using a grid to convey the partitioning:

Also see bgo #579211

While this interface isn’t optimal for the laptop or the machine-with-only-a-single-disk cases, it works a lot better for machines with many disks

Bunch of disks

The Disk Utility also have somewhat complete support for MD RAID (aka Software RAID on Linux) - we’ll show you running arrays and also arrays not yet running. There’s a way to add/remove components and currently I’m working on creating the UI for Online Capacity Expansion (e.g. dynamically expand a RAID-5 array from 4 to 8 drives).

The SMART stuff has been revamped and there is now a way to turn off the desktop shell notifications. As it turns out a lot of our current users are like “I’m fine with using a disk that is about to fail” - kinda shocking but there you go. Another change is that we don’t consider a disk to be failing if there’s only a couple of bad sectors - the disk will have to have many bad sectors to be considered failing. This helps with all the false positives the SMART warnings generated in e.g. Fedora 11. Anyway, it seems like the desktop notifications are useful - more than a handful of people have pinged me on IRC thanking me and Lennart for this feature. Woot!

We also now have rudimentary support for showing the SAS topology - this is really useful since machines with many disks typically use dedicated disk enclosure shelves. Currently we only show SAS Expanders but the plan is to peek (and poke!) SES-2 Enclosures (and in the future also SGPIO ones) to get information about bay numbering and to toggle the FAILURE and LOCATE leds. Ideally we want to turn on the FAILURE led if a member of a RAID array is kicked or if SMART status indicates that the disk is about to fail. Ideally, we also want to do (sensible!) things like starting a RAID rebuild if a kicked RAID member is replaced by a uninitialized and same-size disk. As always, we have to be really careful about automatically doing policy things like this - especially on Linux where it is hard to make any assumptions on how the system is used.

The goal of the udisks effort is to be useful to users. For example, if you do video editing or a lot of photography, you care about IO performance. So I added a way to easily benchmark drives and RAID arrays:

RAID-6 FTW!

This feature was inspired by various similar Windows programs (a pastime on big storage forums such as this one involves trading benchmark screenshots) and the zcav program. It’s pretty cute, actually; I learned that my Intel 80GB G1 SSD only gives me 140MB/s on my Lenovo Thinkpad X61 while I get a full 270MB/s on my workstation (connected via SAS). Looks like the X61 is only using SATA 150 MB/s - a search on the tubes confirmed this. Gee! This feature is also useful when experimenting with RAID setups - for example, a 10K 36.7GB SAS disk benchmarks like this but when you put three of them together in Software RAID-5 it turns that writing is really slow. I know it’s supposed to be slow when writing to software RAID… but, really, this slow? Maybe I need to tweak some kernel tunables (which would be bad - we should be fast out of the box etc etc).

Finally, I did a really cute hack a few weeks ago - I made Palimpsest use D-Bus over TCP/IP over SSH to speak to udisks:

Bunch of servers

With the way this works we leverage SSH for authentication and Avahi for Service Discovery. The feature is still a bit rough - it needs some UI additions (such as a disconnect button) and we probably want to replace the stock OpenSSH dialog with something, uhm, less scary. There’s also the question of authorization - right now you have to connect as root in order to modify things (udisks uses polkit for authorization) - we need to figure out how we want this to work. Right now I’m thinking udisks will ship some polkit configuration bits (a .pkla file) and a udisks-admin group that administrators can add users to. We’ll see.

I’m kinda excited about the remote connectivity bits - it means that the Disk Utility is now very useful for headless server setups and said servers won’t have to run any graphical UI in order to be managed through the Disk Utility. We probably want to make the Disk Utility (e.g. Palimpsest) run on Win32 and OS X too (for users not running Linux on the desktop) for remote connectivity to e.g. Linux boxes running udisks. It shouldn’t be too hard to do this as we mainly depend on GTK+ and D-Bus.

There’s a bunch of plans for both udisks and Palimpsest. We want to add support (meaning: udisks API and Palimpsest User Interface bits) for the following things: LVM, iSCSI (both target and initiator), Enclosure Management (as mentioned above), Event logging (e.g. “raid5:md2: read error corrected”), Multi-path, Btrfs and many other useful storage technologies. There’s also a few rough edges and missing features (e.g. file system resizing) but we’ll get to that in due time.

FWIW, it’s important to realize that udisks and the Disk Utility complements, but does not replace, existing command-line tools. This means that you can go ahead and keep using your command line tools and scripts and the udisks/GNOME/Palimpsest/etc. stack will react to changes. For example, you can start/stop RAID arrays using mdadm(8), partition your disks using parted(8), create filesystems using mkfs(8), mount/umount filesystems and so on - the UI is kept up to date. While it’s approximately ten times harder (with the 10x figure being 65% accurate) to write software this way, it’s really the only way to do things on something like Linux where our set of users is as diverse as it is.

Plumbers 2009

2009-09-29T11:34:00.000-04:00

Had a great time at Plumbers Conference last week in lovely Portland, OR. On the Friday on the conference, Kay and I gave a talk, aptly named “Replugging the Modern Desktop”. The talk covered most of the hal-ectomy that has been happening and included a couple of demos. The slides are here. UPDATE: People on IRC are telling me that the fd.o servers are not reachable, I’ve put up another copy of the slides here.

GUdev, gobject-introspection

2009-06-01T14:02:00.000-04:00

I just committed gudev-1.0 to udev-extras. GUdev isn’t a particulary big library, it’s just a thin wrapper around libudev with integration bits for the GLib main loop and support for GObject Introspection.

While GObject introspection is very neat, see this GUdev example using Seed, I must admit the whole thing feels… half-finished?

For example, we seem to be doing the wrong thing for default annotations in some cases. Another example is the fact that gjs lacks a print() function, Seed can’t handle GStrv properties (works in gjs), lack of support for string arrays in gjs (works in Seed), neither Seed nor gjs supports Shebang support (gjs bug, Seed bug).

So all in all, trying to make my new small library introspectable.. was kinda.. much more work than I expected.. and most of the time was spent with the Seed and gjs runtimes trying to figure out what was wrong.

Anyway, I don’t want to sound all negative.. with all the work going on in this area, it looks like it’s going to rock once corner cases like mentioned above are ironed out. Rock on!

Storage handling in GNOME

2009-05-04T05:40:00.000-04:00

The GNOME 2.26 release in Fedora 11 will ship with a completely different stack for handling storage devices. The plan is to land all this work in the upstream GNOME 2.28 release and most of that work is done already.

Basically, we’ve switched away from one daemon to another and in the process we gained a few new features.

OMG, the disk is dying - using libatasmart

including letting the user inspect what his disk is trying to tell him. Kudos goes to Lennart for adding USB support, fixing issues with false positives in libatasmart and just for providing a really good library. And it looks like it’s working.

To really make this 100% useful, we probably want some kind of directory where system vendors shipping GNOME can drop an XML file and then the user can click a button “Order new disk” and get on with their life. Of course, then we need to handle the much harder problem of backup and data migration, but, hey, we want to solve that anyway.

So I also wrote this Palimpsest Disk Utility application

Kitchen sink of disk information

which also handles things like RAID devices and support for managing things like LVM and btrfs is planned as well. We’re also looking at changing the UI to be a bit more sane - something like this perhaps

Clean, leaner, simpler? Tell us in bug #579211!

Tomas, Matthias and I also worked on a simple formatting tool suitable for replacing the venerable gfloppy tool

Formatting tool

you can see the whole series of screenshots to see how the user interaction is supposed to be like.

Finally, I just merged a series of patches to GVfs for using this new machinery

OMG, pretty RAID icon!

Building this on top of a modern storage daemon, as opposed to HAL, means that your file manager (and any other app using GIO), is instantly updated as you repartition or reformat your disks. It even works if you are using command-line tools, etc. fdisk(8) or mkfs(8) - geek comfort FTW!

We might want some extensions to GIO to make working with RAID and encrypted devices (and in the future LVM) easier (such as unlock/lock/start/stop buttons in the file manager sidepane) or maybe we’re just going to rely on the system auto-assembling such things and leave it in Palimpsest. Decisions, decisions.

Another geek-comfort I added, is the ability to show user mountable fstab entries, that’s bug #536292. I was a bit reluctant to do this, I think instead we want native support in GVfs for nfs:// yet still use the kernel filesystem driver though some kind of setuid helper. Then we’d also fix NFS servers to publish the fact they exist using Avahi and then people can discover it in Nautilus’ computer:// window. Just cleaner and easier, I think. Combined with support for bookmarks on the GIO level, this should be a dream come true for people deplying systems in the enterprise.

Most of this work is part of the bigger HAL-ectomy - my good friend Kay has a good post outlining the plans for this.

Getting GVfs and FUSE right

2008-10-01T12:24:00.000-04:00

One of the cool things about the new IO library (GIO) in the GNOME stack is that it’s extensible. One set of extensions for GIO is provided by the GVfs package. The main (but not only) purpose of GVfs is to provide access to Virtual File Systems by having the file system backend (e.g. file system driver) running in a separate daemon process in user space - sort of similar to what the Filesystems in Userspace (FUSE) framework provides on Linux and, nowadays, most UNIX operating systems. GVfs currently includes backends for SFTP, FTP, OBEX for Bluetooth, Digital cameras and MTP devices (through libgphoto2), Compact Disc Audio, Webdav, HTTP and archive files (through libarchive) to name the most important ones.

One key difference between FUSE and GVfs, is that GVfs file system drivers are not constrained by the rather old-fashioned, limited and design-by-committee POSIX API. Instead, GIO provides (and exposes for implementation to file system drivers) a modern and powerful API that provides asynchronous IO, file monitoring (like inotify) and high-level operations as part of the standard API. The GIO API is at the right level of the stack, it’s part of the GLib tarball. This means that any application that today uses GTK+ has access to GIO. This is a pretty big deal, if you depend on GTK+ 2.14 on newer, you’ll also depend on GIO.

However, since it’s rather naive (not to mention arrogant) to think that people are going to be using only GTK+ applications, GVfs also ships with a FUSE daemon. This daemon provides a POSIX interface to the virtual file systems provided by GVfs. All the fancy high-level operations available when using GIO won’t be available (there’s no POSIX equivalent function to map to) of course, but the basic POSIX stuff (e.g. open(2), read(2) etc.) will work just fine. The way it works is that the GVfs FUSE daemon provides a mount in $HOME/.gvfs where each top-level directory represents a GVfs mount:

$ ls -1 $HOME/.gvfs gphoto2 mount on usb%3A004,002 sftp on people.freedesktop.org sftp on quad.local

For example, this means all the good old command line tools still work

$ stat /home/davidz/.gvfs/sftp\ on\ quad.local/home/davidz/.bashrc File: `/home/davidz/.gvfs/sftp on quad.local/home/davidz/.bashrc' Size: 313 Blocks: 0 IO Block: 4096 regular file Device: 14h/20d Inode: 44 Links: 1 Access: (0600/-rw-------) Uid: ( 500/ davidz) Gid: ( 500/ davidz) Access: 2008-10-01 14:35:28.000000000 -0400 Modify: 2008-04-25 11:55:02.000000000 -0400 Change: 2008-04-25 11:55:02.000000000 -0400
$ convert /home/davidz/.gvfs/gphoto2\ mount\ on\ usb%3A004\,002/DCIM/100NIKON/DSCN0001.JPG -dither out.jpg

and so forth. Lots of command-line people like this, at least they always like to complain when we didn’t have this back in the GnomeVFS days.

But it gets better. Today I committed a patch to GIO to ensure that GIO applications launching applications (such as the Nautilus file manager or the Evolution mailer) will always pass a FUSE path instead of the GIO URI. One implication of this is that if you launch a non-GIO application (such as mplayer or a KDE application) from Nautilus, that application will Just Work(tm) even if the file lives on a GVfs share.

I think this is a pretty big deal; now it just doesn’t matter what VFS system of the week is used in the application, it’s all POSIX as far as applications are concerned. Sure, there’s a small performance hit by having to go through the GVfs FUSE daemon (a couple of extra context switches) but for run-of-the-mill desktop applications this is insignificant. Also, as explained here it’s currently not a very smart idea to pass an URI to an application; there’s the problem we don’t have proper standards defining the URI’s we use; there’s also the problem of having to authenticate over and over again.

Finally, if the application itself is using GIO, this patch ensures we map back to a GIO URI and, thus, we bypass the FUSE daemon. This is best illustrated by the following screenshot:

Bastien in VFS heaven

As you can see from the terminal window, Eye of GNOME is being passed a FUSE path from Nautilus. But thanks to the aforementioned patch, this is getting mapped back to a GIO URI as shown in the properties dialog.

This feature won’t be available in GNOME until 2.26 (and GLib 2.20) - we’re concerned some broken applications might be examining the URI before it’s mapped back so we’re reluctant to provide it in GLib 2.18 / GNOME 2.24 for now. FWIW, we’ve been shipping patches for this feature since Fedora 9 and the feature will be in Fedora 10 as well. Other vendors are encouraged to ship these patches as well.

DeviceKit presentations

2008-08-19T08:32:00.000-04:00

Earlier today Richard and I did a DeviceKit (the set of projects replacing HAL) presentation for some other group here at Red Hat. You can see the slides here and here.

Perhaps of interest to the GNOME community, there’s also screenshots of Palimpsest, an upcoming Disk Utility library and application for GNOME. Most of this is already available in Fedora’s development branch, dubbed Rawhide, but won’t be installed by default in Fedora 10. Right now I’m busy with PolicyKit stuff but the plan is definitely to get a gnome-disk-utility mailing list going soon and get this stuff integrated throughout GNOME (I’ve already made sure it’s easy to plug into gvfs for example).

For example, I learned the other day that my disk is failing (actually, as mjg59 pointed out, it’s wrong to use the word FAILING since it’s Old-Age. Easily fixed.). Now, it would definitely be useful to have a notification bubble indicating this. This is pretty trivial to write using the DeviceKit-disks API; just monitor the org.freedesktop.DeviceKit.Disks.Device:drive-smart-is-failing property. Of course the gnome-disk-utility libraries (there’s one at the GLib level and one at the GTK+ level) wraps this nicely. In fact the gnome-disk-utility library at the GTK+ level should probably provide the code for doing this status icon. Something to discuss. Hence why a mailing list and more community involvement in the gnome-disk-utility project is needed.

Anyway, the goal is to port the most of the Fedora desktop to use DeviceKit instead of HAL for the Fedora 11 time frame.

Resolution Independent GTK+

2008-08-06T13:50:00.000-04:00

Got one of them big laptops where you feel tempted to use a looking glass because the pixels are so tiny? Ever feel cheated when you adjust the font size but the rest of the UI looks like crap in comparison? Or maybe ever felt dirty when hard coding pixel values in your application?

there’s also a tiny and a normal version…

Today I sent a patch to gtk-devel-list adding Resolution Independence to the GTK+ toolkit. Let’s see how that goes.

D-Bus introspection

2008-07-30T21:36:00.000-04:00

Sure, there’s already d-feet and a similar Qt tool but for people who hate using the mouse or are allergic to X, I’ve put together this cute little hack that provides bash completion to dbus-send(1). Unfortunately it depends on the XML introspection parser from dbus-glib so it’s not straightforward to add to mainline D-Bus. Could easily go in dbus-glib though. Then again, it would be nice if libdbus included an XML introspection parser. Decisions, decisions.

Linux Plumbers Conference CFP extended!

2008-07-23T09:46:00.000-04:00

I rarely blog these days, doing the Twitter thing instead and all. Anyway. The Call for Papers for the Linux Plumbers Conference in September in lovely Portland, Oregon has been extended until July 31st 2008. It’s a conference about the core infrastructure of Linux systems: the part of the system where userspace and the kernel interface. It’s the first conference where the focus is specifically on getting together the kernel people who work on the userspace interfaces and the userspace people who have to deal with kernel interfaces. It’s supposed to be a place where all the people doing infrastructure work sit down and talk, so that each other understands better what the requirements and needs of the other are, and where we can work towards fixing the major problems we currently have with our lower-level APIs.

I’m running the “Desktop Integration” microconf at Plumbers. I have two goals for the microconf. One is to be informative about what’s going on in the dusty hallways between the Kernel and the Free Desktop. Which I think is important at a place like Plumbers where we’re going to a lot of people working on similar problems present. The other goal is to actually try and make some headway on actual problems that require people from multiple communities working together. Such as some of the work Jon is focusing on, e.g. how to make fast-user-switching, multi-seat and terminal services Just Work(tm).

The conference is expected to sell out pretty quickly so it’s a good idea to go ahead and register for Plumbers Conference instead of procrastinating about doing it!

See you in Portland!

gnome.conf.au 2008

2008-01-28T18:48:00.000-05:00

Am in Melbourne, VIC, Australia for linux.conf.au 2008; just gave a talk on PolicyKit, the slides are here. Only had 20 minutes so the whole thing felt a bit rushed and I didn’t get to go into much detail. Hopefully it made some sense though, otherwise checkout the design and API docs.

Politics. Mudslinging. Chocolate.

2007-11-26T10:32:00.000-05:00

Whether Murray is right or not, I think his recent rant is rather tasteless and I agree some kind of retraction/apology is needed.

FWIW, Murray’s post certainly don’t reflect what I’ve experienced when I started participating in the GNOME project 2003-2004ish. That said, I acknowledge it may reflect what others have experienced. I don’t know. Also, I tend to stay out of politics… at least politics on that level. That said, I do appreciate others doing all this work; for a project of GNOME’s size we do need people in the foundation and some board, we need people spreading the word around GNOME, we do need to participate in standard organizations etc.

Either way, this kind of mudslinging is very unproductive for the project and everyone involved should know better. An old friend of mine once said “Can’t we just focus on the chocolate?”. This was in reaction to stupid in-company fighting and also a pun on a tv commercial from some chocolate company. The commercial involved three chocolate makers. They were making chocolate. Or rather, two of them were ranting about all sorts of non-sense and as a result they didn’t make much chocolate. At some point the third guy just had enough and he started whining. Much like me in this post.

Guys, can we please focus on the chocolate instead of all this non-sense?

Frustrated…

On Xguest, Access Control and Desktop

2007-11-18T16:34:00.000-05:00

Dan, I think the ideas behind Xguest are nice. But your ideas about using SELinux to implement access control for users leaves much to be desired. If you think that a brutal approach of denying individual users to even access services like HAL, NetworkManager and Bluez is good, then please think again. Please realize that if you do this, the desktop experience will suddenly change. Things like networking and bluetooth applets will cease to work. The file manager will fall back to relying on /etc/mtab and /etc/fstab. I’m not sure PulseAudio will start.

The problem is that your approach is not fine grained; it’s all or nothing. And this is not really useful. For example, for NetworkManager what you want is to lock down the machine so the user can only connect to trusted networks. With e.g. trusted networks being defined as something that is defined in a file in /etc that only uid 0 can change.

I’m not sure if it’s a surprise to you, but over the past year or so I’ve been working on and off on an application-level access control mechanism (similar to various RBAC implementations but with a few innovations on top) called PolicyKit that does all this. It’s specifically designed and optimized for desktop applications but can also be useful for legacy UNIX command line apps. It puts the administrator and users back in control since it provides a way for them to gain authorizations by authenticating and also supports systems like Ubuntu that has no root password. It provides a nice user interface for human beings to manage authorizations. There’s GTK+ widgets to make it really simple to integrate into existing applications (and KDE support is underway). The list of applications that are PolicyKit enabled is growing rapidly. It’s in all the major distributions. It encourages least privilege, e.g splitting a program into privileged and non-privileged bits.

It’s not that I hate SELinux because I don’t. I think SELinux is already very useful, just look at how useful SELinux is in confining network-facing and privileged services. I also have SELinux support in PolicyKit already and I plan (with your help) to make gnome-user-share run in a dedicated SELinux context so it is authorized to punch holes in the firewall without any configuration whatsoever (today it’s totally broken since g-u-s picks a random high port number to listen on). But, Dan, frankly, I think your use of SELinux to lock down access on the desktop is misguided. It’s a bit like if all you have is a hammer, then everything looks like a nail. I feel that it’s a bit sad that that two developers, working for the same company, in the same office are trying to solve the same problems. We really ought to do better.

(Disclaimer: this post does not necessarily represent the views of my employer, Red Hat, Inc.)

Update: A few people pointed out that I sounded too confrontational in this post and maybe they’re right. As I tried (and probably failed at) to explain it’s not either/or. Surely, there’s a ton of value in using SELinux to lock down privileged mechanisms like HAL, NM, Bluez and so on. Having PolicyKit and SELinux work well together is in everyone’s best interest.

Coverage

2007-11-12T06:56:00.000-05:00

Andy Oram wrote a nice piece on PolicyKit detailing what it is and some of the plans for the future. I also did a short walk-through recently on one of our staff meetings here at Red Hat, there are some pseudo slides here. On the topic of coverage I’ve recently add unit tests to all the library sources and gcov tells me we’re at 59%. Not too shabby.

Policy, Mechanism and Time zones

2007-09-20T14:08:00.000-04:00

In my last post I looked at D-Bus system bus activation and provided a step-by-step guide on how to use this. In this installment we’re going to look at a specific application of system bus activation; namely how it can be used from a desktop application to do a privileged operation: setting the timezone. This example will also feature usage of PolicyKit for lock down.

When dealing with programs that needs to do privileged operations, the first thing that one needs to do is to separate the mechanism (the piece of code actually doing the privileged bits) into a separate process from the rest of the program. The main program will then simply launch a small privileged helper whenever something privileged needs to happen. This is a good idea because it’s a lot easier to review a tiny privileged helper with minimal dependencies rather than a huge program that pulls in things like GTK+, Python, image loaders and what not. Typically, the privileged helper is written in a way such that it trusts no-one; it verifies all the incoming parameters and also checks whether the program that calls it should be allowed to do so. The latter part is where PolicyKit comes in - PolicyKit is an application-level toolkit that allows a mechanism to ask “is $USER allowed to do $ACTION?”. As such, PolicyKit comes with a set of data structures to help model these constructs.

Anyway, enough boring theory; the user experience we want is that the user can click on his predefined list of time zones to set it; something like this

If the user is not privileged to do this action, we may show a dialog like this

that may (or may not) feature buttons to make the system remember that setting the time zone is OK. Such that the system won’t ask for authentication in the future. In other words, this is a one-time-pain dialog.

Let’s go through the code and see how all this works. First, we need to write a mechanism. This will need to run as uid 0 (e.g. root) and we choose D-Bus as the interface for using the mechanism. We choose a unique name for the service, org.gnome.ClockApplet.Mechanism and since we’d like to write it in C using dbus-glib, we need to write some XML to define the objects and interfaces that this service will expose. This is defined in this file. As evident, this service exposes a single object, /, that exposes a single interface org.gnome.ClockApplet.Mechanism that currently exposes a single method SetTimezone(string zonefile) where the argument to that function is a path to what time zone we want to use. The mechanism itself consists of two C files, available here and here and a single C header file, available here.

Then, exactly as in the last post we need a D-Bus configuration file and a D-Bus service file. There’s also a cheesy Makefile to tie it all together.

Now, with this mechanism in place, we can poke it via D-Bus
$ dbus-send --system --print-reply --dest=org.gnome.ClockApplet.Mechanism / org.gnome.ClockApplet.Mechanism.SetTimezone string:/usr/share/zoneinfo/Europe/Copenhagen method return sender=:1.346 -> dest=:1.345 reply_serial=2
The usage of PolicyKit deserves some explanation. First of all, as stated in the docs, we need to declare one or more actions for our mechanism. As we only export a single action right now (setting the time zone), we declare only that one action via the PolicyKit .policy file here. When we extend the mechanism to e.g. provide functionality to set the time as well (via e.g. a SetTime(int64 secs_since_epoch) method) we can declare an action org.gnome.clockapplet.mechanism.settime in addition to our org.gnome.clockapplet.mechanism.settimezone one.

Now, whenever someone calls into the mechanism, we simply use libpolkit to query whether the caller is allowed to do the specific action:
sender = dbus_g_method_get_sender (context); pk_caller = polkit_caller_new_from_dbus_name (system_bus_connection, sender, &dbus_error); pk_action = polkit_action_new (); polkit_action_set_action_id (pk_action, "org.gnome.clockapplet.mechanism.settimezone"); pk_result = polkit_context_can_caller_do_action (pk_context, pk_action, pk_caller); polkit_caller_unref (pk_caller); polkit_action_unref (pk_action); if (pk_result != POLKIT_RESULT_YES) { /* bail out */ }
The system administrator can control, using the /etc/PolicyKit/PolicyKit.conf configuration file, exactly what users are allowed to do this and whether they are allowed to keep the privilege to do the action on a forever- or per-session basis. This shouldn’t be necessary since it’s possible to specify the default result for a given action. Distributors can tweak the .policy as they see fit; for example consumer-oriented distributions might want to patch this to make the defaults always be “yes” (to avoid the auth dialog entirely) and workstation-oriented distributions might want the defaults to be “auth_admin” (to require the system administrator password and not give a chance to remember the privilege). Finally, sites can override this using the system-wide configuration file, e.g. if I’m a huge site I can distribute a PolicyKit.conf file that allows certain users to always do an action, certain other users never to do an action, and finally a third set of users for whom I want them to enter their own (or the root) password. Again, see the /etc/PolicyKit/PolicyKit.conf configuration file for details.

The screenshots above show integration with the intlclock applet; that’s thanks to Matthias Clasen. I simply handed Matthias the mechanism code including a simple GTK demo application available here. This application simply attempts to call into the mechanism and if the mechanism throws the org.gnome.ClockApplet.Mechanism.NotPrivileged exception it uses PolicyKit-gnome (via the session bus) to prompt the user for authentication if applicable.

The PolicyKit stuff is mostly done; at least the plumbing is there. What I like to see is some more GNOME-ish porcelain to make this stuff even easier to use. First, it’s simply way too complicated to write dbus-glib D-Bus servers. There’s just way too much code; compare to the Python server in the last post. Second, I’d like to provide a small library with a subclassed GtkButton, e.g. PolicyKitGtkButton such that all the magic happens there and it shows e.g. a lock only if authentication will be needed. I’m also planning to do a song and dance at the Boston Summit about this. The list of users of PolicyKit right now includes only HAL. However, PackageKit, dconf and virt-manager are all in the process of picking it up. I’m also hoping NetworkManager and things like gnome-system-monitor will pick it up (for the process killing bits instead of falling back to running the entire UI as uid 0 just for this). With time, and not too much time, I’m hoping for PolicyKit to be a blessed dependency in GNOME and banish the idea of su/sudo wrappers to enable running X11 applications as uid 0.

System Bus Activation hits Fedora Rawhide

2007-08-01T22:00:00.000-04:00

.. well, almost. Anyway, you can get packages here or wait until Rawhide opens again (it’s currently frozen for F8t1). Credit goes to Richard for finishing this work and getting it past Havoc. This is a really huge thing; it opens up a lot of possibilities for better desktop/OS integration.

Lots of pretty colors

To actually test the packages I wrote a simple test program and thought it might be useful to share it. First, you need a program that claims a name on the system bus. I just copied the one from the dbus-python repository and modified it slightly. You can see it here.

Second, as with any service on the system message bus (as opposed to the session bus which has different security characteristics) you need a file in /etc/dbus-1/system.d. Mine is named dk.fubar.Test.conf (but note that any unique file name ending in .conf will do) and is pretty much run-of-the-mill standard stuff and shouldn’t be a surprise if you’re familiar with services on the system bus. It looks like this.

Finally, for system bus activation to work you simply need to put a service file in /usr/share/dbus-1/system-services. Mine is named dk.fubar.Test.service (again, any unique name ending in .service will do) and looks like this:
[D-BUS Service] Name=dk.fubar.Test Exec=/usr/sbin/dbus-sysbus-test.py User=root
and that’s it! Provided you installed everything correctly (there’s a cheesy Makefile to do this) things should Just Work(tm) and the following command should print
$ dbus-send --system --print-reply --dest=dk.fubar.Test /SomeObject dk.fubar.Test.SampleInterface.HelloWorld string:foo method return sender=:1.517 -> dest=:1.516 reply_serial=2 array [ string "Hello" string " from example-service.py" string "with unique name" string ":1.517" ]
If your service is capable of operating correctly when running as an unprivileged system user (e.g. ftp or whatever) instead of root (think least privilege), simply change the user mentioned in both the .service and .conf files.

No more polling

2007-07-19T12:57:00.000-04:00

I just committed code to hal to take advantage of asynchronous change notification events originating from SATA AN capable drives. This means we can avoid polling the drive and this should result in fewer wakeups in libata. This also opens the opportunity for the drive itself to go into a deeper sleep state when it’s not used. So all in all this should improve laptop battery life. I haven’t tried this on a laptop just yet, but when I find a suitable one I’m sure PowerTOP will tell me.

I took this while testing this GPS receiver. Click to see where to get good coffee.

Credit goes to Kristen Carlson Accardi from Intel. She did the kernel bits, sent me a SATA AN capable drive and pointed me in the right direction. To try this out, you will need a recent kernel and also some patches not yet in Linus’ tree. See RH #204969 for details. I’m hoping this will make it into Fedora 8; the hal bits will for sure; for the kernel bits I suppose asking davej is in order.

PolicyKit release

2007-06-20T09:04:00.000-04:00

Late last night I finally made a PolicyKit release. It’s almost ready for prime time; wait a month for 0.4 to get out and I should be able to maintain a stable API.

PolicyKit in action

With Richard’s great work on D-Bus system bus activation it looks like we’re soon going to have nice upstream frameworks for enabling the desktop to do privileged operations in a hopefully sane way. Instead of having each and every distribution on the planet write their own configuration tools. I, for one, really wants to see some effort of merging such things into upstream GNOME and KDE as far as possible.

Interwebs

2007-06-05T16:58:00.000-04:00

Normally I’m not opposed to what our new web overlords are serving us; you know; social networks and the ability to see what past and present, uh, peers are up to… but sometimes it’s just TMI - Maybe I just need to use the Interwebs less

i’m in ur RAID, striping ur disks all the way to CA

2007-05-07T12:30:00.000-04:00

Recently I acquired a pair of 500GB USB2 hard disks that I use in a RAID-1 configuration (on two different host controllers!); you know, to store my photos and other important, uh, stuff. I had some spare time this weekend to look at fd.o #6426 so I wrote a small patch to teach HAL about Linux software RAID.

RAID rebuilding

I also whipped up patch to gnome-device-manager such as to show the new properties introduced including whether an array is running in degraded mode or is being rebuilt. It’s pretty spiffy.

All that this code is doing right now is basically just listening to kernel events (via udev and priority data on /proc/mdstat) and then representing the RAID arrays as HAL drives and volumes. It’s spiffy because the RAID drive inherits things like storage.hotpluggable from the physical components (e.g. the two USB hard disks) so in my case the RAID array simply appears in Nautilus and is totally mountable just like a regular external drive. Works out of the box. Desktop integration++. However, we can do a bit better so for GNOME 2.20 I’ll be writing some patches to gnome-vfs and gvfs. You know, to use the new HAL properties to choose some helpful icons and icon captions.

Going forward, there needs to be some way of assembling and disassembling RAID arrays; hot-adding components to degraded arrays and so on and so on. Probably neither Nautilus nor gnome-device-manager is the right approach here. And there will probably be some sort of auto-assembly happening at the system-wide level anyway, need to sort that out with Kay (the udev maintainer) and also talk to various distro maintainers too. It’s also probably also time to revive my never properly announced GNOME Disk Utility project. Actually, that’s one reason that I split the Device Manager into a shell and library component; right now the shell is around 1200 lines (compared to 4000+ for the library) and there’s no reason that the Disk Utility shell would be a lot bigger. Plus it’s always awesome to put pretty UI on top of useful infrastructure.

Tomorrow I’m leaving for The Golden State to go the Red Hat summit, then hang out with Kay Sievers and finish off the west-coast invasion with FreedomHEC. The next two weeks are going to be busy.

Battery Recall^WExchange

2007-04-28T09:24:00.000-04:00

Just found, via the interwebs, that the battery in my 15″ Macbook Pro was recalled^Weligible for exchange almost a year ago although not for safety risks, merely because “… batteries supplied to Apple do not meet our high standards for battery performance.” [sic]. So that’s something. Anyway, since this is useful information, I wanted to add information to hal-info such that other Macbook Pro users would benefit as well. By getting spammed with a popup like this

Useful feature in g-p-m

But as it turns out this data is not easily available

$ cat /proc/acpi/battery/BAT0/info present: yes design capacity: 55000 mWh last full capacity: 47150 mWh battery technology: rechargeable design voltage: 10800 mV design capacity warning: 250 mWh design capacity low: 100 mWh capacity granularity 1: 10 mWh capacity granularity 2: 10 mWh model number: ASMB012 serial number: battery type: LION012 OEM info: SMPN012

meaning it’s hard to match on anything sensible from the fdi file. Maybe this is because it’s a Smart Battery? FWIW, I can see that the sbs driver is loaded. Lazyweb, please help - as comments on my blog are busted, please provide answers personally to me or, preferably, the hal list (requires subscription).

Btw, I couldn’t find this information by poking around in Mac OS X either… but it must be there; I mean, normal non-smart batteries record make, model and serial numbers just fine via ACPI. That’s what we use to tag other recalled battery units. So I’m sure it’s possible to get at. And if Apple wanted, they could easily have similar mechanisms for displaying notifications similar to the ones we have in GNOME. But I can understand it would hurt their bottom line. I guess.. this is one interesting aspect of getting your OS from the same vendor as where you get your hardware.

Anyway, I filled out the form and a new battery is on it’s way. I guess, uh, thanks Apple. Just wish you had notified me earlier and I wouldn’t have to find out this way .

Update: I suppose this isn’t technically a Product Recall; Apple specifically uses the word Exchange instead. I guess, from a legal point of view at least, there’s a difference. To me, as a consumer, it’s doesn’t make huge difference however… hmm.. maybe we need to add a new property info.is_eligible_for_exchange to the HAL spec to avoid using the word “recall” at all in such situations. I mean, to, uh, cover our collective asses from angry hardware vendors.

Native languages and poisonous people

2007-04-27T13:19:00.000-04:00

I think, over the past 3.5 years I’ve been working on free software, I have sent more than 5,000 mails, all in English, to various mailing lists, most of them with more than a few hundred subscribers. It’s always a bit weird (even after living in Massachusetts for more than 2.5 years) to use your second language and I believe it will stay that way the rest of my life. Which is fine and all.

Sketchy Dane

In general, I’ve found that native English speakers are very nice to non-native English speakers. In stark contrast, for example, are natives of my own country Denmark; whenever a foreigner attempts to speak Danish everything becomes a bit weird. One common thing is that the native Danes try to switch to conversation to English because it makes them less uncomfortable. I guess there are many reasons for this difference of behavior; first, Danish is a small, insignificant (and dying) language, most people speak English anyway and not a lot of native English speakers have a second language at all. But it’s also a cultural thing. In particular, I find that it’s very easy to live among the Americans; they’re friendly (actually, I jokingly tend to say “it’s like living with the smurfs”; many, not all though, people here are just more optimistic about the future than in Denmark and the EU.). The fact that English is a second language have never been an issue for me at all.

Sketchy German

Until today. I guess there’s a first time for everything, the topic of my native language came up on the Fedora live cd mailing list just recently. Response here. Time to watch How Open Source Projects Survive Poisonous People (And You Can Too) again I suppose.

The whole thing just makes me a little sad inside.

hal 0.5.9

2007-04-02T21:47:00.000-04:00

I just released hal 0.5.9. Lots of changes including dramatically reduced resource usage, Solaris and FreeBSD support, ACL management, ConsoleKit support and many other things. Get it while it’s, uhm, hot. Hopefully the next release won’t take seven months…

Stupid meme

2007-04-02T11:29:00.000-04:00

Gah!

2007-03-12T13:45:00.000-04:00

Just received this in the mail today from Europcar Australia:

As the registered owner of the vehicle stated below we have received a Speeding Infringement Notice and, upon searching our records, we can confirm that the infringement was incurred during your rental period.

Guess I should have looked out for speed cameras.

Random photo plug: Darling Harbour in January

I’m slowly losing my patience with the gods regarding distribution of luck and fortune.

Who Killed The Electric Car?

2007-02-24T16:16:00.000-05:00

Yay, just found out that Who Killed The Electric Car? is available on-demand on Comcast. It’s pretty awesome so far. Update: favorite quote “they’ll make me drive a small car, make me be cold in my own home… in other words, make me live like a European.”. My gods, that’s awesome .

I miss the summer (photo from Sydney, January 2007)

Weather here in MA continues to suck although my relatives in Denmark says it’s much worse there.

Sydney

2007-01-13T14:44:00.000-05:00

Barely back in Boston (UTC -0500) after the xmas break in Denmark (UTC +0100), I’m now in Sydney (UTC +1100) for linux.conf.au 2007. I don’t feel jetlagged though.

Coogee, NSW, Australia (not my bottles)

Managed to sleep a lot on my 24-hour journey; I was fortunate enough to be traveling with blizzard. We even got picked up at the airport; I suppose Chris’ description of us to the organizers was somewhat accurate

Just look for crazy man and fisheye boy:

http://www.flickr.com/photos/christopherblizzard/306028359/
http://www.flickr.com/photos/davidz/235769935/

(Together, we fight crime!)

Now to check out of my hotel and find the campus of UNSW. Times are good.

Fast User Switching

2006-11-17T09:50:00.000-05:00

I’ve just put up some notes about Fast User Switching and how we plan to integrate this into Fedora for our next release.

Most of this work builds upon ConsoleKit by the ever-so-awesome Jon McCann of gnome-screensaver fame. It’s not very Fedora centric so hopefully other Linux distributions (and other free UNIX-like systems like OpenSolaris and FreeBSD) will pick it up too. I’m excited, in particular, about the fact that, with infrastructure like this, we’ll be able to enhance applications to behave intelligently when f-u-s is used, e.g. pause media playback, go to away mode for IM etc.

Don’t forget the beer tonight

Going to pick up Kay later in the PM, he’s flying in from Hamburg. We’ll most likely end up at Flat Top’s later tonight for Dave’s shindig!

A few days ago Artem merged the Solaris backend for HAL. Awesome. The FreeBSD backend is going to be merged real soon now as well, the patch is already on the list.

D-Bus 1.0 release party

2006-11-16T09:39:00.000-05:00

Taken at the D-Bus 1.0 release party

Trying to be Constructive

2006-11-09T22:23:00.000-05:00

Daniel, actually I wasn’t referring to binary drivers per se in that particulary obnoxious post of mine; what I had in mind was primarily out-of-tree drivers / features including ones that explicitly disable rootkit prevention features for example. Using the word damaging to the free software community however was perhaps over the top, sorry it upset you and probably others, but I just don’t think including a bunch of not-so-heavily-audited code (e.g. a driver that is not in mainline) or code that disables security features is doing your users service. Most users of course don’t care, but developers like you and I should. In particular, OS vendors needs to care.

It seems you and I don’t disagree on Linux based distributions including proprietary drivers but I think that one thing you might not realize is how damaging it is when a popular distro starts doing this… years of work in dealing with certain vendors to open up code is undone in a jiffy. I don’t want to be all moody here, trying to be constructive (could use much stronger words but that would serve no purpose), but it’s flat out depressing when stuff like this happens.

With regards to you commenting on the Fedora community, most of your points, I think, are well understood by the community and some are being dealt with. For the record (and this should be clear to anyone involved in the areas I also participate in), I think of myself much more as an upstream guy rather than a Fedora guy as I think it’s silly we have all thesse differences between distributions - just do the work upstream (swim upstream dammit ) instead of adding distro-specific non-sense unless it’s absolutely necessary (things like branding, defauls etc. all fall under this). I think you know this already, so please don’t tag me as a Fedora guy just because I’m ranting about other distributions on my personal blog space. I happen to use Fedora because it’s aligned with my personal views about free software, because it’s a first class distro with cutting edge features… and also, of course, because I’m paid to work on the software by my employer.

introspection, bridges and people driving by

The larger problem, in my view, is really that we have some work left in building bridges between the various communities that participate in writing code for the GNU/Linux system. Heck, even inside a company like the one I work for it’s sometimes difficult to coordinate features between kernel, base os and desktop teams, sometimes even more difficult that dealing with other downstreams. However with collaboration zones like freedesktop.org we’re slowly getting there and things do look a bit better than they did just two years ago - next up, for example, is pm-utils so distros can start sharing quirks and infrastructure to improve the power mangement mess.

I hope this clarifies.

(Disclaimer: I am employed by Red Hat and am paid to work on (among other things) Fedora but my views are strictly my own.)

(Oh, and I also think it’s called “The New Pink” instead of “The New Black” but, uh, maybe saying that just reveals I’m eligible for membership in the GNOME old farts club.)

Software Freedom and Dealing with Resume Failures

2006-11-09T11:48:00.000-05:00

Go read davej’s thoughts on Why Fedora isn’t Ubuntu. Required reading for people who believe in free software and thinks shipping insecure drivers is a bad idea. Choice quote: Maybe we can get a spaceman with lots of money to pay for the subsequent legal costs too.

Someone here at Red Hat was unhappy about a review of early RHEL5 code where Suspend to RAM didn’t work for the reviewer. Though I’m of the firm opinion that the root problem should just be fixed, reality is that it won’t get fixed anytime soon. So I ended up hacking up something so if suspend/hibernate fails… the user is presented with this on his next login:

Problem resuming, buddy!

This (somewhat useless) dialog is just the beginning; the long term upstream plan is to hook this up to Bug Buddy. Determining when to show this dialog is actually really difficult because the most common problem is that the system resumes just fine but the video isn’t turned on. And there is no chance of detecting such a failure programmatically, hence we need to be smart about it; find patches and more information here. I expect this to land in Rawhide pretty soon.

Am still in RHEL5 bug fixing mode; at least there’s light at the end of the tunnel so I can return to doing upstream work.

Can Mr. Smith Get to Washington Anymore?

2006-11-04T07:49:00.000-05:00

Saw Can Mr. Smith Get to Washington Anymore? at the Kendall Square Cinema last night (review here). Movie itself was good. What was pretty awesome was that after the movie there was a Q and A session featuring Jeff Smith himself where we could ask questions about both the movie, the grass root stuff and his current political engagement.

If you have the chance run, don’t walk, down to the cinema and see the movie.

Random Stuff and Random Flames

2006-10-20T14:06:00.000-04:00

Looking at myself or looking at you?

Just booked tickets for LCA 2007 in Sydney, Australia in January 2007. Am also taking some vacation to bum around .au the week following the conference. Am also planning to submit a proposal for the GNOME miniconf.

Sad to see yet another defector. Here’s a very insightful comment from davej about some of the damage done to the free software community.

Less-than-insightful rant about “crappy HAL” that frankly pisses me off.

Spent around 3-4 days optimizing HAL for CPU and memory consumption, here are the difference it makes on the 770, courtesy of Rob Taylor

Am almost done with a huge freakin patch for activation on the system message bus. Will need to write down how this relates to some of the init replacement projects out there; here are some thoughts about that for now, but it really deserves a full blog entry.

With the D-Bus bits (system bus activation) in place, I can move forward and do some PolicyKit stuff that requires the D-Bus bits in order to be useful.

Trying to spend lots of time on work-stuff since personal life is a bit busted at the moment.

Embedded

2006-10-06T07:06:00.000-04:00

At the Embedded summit at MIT Media Lab

Drinks, Software and Gambling

2006-09-11T18:44:00.000-04:00

How to make a David Zeuthen

Ingredients:
5 parts friendliness
1 part humour
5 parts energy

Method:
Add to a cocktail shaker and mix vigorously. Add a little fitness if desired!

Username:

Personality cocktail
From Go-Quiz.com

In other news I just released HAL 0.5.8 “The Skynet Funding Bill is passed.”. Get it while it’s hot!

Only in Vegas!

The girl and I went to Vegas over the Labour day weekend. I’ve uploaded some photos here. I was lucky at the roulette table too (put some $15 at my age, yeay!), got home with $700 more in the pocket.

Sweet Irony

2006-07-26T11:56:00.000-04:00

I’ve just released HAL 0.5.7.1 “Unmaintained piece of crap” and all vendors should upgrade as Linux kernels later than 2.6.17 will break HAL 0.5.7 (e.g. HAL will break in hardware detection, not the kernel). It should be a smooth upgrade and credit goes to Kay Sievers for the patches.

Introspection

Just a few hours after my last entry, Jonathan Corbet of LWN found me at OLS and said it wasn’t LWN badmouthing HAL, he was just reporting what was going on at the kernel summit. So, sorry if I made it sound like it was LWN’s fault, it wasn’t, I do believe in free press and LWN is just an awesome resource. Oh, I just find the whole “unmaintained piece of crap” thing slightly amusing (hence the release name for 0.5.7.1) - the community around HAL is alive and well even though I’m not presently around doing HAL stuff as much as I like.

Linux Symposium

2006-07-20T09:48:00.000-04:00

I appear to be in Ontario, Canada. More specifically I’m in Ottawa though hours last night was spent at a bar in Quebec.

In Logan’s terminal 3 enroute to OLS

Yesterday, Kay and I gave a talk on Dynamic Device Handling on the Modern Desktop and I think it was well received. Yeay!

Oh, and I’ve even got to say that hal-device-manager (the small python GTK+ app for viewing the hal device objects and their properties) was an “unmaintained piece of crap” alluding in a funny way to the recent LWN article badmouthing HAL. Some people in the audience laughed. Lots of excellent questions too. Good times.

I’ve uploaded the slides from mine and Kay’s talk here. Enjoy.

PulseAudio and GNOME

2006-07-13T08:59:00.000-04:00

Talked briefly to Monty yesterday about PulseAudio and how it would fit into GNOME. Here are some notes, I may be way off, but I thought it would be good to blog them anyway.

Monty making an elaborate point about mixing stuff

Multiple Soundcard Configuration: The gstreamer sink for PulseAudio should use the same gconf settings as for the gstreamer-hal-gconf sink as discussed here. Then we would be able to reuse the same dialog which would be nice. I have no idea whether this is feasible, desirable or possible in the context of PulseAudio.. But I thought I’d mentioned it anyway, it makes sense to me and… IIRC, a number of applications such as Totem and Ekiga is moving to selecting outputs (using gstreamer) based on whether they output “Sound Events”, “Music or Movies” or “Audio/Video conferencing”.
Power Savings: To be honest, I’m not sure whether PulseAudio runs as a system daemon or in the desktop session or whether it’s configurable (sorry, haven’t done my homework). But surely, to me, it needs to run in the desktop session. It should talk to gnome-power-manager, via D-BUS, and keep track of whether the user “prefers power savings over performance” (yes, this is a user preference which defaults to on when running on battery and off when on AC).
So if “prefers power savings over performance” is enabled as a desktop session wide setting (or if it changes state - it’s a live variable so need to listen to D-BUS signals from g-p-m), PulseAudio should close the file descriptor to files in /dev/audio when there is silence so the kernel drivers can put the sound card(s) to sleep. Not only that, if the sound card is used only for output it should open with O_RDONLY so parts of the sound card can be powered down. Specifically for projects such as OLPC this is important and Jaya Kumar already done the kernel side of this.
Sure, when the sound card is turned on again there is a slight pop, but users for which this is the end of the world… they can tweak the setting in g-p-m or perhaps use a specific PulseAudio setting to override. But by default, PulseAudio should ask g-p-m so we can suspend sound cards on battery when not used. It’s about saving energy.. think of the kids, Der Grüne Punkt, EnergyStar and all that!

In other news, Mitra got a Macbook and she seems thrilled about it so far. Also, I didn’t know that Mitra’s mom have been in outer space.

GUADEC recap

2006-07-06T12:46:00.000-04:00

Got back from GUADEC and Spain on Sunday. Nice conference; I liked the touch with seven days instead of three even though it was a bit exhausting in the end. I’ve also got sick around Saturday so spent that day in bed in my hotel room before traveling back Sunday. As a result of all this, I’m way behind on mail so to those who I owe a reply, hang in there, I’ll get back to you eventually.

Robert and Joey as batshit insane movie stars?

I’ve now uploaded the slides from my talk (ODP, PDF).

Taking a break from the action at the Fluendo party

I enjoyed the 4th of July fireworks from my balcony in Somerville, MA overlooking Boston

Zeppelin just after dropping it’s payload on Boston, MA?

All my GUADEC pictures are available here.

At GUADEC

2006-06-26T04:27:00.000-04:00

Made it to Spain on Friday, surprisingly Delta brought me there at 7:45am as scheduled. Spent Friday walking around Barcelona with Kay and his girlfriend. Nice but a bit exhausting given we were out until 8pm. Made it to Vilanova late Friday night and, hours later, was surrounded by nice people and cold beer. Good times. Fell asleep around 1-2am.

I can’t believe noone else at GUADEC blogged about this guy at the registration desk:

“Pick me up”

I haven’t blogged for quite some time, should get back in the game. Despite that lots of things have happened in work, HAL, PolicyKit and also RL that I will need to report at some time. Now to get some lunch and see some talks!

System Configuration

2006-04-02T22:10:00.000-04:00

So once again the topic of system configuration and Elektra (aka LinuxRegistry) has come up on fedora-devel-list and since I know most people even on the list don’t keep up because of the sheer volume, I’m going to use my blog to link to my response.

Denmark vs. Japan at WCC 2006 - EOS 20D and 70-200mm f/2.8 lens @ ISO 400

It shouldn’t be a surprise to people knowing me that I consider myself a UNIX hater. In fact, this whole thread on f-d-l made me realize that the with regards to system configuration you really only want two things: session-wide and site-wide configuration. Maybe it’s time to open a CafePress store and pimp shirts and stickers with the mottos a’la # rm -rf /etc. Seriously.

Received my MacBook Pro last week and I’m psyched. Now to get Fedora installed .

System-wide Lockdown and Usability

2006-03-07T22:50:00.000-05:00

I wrote some time ago about how most modern Linux distributions deals with with granting unprivileged users additional privileges when they are logged in at the console. Stuff like allowing them to access removable disks, power down the system and so forth - Project Utopia stuff, that whole story.

While this is great for the home user, stuff Just Works, it’s not always ideal in more controlled environments such as call centers, university settings, Internet Cafes or print kiosks. Also, working for a distributor I’m also sometimes personally on the receiving end of rude people who wants this too. I don’t disagree much really. Yes, it makes sense to easily lock down what the operating system allows a user in front of it to do. Even to the point where it is totally unusable. Notably this is required for EAL style certifications of systems. Yes, Sarbanes-Oxley requires it too, and yes, some companies (search for the word ‘epoxy’ in that link) disable the USB ports of their employees computers.

Copenhagen, Denmark - December 2005

I wrote some notes about how to fix this in what I consider a sane way. Sane here includes allowing a system administrator or home user to override this in a way that just works, e.g. pop up a dialog asking for auth to gain privileges to carry out the operation. Mocking around with configuration files in /etc does not qualify - do not pass start, do not collect $200 - put up a GUI dialog instead; it’s 2006 for crying out loud

Anyho, so I did this policy library and started to integrate it with gnome-mount. Sure, this example is kinda lame and the UI is still totally wrong. But it illustrates the point well. The sysadmin didn’t allow the user to override file system permissions on an advanced file system and we give the user the ability to correct it - there are lots of more interesting use-cases than this. And we need do need lockdown, otherwise all the Michal Jaegermann and Alan Cox’s of the world won’t let us put it in a general purpose operating system. Which is fair enough. Oh, before someone cries the root password is evil; whether we ask for the super user password, users own password or something different for auth should just be up to the OS vendor. It’s a boring implementation detail. It’s also fine by me to have a gconf key to hide these dialogs; that might be sometime a sysadmin in an enterprise setting would do. I think Clark even suggested once to put a button “Send request to sysadmin to enable feature” too. Whatever

Paris, France - January 2006

Whilst implementing this I researched various graphical su helpers including consolehelper / usermode that we ship in Fedora. They all suffer from the same set of problems most notably that they allow to run X applications as root. So, I guess for HAL I’ll be rolling my own just like everyone else . Read on please.

No, seriously, I strongly believe the idea of even allowing X applications to run as root is a dead end from day one. With the su helper I plan for PolicyKit (which is a project I’ve spun off HAL containing said policy bits), your application will at most be elevated to an unprivileged “system user”. The trick here is that HAL allows this unprivileged user to invoke D-BUS methods normal unprivileged console users wouldn’t be allowed to invoke. To get to this “system user”, the desktop user will have to auth (root password, own password, whatever) and will thus be presented with a dialog even (optionally) offering the one-time-pain option by granting this privilege to the desktop and/or other users forever (or until the sysadmin changes it).

Boston, MA - December 2005

The reason I’m spinning it off HAL is that it makes sense for the rest of the desktop bits (not only GNOME) to adopt the same architectural pattern; for example gnome-system-monitor could use this to renice processes.. g-s-m will simply provide a service on the system bus that runs as root and exports a method called Renice(). A PolicyKit policy file provides a list of uid and gid for who is allowed for this. Now, g-s-m will try to invoke Renice() but might get the PermissionDeniedByPolicy exception back. If this is the case, simply bring up said dialog, enter password, optionally grant this privilege to one or more users and, bingo, you’re done.

I plan to provide a PolicyKit-gnome library for these dialogs, e.g. password prompting and policy editing. Mostly because this is what I need for the HAL (and Fedora I guess), but I think it makes sense for GNOME too to consider this architectural pattern.

Yea yea, long boring blog post about admin stuff and it’s not even about cool bling stuff!. I hope at least that the admins love me

Secretary plugged hairdryer into UPS

2006-02-28T22:19:00.000-05:00

Yay! My LUKS integration bits landed in Rawhide and will be in Fedora Core 5. In other news, the latest gnome-power-manager in Rawhide works really nicely with non-portable batteries e.g. Uninterruptible Power Supplies that supports the USB HID protocol.

New and l33t icons

Richard been kicking ass on gnome-power-manager recently - note how he even fixed my somehow pedantic requests to even make this work when hotplugging the UPS.

There is no security on this earth, there is only opportunity

2006-02-21T13:33:00.000-05:00

I’ve been hacking on and off with W. Michael Petullo on integrating LUKS into the GNOME desktop via HAL and patches are now upstream.. I think it rocks. I’ve prepared a small screencast:

Screencast of LUKS integration into GNOME

Hopefully we can get these bits into FC5; the patches themselves are pretty simple but you also need HAL and gnome-mount from CVS. I’ll do releases of the latter two later this week.

We still need some UI for setting up volumes; this should probably be integrated with gfloppy or maybe an entirely new UI for partitioning and formatting media. Fortunately, however, someone is already working on this but to do this securely we need to formalize privileges in HAL and I think there is consensus about how we want to do that.

In other news, danish are renamed in Iran. Some times, the world is so silly that it’s sad.

New year and Policy Daemons

2006-01-09T21:10:00.000-05:00

New Year, Same Old

It’s a new year and I returned late last week from Europe - nice with relaxation but maybe a bit too much travelling… Anyway, pictures speak much more than a thousand words, or something, so nuff’ said. I’m not sure I like 2006 yet, maybe things will change.

Danish food

For new years I went to Hamburg, Germany to visit my good friend Kay Sievers - awesome to see Kay again. With Kay working for Novell and me working for Red Hat we talked a lot about the current state of our various distributions and how we can work closer together in the trenches. Specifically we discussed how to manage policy enforcing daemons and below are some of my thoughts on that.

On Enforcing Policy

Background

Starting with Fedora Core 1, Red Hat started shipping the system-wide bus (D-BUS) in Fedora Core, but it wasn’t until Fedora Core 3 with the advent of HAL and NetworkManager that the bus started to see extensive use.

Today, in what will become Fedora Core 5, unpriviliged applications running in a desktop session on the console can invoke methods for putting the system to sleep, configure display brightness on laptops, configure networking and mounting storage devices. With time, more and more methods will be available for configuring the hardware and/or system. All such method invocation happens in a secure way through the (unprivileged) system bus daemon that checks, validates and passes messages between the sender and receiver. Additional lock-down via e.g. SELinux security contexts are possible too but I’ve yet not seen anything use it.

Configuration

Back in the day, the traditional UNIX way for system-wide daemons prescribed using a textual configuration file somewhere in /etc - acpid and networking scripts comes to mind. With this, users had to resort to using a text editor and learn about the configuration file format; very educating and l33t for the geek hacker, but not ideal for ordinary users; not ideal for grandma. When available, UI tools simply rewrote the configuration file which most people agree is error prone and just plain wrong.

Allowing applications in the desktop session to enforce policy makes a lot of sense from a configuration point of view. Today, in the development tree of Fedora we now ship gnome-power-manager. Settings are read from gconf at /apps/gnome-power-manager. We have a nice, not horrible at least, user interface (Desktop -> Preferences -> Power Management) for configuring the policy. It is per-user, and since it uses gconf it’s possible to configure both default and mandatory settings (for lock-down), these days this can even nicely be achieved with e.g. Sabayon. In almost every way possible way it’s superior to textual configuration files. John has some good stuff about gconf here.

Also, gnome-power-manager sports a notification icon along with a menu for performing actions such as putting the system to sleep or hibernation. These days, when the kernel is in a good mood, power management just works. It’s like using my Mac.

However, there are two obvious downsides to running policy enforcing daemons in the desktop session, namely

when no user is logged in, no copy of gnome-power-manager is ever running; hence no policy is maintained; hence your laptop won’t suspend after e.g. 15 minutes of inactivity when at the login screen.

when multiple users are logged in (either via fast-user-switching or via multiple physical consoles) multiple copies of gnome-power-manager are running. This is bad as e.g. inactivity on one console might trigger e.g. a system suspend.

Exactly the same problems apply for the screen saver (gnome-screen-saver), networking (nm-applet), storage management (gnome-volume-manager) and probably other things too. Wonder why there is no screen saver on the login screen? Wonder why your wireless network isn’t configured before login? That’s why.

The Opera House in Copenhagen, Denmark

A framework

To solve problems 1) and 2) above we first need some infrastructure for tracking what sessions are at the consoles attached to the system. Remember that several users can be logged in at the same console through fast user switching (even multiple sessions from the same user on one console) so we need a notion of who and what is active at each console.

The proposal is a simple D-BUS service on the system bus; in quick and dirty pseudo code it would look like this

service org.freedesktop.DBus.ConsoleTracker { methods: array{tupple{string sessionId,int console, bool isActive}} GetSessions(); int GetNumberOfConsoles(); int GetUnixUidFromSessionID(string sessionId); signals: SessionBegin(string sessionId, int console); SessionEnd(string sessionId, int console); SessionActivityChanged(string sessionId, int console, bool isActive); NumberOfConsolesChanged(int numConsoles); }
The sessionId is simply a unique identifier maintained by the ConsoleTracker service - most likely this is the X11 display number though ConsoleTracker shouldn’t rely on a certain windowing system. The isActive boolean denotes where the session is currently visible on the console. Along the way the ConsoleTracker service may also provide methods such as GetSessionBusAddress(string sessionId), it might provide properties to tell what kind of session is running (GNOME, KDE, …) but that is beyond the scope of this write-up.

A service like this (I’m sure the choice of names and methods / signals can be much improved) simply tracks sessions and, as such users logging in and out. Ideally this service is shipped with the D-BUS tarball and ideally we’d make the system bus dependent on this service insofar that we can introduce policy directives such as “at_active_console” etc. in addition to the “at_console” directive we already have.

The ConsoleTracker service should also be possible to implement without pam_console which means that we can get rid of the optional pam_console dependency in D-BUS. This also means that e.g. HAL and NetworkManager can ship D-BUS configuration files with “at_console” without the Debian people having to patch things out and only users logged in at the console may invoke methods like e.g. Suspend() to put the system to sleep. Of course, the Debian people may still change the policy because they think group membership is more correct. It’s their OS distribution after all.

Rupert spotted in Germany

Now, with the ability to track logged in users and sessions, it becomes almost trivial to write the next system-wide service, let’s call it org.freedesktop.PolicyManager. This is a daemon that reads configuration files in /etc/PolicyManager.d/ (only programs will dump files here, not people)
/etc/PolicyManager.d/power-management.conf storage-management.conf ...

where e.g. power-management.conf contains
[policy-service] exec=/usr/bin/my-power-manager-when-no-one-is-logged-in id=PowerManagement per_console=false

which simply states that the PolicyManager daemon should run the program /usr/bin/my-power-manager-when-no-one-is-logged-in when no user is logged in (the console number is passed in the environment). This configuration file also provides a unique identifier for the kind of policy service it is, in this case power management. Part of the “specification” for the PolicyManager daemon contains a list of well-defined policy services, e.g. PowerManagement, StorageManagement, ScreenSaver, NetworkManagement and so forth.

My sister has a Pony called Conny

The per_console field requires further discussion, but let us first take a look at the D-BUS interface offered by the PolicyManager daemon:

service org.freedesktop.PolicyManager { methods: bool ClaimPolicyService(string id); bool ReleasePolicyService(string id); signal: PolicyServiceClaimed(string id, string sessionId, int console); PolicyServiceReleased(string id, string sessionId, int console); }

The thinking here is that when I login to the GNOME desktop, then gnome-power-manager is started and it invokes ClaimPolicyService("PowerManagement") on the PolicyManager system service. Since D-BUS now sports the ConsoleTracker service, the PolicyManager can determine what console, if any, the method call from the gnome-power-manager instance stems from. Specifically, if the method invocation stems from a process not at the console it is denied.

The PolicyManager service now kills the existing process responsible for power management, e.g. /usr/bin/my-power-manager-when-no-one-is-logged-in, and returns TRUE which means that our copy of gnome-power-manager is set to go. This daemon starts enforcing policy, e.g. suspends my laptop after inactivity and so forth.

Whenever my copy of gnome-power-manager terminates (normally when logging out), then the PolicyManager is notified through a signal on the system message bus (already part of the system bus) and starts up /usr/bin/my-power-manager-when-no-one-is-logged-in to enforce policy. Also, at any point my gnome-power-manager copy may invoke ReleasePolicyService("PowerManagement") if the user e.g. manually disables it (though it is unlikely why g-p-m would provide such functionality).

Suppose that user A is logged in at console 0, it’s session "s100", and has a copy of gnome-power-manager runnning. Now suppose that user B logs in via fast user switching at console 0, session "s101". The following happens.

First, the ConsoleTracker service emits a signal telling that session "s100" is no longer active, SessionActivityChanged("s100", 0, FALSE), and the gnome-power-manager copy in session "s100" receives this signal and stops enforcing policy (it doesn’t invoke ReleasePolicyService() though). Now another copy of gnome-power-manager for user B starts up in session "s101". It also invokes ClaimPolicyService("PowerManagement") on PolicyManager and it recieves TRUE and this copy starts enforcing policy.

When switching back to session "s100", the g-p-m copy in session "s101" stops enforcing policy, due to SessionActivityChanged("s101", 0, FALSE), and the g-p-m copy for session "s100" starts enforcing policy in response to the SessionActivityChanged("s100", 0, TRUE) signal.

Frightening Sky

The per_console field, which is FALSE in the power management example, defines whether the policy enforcing daemon is per console or per system.

For example, consider a system with two consoles, 0 and 1. Since per_console is FALSE for power management only one copy of /usr/bin/my-power-manager-when-no-one-is-logged-in is launched. When there are two user sessions, one on each console, only one of them get to enforce policy, e.g. ClaimPolicyService("PowerManagement") will fail for one of the two copies of gnome-power-manager. The loser will probably revert to only showing battery information, not enforcing policy, and it is up to the two copies to negotiate when to suspend the system. This may be done in a number of ways.

So when is per_console=TRUE useful? It’s useful for policy enforcing pieces like the screen saver or the storage manager. In general, it’s useful when you want a copy running for each console. For example, when PolicyManager starts up on a system with two consoles, it will launch two copies of the /usr/bin/my-screen-saver-when-no-one-is-logged-in (how it gets access to the X display is another matter though). Specifically, for storage management, e.g. gnome-volume-manager, one also wants to ensure that policy is only enforced for drives local to the console, e.g. only mount USB storage for the USB ports local to the console. How this is determined is beyond the scope of this write-up for ConsoleTracker and PolicyManager but it’s not difficult to imagine that this information could come from HAL.

Reusing desktop policy daemons for system use

One problem remains to be solved. How do you configure the policy for the policy daemon when no user is logged in? Also, why would you want to maintain two code bases; one for desktop case and one for the case when no use is logged in? The answer here is simply to reuse the desktop policy daemon. For instance, for power management we get

[policy-service] exec=/usr/bin/gnome-power-manager --no-one-is-logged-in id=PowerManagement per_console=false

and gnome-power-manager simply don’t provide any UI (or maybe it paints the UI on top of the login screen, e.g. gdm). Since this copy of gnome-power-manager is invoked by the PolicyManager daemon it runs as nobody or some other unprivileged system user. It still reads it’s setting from gconf but since this is not a real user it will get the default settings.

With this approach the UI for gnome-power-manager may include a button “Set as system default” that simply copies the settings from the logged in users gconf schema to the system default ones in /etc/gconf/gconf.xml.defaults. This of course requires administrative privileges and we may even hide this button if we can detect whether the user is an admin or not. Notably this will also work for other policy enforcing daemons, e.g. NetworkManager’s nm-applet, the GNOME screen saver and so forth.

Confetti is Everywhere

Not tied to one desktop

I’ve used GNOME as an example above, mostly because I’m a GNOME user and developer myself, but nothing prevents KDE or XFCE to do the same. Indeed, if there is agreement on all desktops using the ConsoleTracker and PolicyManager things will “just work”. Of course, someone will have to write this stuff , I’m just rambling about one possible framework.

This here’s a recipe for unpleasantness

2005-12-09T13:23:00.000-05:00

We had snow today - good thing I was working from home

How the world really is

Sunset

I fought the mountain and the mountain won

2005-12-04T18:12:00.000-05:00

Today, Kristian and I drove up to Killington, VT to go skiing. It was my first time out on sloped terrain, let alone the first time in 15-20 years wearing a set of skiis.

I fought the mountain and the mountain won

After some moderate success on the learning slope, Kristian convinced me to try the green slope pictured above (or was it blue? Supposed to be an easy one either way). I was told that I looked rather freaked out after safely returning to the base camp but that it was somewhat decent attempt; go go go! Heh, things take time I guess.

Either way, it was awesome and I’m definitely going to go again soon. Especially as I shelled out quite an amount of money for skiing clothes but ended up renting the skiis, boots and other stuff.

Bokeh Galore

I also brought my camera and a set of lenses (the 10-22, 24-70, 70-200 and 2x extender) though, since the weather was quite bad, I only used my recently acquired 50mm f/1.4 - the bokeh on that one is awesome, though it’s a bit difficult to focus correctly although that is a known issue.

Last few months in review

2005-09-21T19:58:00.000-04:00

I realized today that I haven’t blogged in almost three months.

Conferences

In July, I went to the Desktop Developers Conference and then the Ottawa Linux Symposium after that. One busy week where I shared a room with Kay Sievers, good pal and now proud maintainer of udev (congrats Kay!). It was my first time going to one of the DDC and OLS conferences, and it was good to meet both old and new friends. Pictures here. Next up is the GNOME Boston Summit.

Hackers

Visit

An old friend Søren, from Denmark, was in California so he stopped over in Boston for a visit in the start of September. We drove down to NYC, had a good Saturday night out clubbing with friends, hung out in the city on the Sunday. Labour-day Monday, we headed south for Washington DC for another 2.5 days. Pictures here.

At the corner of 33th street and 7nd Ave

Photography

I’ve upgraded to the Canon 20D and also acquired my first L lens, the 70-200 f/2.8L zoom lens. It’s awesome. Coupled with the 2x II extender I grabbed a picture of a full moon the day before yesterday, see below. For candid shots and low-light-photography (good for capturing people that has issues with fire alarms) I still swear to my copy of the Sigma 28mm f1.8, it’s pretty good, though I do admit that I miss a zoom and eventually I’d like something like this instead .

Full moon

Mass Drivers License

I passed the written test last week and the road test yesterday, so now I’m licensed to drive here in the Commonwealth of Massachusetts and, for that matter, in the rest of the US. Compared to the tests I had to take in Denmark, some 11.5 years ago, this was a piece of cake. It is pretty fortunate that I finally got this done, as after one year in the US my Danish license will stop working and that is nine days from now. Btw, thanks a million to J5 for being my sponsor.

Work

Been busy with new projects. I feel a little bit sad that this means that I’ve been slow to reply on mail on the HAL list, but this is also because life have been somewhat busy and trying lately. Fortunately Danny Kukawka and Richard Hughes have been helpful looking after the project. Also happy to see that GNOME Power Manager will soon enter Rawhide. I’ve been using g-p-m for the past few weeks and it makes the whole laptop experience better.

This is where mechanical excellence and one-thousand four-hundred horsepower pays off

2005-07-06T19:39:00.000-04:00

California
Impulse struck me last week and Wednesday I booked a quick getaway on site59.com - a site that Seth told me about. I got an awesome deal; highly recommended.

California Route One

I left for sunny California (wasn’t that sunny though) very early Saturday and arrived back in rainy Boston (very rainy when we landed) very early Wednesday. I didn’t really have any specific plans for the trip so things just happened along the way which happened to work out very well. Spent Saturday night in San Francisco (notably at the Irish Bank and The Swig) where I met and chatted with a bunch of different people including one professional photographer (which was interesting as I like to think of photography as a new hobby for me). Sunday I left for Monterey and stayed there until Tuesday.

The most amazing part of the trip was probably driving on the very amazing 90 mile stretch of Route 1 from Carmel to San Simeon. Well, pictures speak more than a thousand words.

Spent most of Wednesday catching up on mail and on meetings / IRC; still got lots of mail and issues to attend to though.

Frankly, my dear, I don’t give a damn

2005-06-26T18:33:00.000-04:00

Zoo

Went out to Franklin Park Zoo this morning and took a bunch of pictures.

Waterfall in the Tropical Forest

Most of them ended up pretty bad due to lack of light (thanks to the somewhat hazy weather here in Boston). I’ve uploaded some of the better ones in this gallery.

Volume handling

Been spending quite some time on a rather large gnome-vfs patch (it’s about 80k now) that enables gnome-vfs-daemon to recognize drives without having them in the /etc/fstab file.

Look mom! No /etc/fstab entries!

There’s a bunch of other benefits including the fact that we actually show drive icons even when no media is in the drive and, of course, that we stop using /usr/sbin/fstab-sync in Fedora and instead use mount/umount/eject programs that can integrate nicely with the GNOME desktop. The latter is quite important, actually the point of this whole exercise.

The main benefit is that we can move all policy into the desktop - this means that we can write a GNOME-based mount wrapper that queries your settings from gconf about where to mount volumes (the mount point) and what mount options to use (e.g. iocharset comes mind). Of course, we have to be extremely careful in what options and mounts point we’ll allow. It will be almost trivial to write a Nautilus extension so this can be edited in a tab per drive/volume. This is planned too. While this sounds like a super-31337 geek feature it’s actually something that both Fedora and upstream HAL has gotten a lot of bug reports about. Another benefit is that we can actually put up meaningful dialogs if the mount/umount/eject fails. For instance, we may put up a dialog with a list of the applications that is blocking the unmount (of course, applications like these needs to be fixed so maybe not worth the effort).

Finally, we’ll be able to do lock-down much more effectively - for instance, administrators should be able to say “Require root password to mount external storage device read/write” (which is done by setting a mandatory gconf key) and we pop a dialog telling the user that we can only mount read-only unless the user authenticates (using the root password or other mechanishms is another discussion). While this sounds like a stupid feature check this out (search for “solder”) and also take a look at some of the discussion regarding what Sarbanes-Oxley is requiring from businesses.

Implementation-wise, I’m planning to use pmount as the backend for the GNOME-specific mount/umount/eject wrapper but right now I’m running into some issues with the privileged parts (that should trust nobody) insofar that they have to check things in gconf / HAL. Have to think a bit more about that and also email Martin.

Chopsticks

So, this female Asian coworker (who has requested anonymity when I told her I was going to blog this, so can’t be specific gave me a set of chopsticks last week. They’re supposed to be easier to use than what they give you at restaurants.

Chopsticks. In orange. They even match my sleeping sofa.

I’m still undecided whether it’s something I should bring to a fancy restaurant, so I carry them around with me until that matter is resolved

Size does matter

2005-06-15T18:44:00.000-04:00

New lens

Went to the Cambridgeside Galleria tonight and picked up the EF 75-300mm f/4-5.6 III Telephoto Zoom Lens. It’s awesome. Can’t wait for the sun to get up as it requires a bit more light than the standard 18-55mm lens that came with my Digital Rebel XT

Size does matter

Now to start thinking about a quick weekend get-away to try out this beast.

D-BUS

Been spending some time on D-BUS the past few days, got a patch for automatically reloading policy in and also some bitching about not breaking the ABI. John put out the 0.34 release that also includes glib/gobject goodness from Colin and Ross. Sweet.

Playing in the band

Friday, me, Kristian, Chris, Søren, John and clee went to this Somerville bar since it was country/southern rock night. It was pretty cool. So, I was standing out smoking, casual and minding my own stuff you know, and this old timer who reminded me of Johnny Cash came up and asked if I was part of the band. I had to say no but he insisted that I looked like a natural-born guitar player and gave me several addresses where to buy a guitar and encouraged me to start playing. I contemplated starting singing to get him on better thoughts

Fedora Core 4

The Amazing Fedora Core 4! is out. Go get it.

A man who doesn’t spend time with his family can never be a real man

2005-06-12T13:35:00.000-04:00

NetworkManager

So, I finally got round to finishing up and committing my NetworkManager VPN patch; here are the details. Also, since all the cool kids do it, I made a screencast to show how it works. Peruse it by clicking the picture below.

davidz does screencasting

The next step is to add support for more VPN software, not at least so we can finalise the extension points (interfaces really) for how to integrate VPN software with NetworkManager. Things like OpenVPN and Openswan comes to mind. When that is done, we’re probably going to lock down the interface and keep it stable. You know, even so proprietary software vendors may integrate their VPN software with NetworkManager.

Denmark

I had a very very nice time in Denmark; arrived on Wednesday the 1st of June, spent the evening at my moms new house in Trørød with my mom, my sister Maria and her husband and two kids. On Thursday my mom and I embarked on a journey to Fyn

the uncle and the nephew - who’s who?

to visit my sister Birgitte and family. It was awesome, Birgitte and co. lives on an old farm and there was a ton of motives which meant that I put good use to my new camera. I need to upload those pictures soon now that I got an account at flickr (again, like all the cool kids .

I returned to Copenhagen on Friday and met with a lot of friends at Vesterbro Torv - I think we were 10-11 or so. It was very nice to see so many of my old pals and it got very late until I returned home (in more than one sense). It was totally awesome.

Beantown

When I got back to Boston about a week ago the weather had changed. It used to be cool. How did that happen? Now it’s about 90F (32C) and extremely humid. We have already had two thunderstorms IIRC. Thank god for AC - except that the one in my car is broken and the one in my apartment is extremely noisy. Oh well, can’t win them all

Stuttgart

2005-05-31T01:55:00.000-04:00

GUADEC

Hanging with Kay Sievers in the Bier Garten

Jeff rocked

Stuttgart is rather nice

Copenhagen

I’m going to Copenhagen tomorrow. As mentioned here I’m up for beer but we moved it to Friday from Thursday. Send me mail!

Live Free or Die

2005-05-24T15:32:00.000-04:00

Hacking

I posted my patch for a small NetworkManager feature. There’s still some work left. Details here.

Also, Bastien proposed to integrate his small tool for importing .PCF files from the Cisco VPN Client. So, today I added a button for this.

K-rad settings

Visit

Yesterday, Marco arrived:

As you can tell, Marco is dutch

Cigs

Not that it’s really anyones business but mine , but due to various, uhm, personal reasons I succumbed and started smoking again. I only smoke at nights though, not at work. Some day I will really stop but now just isn’t the right time… I did go about 13-14 days with a single cigarette. Bah.

Copenhagen

I’m planning on meeting up with friends on the evening of Thursday the 2nd of June in Copenhagen somewhere at Vesterbro Torv. Drop me a mail if you feel like having a beer and hanging out! Hope to see a bunch of people! We should probably start at around 7 or 8pm at Obelix. Send me mail!

Refined Travel Plans

2005-05-16T18:10:00.000-04:00

As mentioned earlier I’ll be travelling to Europe for GUADEC in less than a fortnight.

Friday night I felt like spontaneously visiting Revere Beach, MA

Tonight I just spent some airline miles so after spending $51 for taxes etc. I got this in the mail
OUTWARD FLIGHT: SK658, SCANDINAVIAN AIRLINES 01 June, 10:40, Stuttgart Stuttgart Apt Terminal 1 01 June, 12:20, Copenhagen Copenhagen Apt Terminal 3

RETURN FLIGHT: SK655, SCANDINAVIAN AIRLINES 04 June, 15:15, Copenhagen Copenhagen Apt Terminal 3 04 June, 16:45, Stuttgart Stuttgart Apt
which leaves the Saturday night in Stuttgart before I return to Boston on June 5.

Time: To measure, as in music or harmony. 1913 Webster

I hope to see all my buddies and family in Denmark! Even LKL despite he haven’t responded to mail for the past month. Slacker

OMG!

2005-05-16T07:52:00.000-04:00

OMG!

No Cigarettes

2005-05-10T16:41:00.000-04:00

So, for some reason I decided to stop smoking last week. It was Tuesday morning on my way to work. I haven’t touched tobacco since and this is more than a week ago now.

The Two Towers: John Hancock and Prudential

The funny thing, however, is that it’s not completely clear to me why I want to stop. I’ve got a multitude of small reasons but not really any big ones. Part of my brain is trying to convince the other part about only smoking at night / at parties. I’m still clean though.

An interesting side effect of this whole ordeal is that my senses have improved. Somerville really does stink when the wind isn’t right

View from my study: City of Boston

It’s probably been helpful that while living in my new apartment, I’ve only smoked outside on the balcony and for the past 7-8 months I’ve not smoked more than about fifteen a day. To compensate and be nice to myself, I picked up a copy of Tiger which is installing on my good old Powerbook 12″. I’ve also got myself a mini tripod for the new camera which I used for capturing the (long-exposure) images above (click the pictures to view the full 3456×2304 images).

Now.. I’m guessing that all you people now think “It will be interesting to see how this continues”. I know I do..

Following the crowd

2005-05-02T13:14:00.000-04:00

Seems to be sign of the times at planet fd.o

I like beer

Good times

2005-05-01T17:44:00.000-04:00

M&M visit

My mom and older sister, Maria, visited last week; we went to Niagara Falls (on a two day trip), Cape Cod and did other tourist things here in Boston.

Visiting Niagara Falls, NY early Sunday

It was absolutely great to have the girls visiting and everything panned out very nicely. Also, now I’ve also got the right gear in the kitchen to cook food (my mom remarked that it’s an useful asset in order to pick up a girlfriend)

The Falls!

Visitors

Given I’ve lived here for about 7 months, I’ve been quite fortunate with friends/family visiting me; Dennis was here in December, Maz dropped by in January, the girls in April and it looks like Marco and Lars is coming in May. Yay!

Visiting Race Beach near Provincetown on Cape Cod

New camera

I’ve finally bought a new camera, the Canon Digital Rebel XT which is a very nice 8M pixel SLR camera. It nicely complements my (nowadays, rather old) Canon Digital Ixus V - one for serious pictures, and one for fun.

My view of Beantown through the new camera

Conferences / Travel

I will be at 6UADEC in Stuttgart, Germany and will also spend five days after that probably travelling around Europe (returning June 5). Also, I’m going to OLS in July and will probably fly up there with Blizzard in his small plane . Also, Kay is talking about visiting Boston just before OLS.

Optimistic

2005-04-15T14:50:00.000-04:00

Had my car at the dealership for service; the headlights, horn and wipers got fixed. Cost me nearly $500, yikes. All this is part of the vast array of preparations for the M&M visit (Mom and Maria) who arrives on Thursday; looking much forward to that. Took off the top of the car for the 30 mile (50 km) ride back:

It’s the lighting that makes my chin looks fat. It isn’t fat really

Note to self: Never ever again ride with the top down at 75 mph (120 km/h) when it’s 45F (7C) outside. It’s fucking cold.

I like art

2005-04-09T13:33:00.000-04:00

Played around with Camorama today; this software got some wicked filters, check my piece of art.

Check out the new haircut

It would be cool if setpwc was integrated into the capture tool so I could control my Quickcam Orbit. Maybe we need an integrated video capture widget in GNOME, maybe there is one already?

Power Management

2005-04-04T22:18:00.000-04:00

I’ve consolidated my thinking on Power Management in GNOME. I hope people like it.

Taxes

2005-04-04T18:12:00.000-04:00

Inspired by the other Dave I sorted out my taxes tonight. Unlike Dave, I couldn’t really get H&R Block to work (since it determined I was in a special situation being a H1B alien and all), but after surfing a bit on IRS.gov I found a link to Turbo Tax which rocked. I can wholeheartedly recommend it.

After about 60 minutes of my time and the parting of $29.95, I had electronically submitted my federal tax papers and printed a copy of my Massachusetts tax papers for snailmail submission (for some reason, the Mass papers couldn’t be submitted electronically).

One really funny question during filling out forms and answering questions in Turbotax (btw, the UI wasn’t that bad actually), was whether I wanted to pay the optional 5.85% instead of the ordinary 5.3% Massachusetts income tax. TurboTax said that it was unclear why one would want to do that (go figure), so I opted for no. I later found some rather funny commentary on the subject . Anyway, the bottom line is that I’m getting about 70% of my US taxes paid in 2004 back. w00t!

Now to look into sorting out my Danish tax filings. Not so w00t!

The Office, American style (!)

2005-03-29T16:57:00.000-05:00

I was just channel surfing and ended up on NBC and found an American remake of the totally rad BBC original.

I didn’t like it much and neither did this guy; in fact, to get this out of my system, I had to put one of my BBC Office DVD’s on

DOOM3

2005-03-28T11:58:00.000-05:00

Had Friday off, so I picked up a copy of DOOM3. On Sunday evening I completed it after about 25 hours of playing (with almost no cheating). DOOM3 rocks. That is all.

In other news I wondered this morning what the proper Danish translation of “Denial of Service” is. krh proposed the word Betjeningsstop which sounds a bit too funny to me - I’m sure my good old collegue Martin Koch knows what it is (email me, Martin!).

Build System 1 - davidz 0

2005-03-18T11:19:00.000-05:00

Today most of my time was spent fighting the Build System and right now it looks like I’m loosing.

Hot New Stuff

2005-03-14T11:35:00.000-05:00

Got it delivered today from Design Within Reach - they got lots of great stuff

Next time I’ll carefully read the details when it says some assembly required - my hands are all worn out (from the assembly, that is :-8).

More Banana Republic coverage

2005-03-08T15:52:00.000-05:00

Here is a less sober take on the recent democratic developments in Europe.

However. I’m not happy about this development at all. People should care more about freedom and the fundamental values of our society instead of just caring about themselves and the next reality show on the telly (that’s one for you Joey Punchclock!). Because, before they know it, they’ll have to pay a tax to some big corporation in the northwestern part of the United States to even watch TV.

Just like they do today to be able to exchange documents with the rest of the world.

Someone please explain how this is freedom.

Banana Republic

2005-03-07T17:26:00.000-05:00

We Europeans should be ashamed of ourselves.

Sod it.

Earthquake

2005-03-06T12:34:00.000-05:00

I could actually feel the Quebec earthquake last night down here in Massachusetts. It was about 1.30am and like someone was knocking on the closet door and the train was driving by. I couldn’t hear any train noise and there was nothing in my closet except for a suitcase leaning against the door. Scary. When I checked the news today it all started to make sense.

Native Eclipse

2005-02-27T07:27:00.000-05:00

I just got the 100% native eclipse from Rawhide up and running. It’s using gcj4, a 100% free and open source software stack which means that it can actually be shipped with Fedora since all other viable Java platforms are non-free.

Writing a GNOME application in Java

The Eclipse RPM’s themselves are compiled natively to i386 code so it feels very snappy since there is no bytecode interpreter in place. However, gcj does provide both a bytecode compiler as well as a bytecode interpreter - it’s a full Java 1.4.2 platform. I’m using the 3.1.0 M5 RPM’s from Andrew Overholt - he told me last week at FUDCon that the CDT (C/C++ Development Tools) would hit Rawhide soon. Also check out Andrew’s Java-GNOME/Eclipse demo - it’s totally sweet (requires Flash).

GTK 2.6 provides excellent platform integration

Free Java has come a long way.

Gejsten, Hunter S. Thompson and Nirvana

2005-02-25T07:06:00.000-05:00

I’m happy to see that GEJSTEN.DK is back online - if you understand Danish you don’t want to miss this. In other news, is it just my brain playing with me or does the radios play a lot more Nirvana now that Hunter S. Thompson also shot himself in the head? I’ve heard like three Nirvana tracks on the radio this week. Maybe it’s just inside my head though.

Busy week and finger prints

2005-02-20T17:38:00.000-05:00

Had a busy schedule last week; went to Linux World here in Beantown on Tuesday and then FUDCON at Boston University on Friday. Got pretty well wasted with the desktop crew at a bar in Boston Friday evening. Neat.

Captured with my shiny new driver

Spent most of today hacking on a driver for the APC Biopod that I mentioned earlier. I actually managed to find detailed specs and, with some help from usbsnoop, I wrote a quick one-off user space driver using libusb. Happy about that; been wanting to get this device to work for a long time and now I just need to clean things up.. and decide whether it should be a kernel or user mode driver. Then to find some good finger print verification software; this looks like a good starting point. I’m afraid getting all the pieces done in a nice way (e.g. integration with GNOME Keyring) will require major work (finger enrollment, UI, PAM stuff etc.). Still, sounds like a fun project to do on the side. And it would be awesome to have this in Fedora.

World Map

2005-02-16T17:49:00.000-05:00

Heh, even after a little more than four months over here, I found this quite amusing

New Car Stereo

2005-02-13T08:42:00.000-05:00

I got off my ass today, to buy a new car stereo.

Cracktastic, isn’t it?

It’s a Sony CDX-F700. It passes the davidz checklist:

Detachable front to deter stupid pricks from breaking into my car

Ability to hook-up my ohh-so-stylish iPod Mini

Plays MP3 CD/CD-R/CD-RW. Maybe even regular Audio CD’s as well..

It also got a lot of cracktastic features like a remote control (uhm, right, so the girlfriend in the trunk can change the tunes) and other stuff I’m almost certain that I wont need.

It’s getting installed on Tuesday night, so I’m a happy man.

Google does it again

2005-02-08T15:10:00.000-05:00

Google Maps Just Works™.

davidz’s place - click to go to Google Maps

That is all.

Superbowl

2005-02-06T11:12:00.000-05:00

Bryan: We got some beer

beer : n : fermented alcoholic beverage brewed from malt and hops [syn: suds]

Seems that krh is already all excited:

krh digs Xprint

Now, GO PATS.

Blizzard Warning

2005-01-23T10:51:00.000-05:00

It’s been snowing and then some

Right now there’s a pause but it ain’t over till the fat lady sings

Check also the blog entries from Nat and Joe for more action packed pictures.

Re: Applications instead of Commands

2005-01-22T20:50:00.000-05:00

Fernando writes about improving the gnome-volume-manager capplet interface and I think everyone agrees that it’s needed - the current behaviour with asking for a command line is a bug. A severe one indeed. What I don’t like, however, is the suggestion of mixing MIME types into the picture.

When you look at it, what g-v-m provides is really a way to configure what your desktop does when certain system-wide events happen, while a MIME type is used to describe content, be that a message, file or whatever (sure, some of the heads on xdg@fd.o will disagree and say that MIME can used for anything). Whatever.

In the future I’m pretty sure we want to expand on what actions to take when system-wide events happen; for instance there should be some way of saying “Start Rhythmbox” or “Start JuK” when a music player is attached and I’m pretty sure we don’t want to invent a MIME type device/audio-player.

There’s also the “action” aspect involved; you want to do different things when a device is attached versus removed.

It looks to me like the best solution for this involves extending the fd.o desktop entry specification. One way of doing this would be to add appropriate sections


[System Action MediaAudioDiscInserted]
Exec=gnome-cd %f

[System Action MediaDVDDiscInserted]
Exec=totem %f

[System Action MediaPhotoMediaInserted]
Exec=nautilus %f

[System Action DeviceCameraInserted]
Exec=gthumb %h

Note that the %h parameter is the HAL UDI which is a unique ID representing the device; this is not needed for the Media* system actions. To get rid of the HAL dependency and provide interoperability with software not ported to HAL, this could be extended to


[System Action DeviceFooInserted]
ExecHAL=my-app --hal-udi %h
ExecDeviceFile=my-app --device-file %d
ExecPath=my-app --fs-path %f

and the launcher (e.g. g-v-m) would select the appropriate one. The order of the Exec lines would matter if more than one parameter is available.

KDE 3.4 with HAL backend

2005-01-22T06:40:00.000-05:00

I might have been the last person to notice this, but it seems that KDE 3.4 will have a HAL backend for discovery of storage device much like the patch I got into GNOME VFS last year. Rock on.

Shameless Plug + Happy Bday

2005-01-20T18:18:00.000-05:00

If you’re wondering what I’m working on, here is an article I wrote for Red Hat Magazine. Also check out the other goodies in that issue.

On another note: Happy Birthday, mom!

64-bit Living Room

2005-01-12T09:39:00.000-05:00

Got my furniture today!

Yay!

Check out the foot rest - that’s also on the Love Seat

The glass table I blogged about earlier

On Sunday, I also picked up a new 64-bit system. It’s a HP Media Center Photosmart PC.

It came with Windows XP Media Center Edition 2005 which is kind of cute (spiffy and smooth animations) and a boatload of OEM software. If people think that most usable Linux distros are like Frankenstein, e.g. various different UI pieces thrown together in a hurry, try this - it’s so craptastic; will post a screenshot later. Ended up installed the x86_64 version of Fedora Core 3 to have a real 64-bit operating system.

Snow and Furniture

2005-01-08T13:48:00.000-05:00

Went up to Nashua, NH today. Just after I left it started snowing which meant that it took 80 minutes rather than 45 minutes to get there. Driving conditions were terrible, I saw at least four cars parked on the shoulder, all out of commision.

Went to Furniture World and spent around $1700 on a Love Seat and a matching Recliner. Both in a contemporary green/grey colour (or at least that what’s the nice girl in the store said). Man, a European has to get to used to the American language. Also picked up a cocktail table in glass with a fossil stone base (sic) - the table is a bit “I don’t know”, not sure yet it that was a good purchase though I’m sure Pat Bateman would approve . We’ll see.

The Goods are being delivered on Wednesday, I can’t wait, it’s beginning to get kind of creepy not having any furniture (except for the telly and the slim PS2 that I got within a week of moving in - good for a man to have his priorities straight). That brings my short list of furniture yet to acquire down to: Futon, Desk, Bed, Dining table, AMD64 Shuttle, Misc Items. Oh well, the AMD64 Shuttle isn’t exactly furniture but, hey, it’s all the same budget

Off to dinner at the Middle East (very good food, been there once before) and then some Karaoke later on.

Back in Beantown

2005-01-01T14:51:00.000-05:00

After 8 days of vacation in the old country, I’m now safely back in lovely Somerville. Refreshed and ready for a smashing 2005, hope it will be less disruptive than 2004.

Happy new year to everyone!

I’m portable

2004-12-20T16:26:00.000-05:00

<img src="http://www.bbspot.com/Images/News_Features/2004/10/file_extensions/pdf.jpg" width="300" height="90"
border="0" alt="You are .pdf No matter where you go you look the same. You are an acrobat. Nothing is more important to you than the printed word."/>

Which File Extension are You?

The Almighty Buck

2004-12-20T12:06:00.000-05:00

Østerbro-firma udvikler spil til tv-imperium (in danish)

inactivity log for davidz

New D-Bus features in GLib 2.30

C Code Generator

Object Manager

Writing a C library, intro, conclusion and errata

Table of contents

Topics not covered

Errata

Writing a C library, part 5

API design

Checklist

Documentation

Checklist

Language bindings

Checklist

ABI, API and versioning

Checklist

Writing a C library, part 4

Helpers and daemons

Checklist

Testing

Checklist

Writing a C library, part 3

Modularity and namespaces

Checklist

Error handling

Checklist

Encapsulation and OO design

Checklist

Writing a C library, part 2

Event handling and the main loop

Checklist

Synchronous and Asynchronous I/O

Checklist

Writing a C library, part 1

Base libraries

Checklist

Library initialization and shutdown

Checklist

Memory management

Checklist

Multiple Threads and Processes

Checklist

GNOME Online Accounts

GNOME 3.0 is out

PolicyKit textual authentication

GNOME 3 Authorization

GNOME 3 Calendar

GNOME 3 Calendar Integration

gdbus(1) bash completion

This blog has moved...

udisks and gnome-disk-utility - past, present and future

Plumbers 2009

GUdev, gobject-introspection

Storage handling in GNOME

Getting GVfs and FUSE right

DeviceKit presentations

Resolution Independent GTK+

D-Bus introspection

Linux Plumbers Conference CFP extended!

gnome.conf.au 2008

Politics. Mudslinging. Chocolate.

On Xguest, Access Control and Desktop

Coverage

Policy, Mechanism and Time zones

System Bus Activation hits Fedora Rawhide

No more polling

PolicyKit release

Interwebs

i’m in ur RAID, striping ur disks all the way to CA

Battery Recall^WExchange

Native languages and poisonous people

hal 0.5.9

Stupid meme

Gah!

Who Killed The Electric Car?

Sydney

Fast User Switching

D-Bus 1.0 release party

Trying to be Constructive