Sean Middleditch

Frustration Alert Level: Orange

Why do the super simple programs in GNOME all have a bazillion features and plugin support these days? Gedit is a monstrosity of complexity. It crashes viewing medium-sized HTML docs (Epiphany loads gedit to “View Source” on a page, unlike Firefox which uses its own HTML viewer) and I have no idea if it’s gedit itself, gtksourceview (because a text editor just has to have syntax highlighting), or some other plugin.

Why can’t GNOME just ship a bare-bones text editor a la Notepad instead of a giant monstrosity that nobody in his right mind is actually going to USE for anything? How many developers - the Gedit developers themselves included - actually use Gedit to work on code instead of Vim, Emacs, or some IDE? It doesn’t need syntax highlighting. It doesn’t need plugins (are there even any third-party Gedit plugins?).

Now Eye of GNOME has also gained the bloat. I’m sure it comes accompanied with an abysmal startup time, just like Gedit. Notepad, in a Virtualbox VM, loads instantly on a clean Windows XP box. Gedit takes 2 seconds on the same machine running native Linux, and that’s a warm startup at that.

Nautilus has long been a beast, although I think it at least has an excuse, what with its direction having changed course several times after Eazel disappeared. I’m not saying it needs to be rewritten, but it probably has some cruft that could get the axe. Evolution has been a nightmare from day one, especially for the 95% of us who don’t use notes, todo, calendar, or various other groupware features. Most of us just check email, and use a contact database to make the checking and sending of email a little easier.

I use GNOME instead of KDE because I don’t want my desktop bogged down by tons of useless crap some developer wanted to add so he could wank off to how many features his program has. Gedit is full of features that nobody needs, and all those features destabilize it to the point where I can’t even trust it to do the one or two things I actually need it for.

I have a feature request for GNOME 2.22:

Cut out the pointless bored-programmers-induced bloat that nobody needs from Gedit, EOG, Nautilus, Evolution, gnome-panel, and so on. I’m sick of GNOME taking longer to load a working desktop (starting from when I finish logging in at GDM) than it takes for a virtualized Windows XP to load a working desktop (starting from when I click the “start machine” button).

This is a bit rambling. It’s late and this post isn’t as cooked as most of my posts are, so please forgive the wandering focus and half-complete ideas.

James’ Comment to the Linux Packaging Sucks entry says:

Would significantly expanding the LSB help, or is it also not the right solution?

I don’t think that standards in terms of the base platform are really a problem. That is solved, in my experience. I can take a binary and get it to run on a large variety of systems, especially if it’s compiled with a tool like apgcc that works around the incompatibilities glibc causes with each release (apps compiled against new versions of glibc don’t work on older versions of the library, but the reverse is not true).

The vast majority of compatibility problems for your average applications are 100% artificial and are caused by the package system, or by build systems that don’t take into account the incompatibilities that glibc and certain other libraries cause.

For example, if you compile a GTK app, it will be locked into a certain version of the library even if you only ever use any public API from the 2.0 release. This is because macros are used to silently convert function calls to newer functions. The old function remains for backwards compatibility, but your application now depends on a version of the library with the new version of the function even though you never asked for the new version. If you know that you only need 2.0 features, compile against 2.0 or you’ll be artificially locked into a newer version.

Perhaps LSB could help, but I’m not sure how. I’d rather that developers of libraries (and there are only a few guilty parties here) just took some care for those of us who don’t want to waste our lives compiling binaries other people have already compiled for us and fixed these silent ABI breaks. It would be far clearer, in my opinion, for a developer to explicitly state which version of a library his application is expecting, and to just not use any of the macro tricks.

For example, GTK could just include headers for all past versions (at least in cases when the ABI of a function is updated), so if an application does soemthing like:

#include

The app will compile against the 2.4 ABI, will not be able to use any of the 2.6+ features (even if GTK 2.10 is installed), and no ABI changing tricks will be employed. The app will work on any machine with GTK 2.4+, even though it was technically compiled against GTK 2.10. Yeah, that means you need a lot of header file symlinks, versioned files, and #if checks for defining new functions… I think it’s worth it. The actual additions will not be that ugly or complex… certainly not much worse than the macro hacks in place now. :/

Instead of using header include paths to denote version, you could also just require a developer to state the version they want to use with a macro. Something like:

#define USE_GTK_VERSION 206 /* Version 2.6 */
#include

If the macro is not defined, then implicity define it as 200 (2.0). To use a newer set of functions, the developer must explicitly state which version of the API he wants.

This is really quite useful even if you live in a world where apps are only distributed as source. It makes development easier for projects like Mozilla, which want to ensure that their binaries can run on a variety of Linux distros, including those that ship (by FOSS terms) ancient libraries. It ensures that when a developer is working on the code and accidentally uses a function or other feature from a newer version of the library that compilation fails for him up front, instead of having to wait for it to get processed by a build daemon (which only has the oldest supported version of the library installed) or for a bug to be reported. It makes sure that the developers get what they asked for and nothing else, helping them to keep control over what their code is doing.

Make no mistake, both glibc and GTK are excellent examples of projects that “do things right” in terms of maintaining ABI and API compatibility with older versions. Old binaries run against newer versions of the libraries and old projects compile against newer versions of the libraries. Not too long ago in the FOSS world even that was hard to get. These libraries are huge, complex, and they’ve had to pull some tricks to be able to both fix bugs and maintain that compatibility. I just think that they could go one step further and solve the last remaining problem that the platform in general has with version portability.

At that point, it’s all up to the distributions and their myriad of incompatible and frankly archaic package systems to fix their user interface (mis)designs and try to either standardize their systems or include a new vendor-neutral second layer.

The second layer idea is one that’s been tried a lot by people fed up with the packaging solution. AutoPackage is a “second layer,” a system which allows users to install software outside of the core package framework. I don’t feel that it’s ideal, but there’s no reason this couldn’t work. I don’t expect it to be able to handle upgrades to large frameworks or core OS tools, but that’s not really a problem I think - even something like the core version of GNOME installed is, to most non-geeks, a part of the OS itself, and not just a set of add-on packages. (This is part of the reason why I disagree with people who feel that X isn’t part of the OS - maybe it’s not on your server, and maybe it’s not on our desktops, but if X disappeared from my mom’s machine, I guarantee you that the system would no longer be anywhere close to operating to her needs, as it could no longer run even a single one of her applications.)

I would have no problem at all with a distro that used two totally different package systems: one for its core libraries, tools, and required apps, and another for all the stuff that is optional. Just so long as the UI was sane, which includes a single GUI app to handle updates; there is no need for a GUI app to install core packages, since for most users they will always all be installed.

Some add-on package systems, like AutoPackage, try to use fancy autodetection code to pick up the various libraries they need. This I rather strongly disagree with as an optimal way of handling things. Preferably, and this is something the LSB perhaps could help with, every library should be easily detectable without any kind of custom logic. It should be as easy as checking if libfoo.X.Y.so exists. Some libraries don’t hold to this, unfortunately, as they keep the same .so but change data files or just don’t update the ABI version appropriately. In these cases, the OS needs to offer integration with its native package system. AutoPackage (or another add-on system) should be able to ask the core system if “libfoo.1.2″ is installed, and the core system can then look in a database that maps these logical names to package names and can tell the add-on system if the library is available. It can also then offer hooks to ask for automatic download and installation of the library using the OS’s native software installation system.

Ideally, though, all software should follow the existing rules and practices and make checking for libraries easy. Existing package systems should be able to take a library name and find and install the appropriate package. Standardized utilities and scripts (eg /usr/sbin/sendmail) should be enough to check for instead of needing goofy hacks like DPKG’s and RPM’s meta-depends. It should be stupidly simple to write a wrapper for YUM, APT, and so on that just takes a library or binary name and installs a package that includes the appropriate library or binary that is guaranteed to be compatible as one would expect. Things like Perl modules and such also need support, of course.

Even if the LSB had a tool that used those principles to make it easy to install packages using vendor-neutral names, it still wouldn’t come close to solving the real problem. Packages still need a way to call those tools, requiring some standard package format. (The existing “standard” LSB uses is pretty much a joke in terms of actually being useful.) The distros tools still need a huge amount of polish. The standard package format needs the meta-data additions to make logical (from the point of view of users) package grouping possible. Updates have to be handled sanely. Upstream developers have to be able to rely on the ability to make a single package that works on every distro (well, one package per architecture, obviously).

Someone out there has a hacked credit card database. I know this because there is a new $280 purchase from Old Navy in Goodyear, AZ on my bank statement. I, of course, live nowhere near Goodyear, AZ, and I have never shopped at Old Navy.

Tomorrow I get to go bitch, get the charge removed, and get my third new debit card in a year. The first two were pro-active replacements from the bank after some other site got broken into, and banks replaced all the cards in those vendors’ databases.

Seriously, people, it is NOT hard to secure credit card databases. Starting with the fact that in the vast majority of cases, you don’t even need to store the credit card number if you’re doing immediate authorization. I want to blame shitty programmers. It probably was shitty programmers. It could also just be dumb management. In my experience, most managers and store owners (the people who pay programmers or contract to companies who pay the programmers) don’t understand a damn thing about security and literally demand (on threat of unemployment) that the programmers do stupid-ass things like store not only credit numbers but also CCV codes because the billing department thinks that they need that info to do their offline credit card processing.

Then you combine that with either shitty programmers or restrictive budgets and deadlines and you get unencrypted or wealky encrypted databases storing this info. Then the site eventually gets hacked into and aside from all the other usual treasures in the database, the hacker also has a huge list of complete credit card numbers with expiration dates and CCV numbers that the vendor didn’t even need to have because CCV numbers aren’t required to charge a card.

I’ve seen vendors demand that they take the CCV number for offline processing because they think that customers won’t use a store that doesn’t take a CCV number. The problem here is the gross misconception that CCV numbers are there to protect the consumer. They’re not. The card is already insured. I will be getting my $280 back for sure. The CCV there is to protect the _vendor_ to ensure that any cards they accept are less likely to be fraudulent. The CCV is there because the credit card companies don’t want to lose money to thieves, so they enforce PCI compliance when they can to make sure that vendors make it difficult to use stolen cards. Unfortunaetly, most vendors don’t comply with PCI standards, so they not only fail to use the CCV code properly, they also fail to maintain all of the security standards that PCI requires to protect stores that use CCV codes, thereby making the whole thing moot.

I looked through my statement for the last three months (about half the time I’ve had this particular card number), and I can’t find a single shoddy/no-name store in the list. All of my payments have been to large well-known chains (Krogers, Outback, etc.), upscale local businesses (Yotsuba, Zingermans), local convenience stores (Marathons and some other I don’t recall the name of), big-name online retailers and payment services (Amazon, Paypal, etc.), and the US government (taxes and stuff). I haven’t entered my credit card number in any other sites, lent the card to anyone, or otherwise done anything stupid or reckless with that data. So some fairly big well-known company is at fault here. I haven’t seen any recent announcements of hacked databases, so someone has access to one without anyone knowing (yet), or the company is breaking laws and keeping it a secret out of fear of the legal and public backlash that announcing cracked insecure credit card databases always brings.

In any event… I’m pissed and irritated.

And to whoever has nice new clothes from Old Navy - you’re a douchebag. (I’m not sure if that applies to everyone who has recently shopped there or just the thief - I mean, it’s Old Navy for goodness’ sake, what the hell is wrong with you people?)

haruspex:

salimma: “once” anti-GPL?!

Are you trying to imply that Novell is anti-GPL? What in ****’s sake gives people stupid ideas like that?

Novell is one of the largest producers of GPL code in existence at the moment, employing people who work on a huge range of projects including the kernel, KDE, GNOME, various core GNU tools and libraries, RPM, GCC, Apache, Samba, and whole tons of other stuff.

Some of their products being closed-source does not mean they’re anti-GPL. It doesn’t even mean that they’re not pro-GPL. It just means that they want to keep making money in order to keep employing engineers and tons of other people so that they can eat. Not everybody gets donated tens of thousands of dollars to preach ideological bullshit like Stallman does. It’s easy for him to say, “If you can’t work on Free Software for a living, go into a different field instead of making proprietary software.” He’s never had to actually make that choice.

minor note: yes I realize that not all of the projects I listed above are under the GPL. My point was merely that Novell works on a huge range of Free Software and Open Source projects without trying to subvert them.

Linux packaging sucks. I’ve written about this before, and I’ve planned many times to tackle the issue with a new Ultimate Package Manager That Doesn’t Suck(tm), but the constant realization that it’ll never get used by all mainstream distros and thus never actually solve anything worthwhile keeps coming around and killing any motivation I had for the project. Every few months the bug bites me again and I pick up work on the design for a new system, and I start by identifying the problems of the current Linux packaging landscape. Thankfully, even with all of the projects out there trying to tackle this, Linux packaging sucks the same as it ever did, so I don’t have a lot of work to do to reidentify the problems.

Appliances

Linux distributions are almost invariably built using the appliance model. You get a distribution, at some specific version, which comes with some set of packages (many of which may not be installed by default, or even on the install media), and that is all you get. Your version of Red Hat or SUSE or Ubuntu has a specific set of packages at specific versions. If you install software outside of that package set, or you install a newer version of one of those packages, you lose ALL support from the official packaging system: you will not receive any security updates for the software you install. If you didn’t use a third-party packaging repo, then you even lose all the support of the packaging system mechanics themselves, such as easy removal, or even the guarantee that your custom-installed apps and libraries won’t be over-written.

This really isn’t about proprietary software. Excellent Free Software gets screwed by this setup. A new version of Abiword comes out? You have to upgrade your whole OS to get the new version. If there’s even a new version of your OS out yet that has that new version - which isn’t likely to be true for another 3-12 months after that version of Abiword comes out. Some softwre, like Abiword, tries to offer packages for a variety of distributions, but they can’t cover every distribution, or even every recent version of the distributions they do support, and they usually don’t have the solid update services for their packages that the distribution has. They can’t be expected to, since each distribution needs a totally different update mechanism.

I really can’t stress this enough. So long as Linux distributions are built as appliances and not as compatible platforms that allow for one package from the upstream authors to work on a wide variety of distributions, Linux packaging is going to keep on sucking.

Best of all, this is already known to work. Packaging add-on solutions like AutoPackage have already solved many of the pure technical hurdles. The only problem left is the social and beaucratic hurdles imposed by the distributions. (I’m not saying that the AutoPackage software should be used. I don’t really care for their system. But their tools like apgcc, binreloc, and other research has shown that it is quite possible to make binaries of Free Software run on multiple Linux distributions with no problems.)

Terrible User Interfaces

Let’s get off the appliance topic. Let’s pretend we’ve got the perfect Linux distribution that packages every single Free Software project in the world, and includes automatic updates indefinitely for all of them for every version of the distribution.

Let’s say you’re trying to OpenOffice.org. You fire up your nice easy to use graphical package manager - Pirut, Synaptic, whatever. You search for OpenOffice.org. You are cheerily returned 20+ something packages (in most distributions) with cryptic names like openoffice.org-common, and are expected to figure out which ones you want. Sure, dependencies are marked for you, but do users really have ANY need to even see packages like *-common in the package manager, really? No, they don’t. Those are internal implementation details of the packaging structure. The *-common packages are useless by themselves are are installed automatically if you ask for the actual application. Many distros have a lot of these pacakges, and Debian-derived ones (eg Ubuntu) have more than any other. These are noise. They don’t need to be shown. At all.

So we add some package flags or rules or something else clever to make sure that *-common and other “internal” packages are never shown. Then what? We get a nice list of OpenOffice.org packages that may or may not be organized in a nice way. If we’re lucky, the matched package list will show:

OpenOffice.org
OpenOffice.org Writer
OpenOffice.org Calc
OpenOffice.org Presenter

Wait, what’s the first package? If I just install that, do I install need to install the Writer? Most of you reading this will recognize it as a “meta package,” a package which depends on other packages to make them “easy to install.” Other than the fact that they’re really only useful for people on the command line who want to type less and in a graphical package manager it’s just noise. So we go ahead and add meta-data or rules to hide those in the graphical package manager, too.

Now we still have some 20+ packages left, because software like OpenOffice.org has some bazillion add-ons. Language packs, clipart, templates, and so on. Complete pain in the butt to scan through when it’s all in a nice big useless flat list. It would be so much easier for most people to just have one entry for OpenOffice.org that installs everything, because let’s face it, most of us have the disk space. But not all of us, and some of us have other legitimate reasons to not install everything, so we can’t just turn OpenOffice.org into one big package.

Thankfully, this would be pretty easy to solve with a little more package meta-data. Users of installation engines from the Other OS having likely seen what I call the “component” model to packaging. There is a single installer (ie package) for each piece of software, such as OpenOffice.org. That package then has a number of optional components (also packages). The installer will display all of the components, usually in a nice tree format if there are many of them, with some marked for installation by default and others not. This would be easy to add conceptually to almost any graphical package manager, once the underlying packages contain enough meta-data.

For example, the OpenOffice.org meta packages on most distributions could be marked explicitly as a meta-package, and its dependencies then becomes components. Package formats like dpkg that offer Suggested or Recommended packages already even have a means of marking install-by-default components vs purely optional components.

The graphical package manager, using the new meta-data, knows to only show the OpenOffice.org package, and none of the other dozens of packages that make up a complete OpenOffice.org install. Either by using an expander, a pane, a popup, or whatever other UI element the designers happen to prefer, the user can view the list of components of OpenOffice.org before installing it. Most users are likely to just check the meta-package and run with it, which should get them all the standard components and most useful extras. The UI is as simple as you can make it for average users of OpenOffice.org (which, btw, also includes highly technical skilled users who simply have no reason to waste time with largely inconsequential sub-package selections) and still has flexibility for those who actually need it.

The user could later open the package manager to view his list of installed applications. He can look at the OpenOffice.org application and look at its list of components and add or remove some if he wants. Compare this to opening up the package manager to look for “uninstalled” packages to find add-ons for the “installed” OpenOffice.org. Sure, that makes sense to us techies (the add-on package is uninstalled, after all) but it’s an illogical complication for people who are, quite justifiably, thinking about _applications_ instead of about RPMs or DPKGs or whatever.

Updates

A new security update comes around. This is an update to KDE, which affects a number of packages. Instead of the updater showing a nice, “There is a security vulnerability (CVE-XXXXX) patch available. Please download.” the user instead sees:

There are updates available:
kde-konquerer
kde-kioslaves
kde-foo
kde-bar

What is that all about? Are there new features? Security updates? What the hell is kde-kioslaves? I’ve never run that program!

I’m rather surprised this one hasn’t been fixed already. Developers are already used to this excellent technology called “version control” and are used to writing these awesome documents called “changelogs.” With package update services available now, developers slap a new version of a package into a repository, updates check to see which new packages are available, and give the user no more information than that list of updated packages.

Instead, the update repository could include each updated set of packages as a cohesive changeset. Along with the list of packages could be something as simple as a changelog entry indicating why those packages are being updated. The user now gets much more informative update notices, and updaters could still allow individual package selection if they wanted (as they’d be able to compare package updates along with the changesets as a whole - they’d even be able to see which changesets apply to a single package update if they haven’t updated that particular package in a while). Some distributions already include per-package changelogs, such as Debian, but this information is sometimes overly technical and boring, even to technically skilled users, and per-package changelogs aren’t always the best fit for updates that span multiple source packages.

Of course, if the component meta-data is available, the update notices can be even nicer, by just showing “OpenOffice.org” as the updated package, which is all that most of us really give a hoot about, even if all 20+ OpenOffice.org packages are being updates. Those of us interested in seeing which components are affected could still look at that information if the updater UI wants to provide it.

Summary

- Packages need to be able to be distribution and version independent so users are not locked-in to their distributions’ package sets
- Large software suites that are split into multiple packages need meta-data to associate the packages logically with the application
- Update services need to group related updates and include proper update notices