Sean Middleditch

DistCC and CCache

I setup DistCC and CCache today to see how well they work. Both were rather easy to setup, aside from having to get my Fedora 1 compiler onto my Debian Sarge nodes. (Not that hard, really.)

ccache will help a lot given how I often have several trees of AweMUD I work on (yay Arch!), compilation times already are cut down quite a bit thanks to that. I’m not too sure how much distcc will help me, though. Both at work and at home I have one fast machine (my workstations) and a ton of very slow and over-worked server machines.

Two changes to distcc would be quite useful that might improve the situation for me. The first are the Apple zeroconf patches. If I were to install distcc on all the spare machines floating around, trying to maintain DISTCC_HOSTS would be a nightmare. Second, the load balancing would be nice. I.e., if machine X is very fast and machine Y is very slow, send jobs to X first, and only to Y when X is full (or it has slowed down due to load).

Another nicety, which I’m not sure how to implement on my own, is to have make automatically use the -j (# of jobs) option when distcc is being used, and not use it when it isn’t. No idea how to do that automatically, other than make a make wrapper that both checks the environment variables for CC and CXX, and/or looks into the target Makefile to see if it has CC or CXX set to something with distcc. And then of course to figure out automatically how many CPUs we have to know what -j value to use. (A command line option to distcc to query alive nodes and compute some decent output for this option, coupled perhaps with the load balancing feature above, would be great.)

Ouchie

Dear gods am I sore. Very eventful couple days on the LARP field. Yesterday afternoon five hill giants attacked our keep, which wasn’t very pleasant. The “stepped” over the walls and got in no problem. (This is played by the NPC players declaring that they are stepping over the walls, and we then have to let them in the gate.) The simple fact that we were “fighting” at all was a bit of anomoly, because one couldn’t really turn away the spears of fiften foot giant. Given that all “monsters” are just regular people though, of course, there was some actual battle for a while. I was holding off one giant on my own fairly well, until the two behind me finished off some companions and proceeded to spear me. My friend, the captain of the guard, was the last standing (mostly due to the incredible amount of armour he was wearing) but even he fell under the might of five giants. I mean, come on, the out come was pretty much a given.

This day I was accepted into a military company, which among other things means I now have to have responsibility and purpose, which kinda sucks. On the other hand, it’ll increase my available resources. Guess I’ll see how it works out. One thing I’m not happy about is that now I’m being required to learn to use a shield (it’s like a chain on my arm, and doesn’t help me much given my prefered style - quick and lite), and furthermore the military company’s colors/standard are blue and yellow which clash horribly with my black and maroon outfit and gear. Hopefully I can get out of the shield bit, and I can live with having the standard on a cloth bit hanging from my belt (not sure what those are called; most of the house members have similar bits of cloth on their belt though.)

I took a serious head shot yesteday. We were doing night fighting (I was a giant acid-secreting ant; more fun to play than it sounds) and I had several people battling me. One of my assailants swung hard, my head a bit low, and I took a direct hit to the forward. I usually make a big show of falling over when I die as an NPC (it’s more fun for the players, and I’m rather good at controlled collapsing, due to some acting parts I’ve had in the past, so why not?) but I didn’t need to this time. It was all natural. Smack to the head, eyes crossed, I sword the world rotate around me and smacked into the ground. Rather comical to witness, I imagine. Hurt like a son of a bitch, though. Still have a tad bit of a head ache. On the upside, I was quite alright to continue playing, played another ant not 5 minutes later, and played some trolls not long after that.

Anyways, this is all rather not FOSS related, and you Advogatoans are probably not interested in reading about it, so I’ll lay off the subject for now. Don’t expect that to last, though. ;-)

LARP

Ugh, am I sore. Spent all yesterday lugging around the swamp/woods/lands fighting various things. Not usually winning, either. And it’s time to head back out in a few minutes…

Releases

It’s been suggested to me that perhaps I release AweMUD 1.0 soon, just to garner a bit more attention. Basically, a few more urgent essential features would be added, and then the release would be made, and I’d hack on the other features in the post 1.0 releases.

Not sure how I feel about the idea. I’d like 1.0 to mean “mostly” done, but I realize that isn’t how reality usually works out. I do want to make sure that any 1.0 type release be stable and usable. Which means there is definitely some hacking ahead. :)

But, I should get off the computer, get back in my armor, and head back out to the field. Kill more stuff. Maybe this time actually find some valuables on the buggers. My character is bloody poor…

Arch CheckSums

Got my arch repository converted over to using checksums.

Also moved my development folder on my work box to the local hard-disk, since Arch has this incredibly unbelievable habit of not working on them because inodes change. Which is a pretty damn dumb reason. But nothing’s perfect, I guess.

I also found out my GPG key is still intact and usable, so I might start signing my archives too. Not sure what use that’ll actually have in my case specifically, but it’s Cool(tm), right?

She’s a Geek

You know a girl is a geek when the following valentine poem actually pleased her. One of the geeks on the “you’re way too much of a dork” end of the spectrum, unfortunately. ;-)

Roses are red,
Violates are blue;
All of my base
Are belong to you.

The freak. (Ya, you know who you are.)

April 1st

It was suggested to me that something be done regarding the AweMUD site’s last news post. The last news post I’ve made was the April Fools post. It was suggested this be removed, changed, or another post be made, as people may read the article, not notice the date, and think the project dead or worse. (Yes, worse.) So I changed the news post’s title to “April Fools ;)” If they still don’t get it, then I probably don’t want them anywhere near AweMUD anyhow. ^_^

Fingers

… killing me. Been playing my bass a lot lately. Now that I know several songs, and am capable of playing fairly well enough to pick up new ones, I’ve found that playing is… addictive. You just don’t want to stop. And everytime you’re not playing, you have a big urge to start doing so.

Kinda reminds me of what coding used to feel like, long ago. ;-)

/etc/roles

Red Hat Bugzilla #120571

I wrote a script and patch for adding /etc/roles support to SELinux. So, instead of needing to hack in m4 macros and botch the ability to upgrade sources with RPM, you can just edit /etc/roles and rebuild the policy nice and clean like.

Need to figure out how to tell the policy (or system utilities) what the default login role should be. A user with user_r and sysadm_r roles, for example, should not have sysadm_r as the default. Not good.

Also, some tools like addrole and delrole would be nice, for modifying the /etc/roles file and automatically rebuilding/reloading the policy. useradd/userdel should also support this functionality. Currently, there are separate seuseradd and useradd commands, which is absolutely retarded; no idea which “genius” thought that one up. Just make it so a flag to useradd gives a default role, and if the default role is ommitted, don’t add an /etc/roles entry and, by extension, don’t register the user with selinux. (Which is done in this patch by adding them to /etc/roles.)

Of course, the whole “SELinux user entry being separate from UNIX user entry” thing is absolutely stupid. There is *no* good reason for it at all. All it does it *limit* what SELinux can do, make administration more complicated, and break a lot of good models like “configure things in one god damned place only.”

The only excuse I’ve heard so far for why SELinux has separate user IDs is to prevent “privilege escalation” when switching user IDs. Which is a bogus claim. For one, with a properly setup system with SELinux, you could remove the setuid() syscall entirely. Just make it so the few login daemons support it. No other process will *ever* need it. Second, you’re making security worse since the system now has two separate user IDs that can both cause various problems if misconfigured. Complexity is the bane of sanity, and UNIX has done an excellent job of avoiding this thus far; how kernel hackers so intelligent as Linus himself could have actually applied this god forsaken mess to the kernel is beyond me.

But, at least, with the patch above, the system can be a little more tolerable. You can almost pretend SELinux is something that was actually designed for, well, Linux.

Object Oriented Database

For shits and giggles, I started working a simplistic object oriented database system. I’m laying it on top of SQLite for now. Probably won’t be made “portable” or anything since there’s little point to do so given my target applications.

The model is quite simple. The database holds a number of objects. No namespacing or anything like that. Just objects with unique IDs. Each object can have zero or more attribute, identified by name (unique to each object) or ID (unique across the whole database). Attributes are either strings, integers, booleans, object references, or a list of one of those types. (That is, each list only holds one type, like a C/C++/Java/C#/etc array, and lists cannot contain other lists.)

This makes the core operations of the database very, very simple. 1) What is the type of attribute foo? 2) Give me the value of attribute foo. 3) Set the value of attribute foo. 4) Delete attribute foo. 5) Create object. 6) Delete object.

The attribute based operations can either take an object ID and attribute name or just the attribute ID as the key.

Now, the only problem left is how do you find the object you want? A good developer will have come up with some kind of object schema (which aren’t enforced by the DB). Perhaps they put an attribute called “type” on each attribute and another called “name”. These are more or less used the same as the object ID; the identify the objects. In AweMUD, I might have a database for all game entities with the type being “charcter” or “room” or whatever.

What we need is a way to find an object based on its attributes. “Find the object with the type ‘room’ and the name ‘temple_hall’.” Or perhaps something more complex. “Find all the objects with any type except ‘zone’ and the value ’spawn’ in the string list attribute ‘tags’.”

I’ll probably need to put together a simple query language. This language would just generate the appropriate SQL commands and send those off to the SQLite layer. The language would be SQL-ish, but simplified for object lookups. There wouldn’t need to be a command keyword, since there’s only one command (SQL equivalent of SELECT), so you’d just give it the search terms. Here’s some theoretical examples

name="test" AND type="object";-- simple enoughtags HAS "spawn" OR "respawn" IN tags;-- two possible syntaxes for the same thing: see if a-- value is in a listtype="item" AND parent.type="room" AND parent.name="foo";-- using object references, check for any item object-- whose parent is an room object named footype="object" AND weight<1000 AND cost>70 ANDparent=NULL-- more complex example

Of course, the SQL could get pretty ugly for some of those when object references are used. Thankfully SQLite’s typeless columns make the SQL a *lot* easier than would be the case with other RDBMS’. I’d have to use a pretty contrived set of tables and SQL statements to do even the basic things. Still, the inter-object relationships will *not* be easy.

If it proves to be too much effort (given the scope of this project, any noteworthy amount of effort is probably too much to be worth it) I may just disallow object checks like that. Instead, you’d need to break your work into several queries. For the example above, you’d find the room object you are checking for and then submit a query for the item objects with the appropriate parent object ID.

The parse can be written in Lemon, which is a parser-generator written by the SQLite author. (And used in SQLite, of course.) A quick perusal of the documentation has left me with the impression that Lemon is much, much more functional and developer friendly than Bison/Yacc, which is a good thing. In fact, I might even rewrite the Scriptix compiler using Lemon, given what a pain it is to maintain the Flex and Bison components. (And how they *still* are incapable of some of the basic features I want; like being able to call the parser several times to complete a single task, letting me easily do things like have an input mode that parses as you type and can indicate when you have an incomplete or complete statement.)

All in all, I think this is a rather fun and useful project. Who knows, maybe it’ll form the basis of some of the AweMUD file formats in the future. ^,^

Test run output:

smiddle@support02:~/Source/odbm$ ./testValue of 1/foo is string:"bar"Value of 1/int is integer:123456Value of 10/name is string:"baz"Value of 10/cash is integer:2000Value of 99/invalid is NULL:N/ASetting 1/new to string:"testing"Value of 1/new is string:"testing"Setting 1/new to string:"different"Value of 1/new is string:"different"

I Hate SELinux

SELinux is an LSM based security framework for Linux, offering MAC (Mandatory Access Control) facilities. Recently, Fedora Core began using SELinux in the main distribution. Through this, I have been exposed to this wonder of mis-engineering.

First off, I don’t feel that SELinux is useless or a bad idea. Quite the contrary; I’ve been drooling over the thought of a similar system for some time, along with other “advanced” security measures that most free UNIXes (including Linux) have classically lacked.

The problem with SELinux is, simply, that it is horribly designed from a system administrator aspect. The system is flaky. It’s hard to use. It’s slow. It reinvents many wheels. It introduces new concepts and facilities while ignoring those the administrator may already be using. It’s a pain in the ass.

User Roles:

First off, take the concept of user roles. User roles are an excellent facility that allows great access control to the system. The concept of a user role is that while a single user (say, “elanthis”) may have elevated privileges, only some of his processes have those privileges. For example, when first logging in, I’m only a user. My desktop utilities and applications only have normal user access. However, when launching a system administration tool, like say system-config-users, my role changes to system administrator. Now, only that process (system-config-users) has access to modify user accounts. I need to enter my user password to change to this role. This is much better than the typical su-style execution, because there is no root password; jsut my user password. So I can be given access to this tool without being given access to all admin tools and the root acount. It’s better than sudo because the tool itself doesn’t run as root, so any security holes will be limited in scope - only able to do stuff my normal user can do, and edit users (which is serious, but still limited in scope; perhaps imagine if we were talking about system-config-time). It’s better than putting my user in a group that has access to edit the user accounts, because my password is required to change roles and thus malicous code executed in my user role can’t abuse the elevated privileges. It’s better than a group password because again I only use my own password, so there’s less passwords to forget (and thus less need to have passwords written down to remember them) and with the group you have to change the password for all users to stop one user from accessing it.

So, user roles rule. The problem with SELinux? How they’re handled. SELinux has a big binary policy loaded into the kernel that includes users and roles. If you modify a user, you must first modify a configuration file, rebuild this binary policy, and load it into the kernel. The configuration files are m4 source files which also contain a bunch of system information and system-dependent configuration (things that, say, RPM needs to update). This makes it an absolute pain to manage. It’s flaky and easy to break, especially when you need to upgrade the policy; which you need to do a lot, as I’ll explain a little later.

A better solution would be to just use a plain /etc/roles file, with lines in the form ::,,… Each role has system-specified ID that serves a purpose similar to a UID. Access checks then compare either the rid, uid, or both. This is similar to how SELinux works now, except you can’t change role sets without recompiling and reloading the policy, and a SID it computed from the user/role for access checks. This system would work pretty much identically to how SELinux does now from a user perspective, and a lot cleaner from an administrator’s perspective. The framework would just need the newrole command to check this file similar to any of the other user administration commands like su or passwd or whatnot.

File System ACLs:

The next big problem with SELinux is the idea of file tagging. File tagging is when each file is marked with a special attribute telling SELinux what the file is. For example, is the file a Bind configuration file? These are used for file access checks. The tagging must be done manually (or by a script), which runs by reading the configuration files of SELinux and tagging your entire system. Which is slow. And must be done everytime you change your configuration. And, if you want to do anything simple like manually change a file’s tag, and you don’t do it through the “source” configuration files, the next retag will probably undo your changes. Basically, we’ve resulted in the same very-important information being stored in two completely different places (the file system itself and in the configuration files) which, as happens with all the other places this setup exists, is a pain and often causes severe problems. This sort of dual-configuration storage killed linuxconf, it raises a lot of ire again various system administration tools, and now its forming the core of important security tools.

And there isn’t any good reason for this! Why? Because we already have these nifty things called file system ACLs. Which any filesystem that supports SELinux tags will also support. Sure, ACLs don’t support roles (all users of the system administration role can access me, for example) but if you’re going to add non-portable custom tags, you might as well just extend ACLs to support roles. Tools like file managers that already support ACLs can then easily add support for the new semantics, vs now where they probably dont or wont ever understand SELinux tags.

Of course, you don’t want to really modify file access or locations of files that SELinux cares about at all. Not using the same trusted tools you’ve used for the last 30 years, anyhow. Oh no. You must instead edit the m4 macros in the SELinux policy sources, rebuild that, reload it, and then wait a while as your entire file system is retagged. If you move files or modify permissions without editing those policy source files, the next retag will just lose your settings, because the sources can only identify files by path. You can’t move your bind files from /etc/bind to /svr/dns/, for example, unless you remember to (and know how to) edit the policy sources and then retag.

Context Madness:

Of course, ACLs wouldn’t be quite as flexible as the tagging still, because of another bad habit SELinux encourages. Namely, running all system services as root. Oh, don’t worry, it’s safe. Because SELinux has added the concept of contexts. Contexts are a sort of cross between users, groups, roles, and powerful drugs. Contexts allow the SELinux policy to control access; the app running in the bind context will only have the minimum access necessary for bind to do its job. Of course, why do we need the context when you could just make a bind user, run the daemon under that, and use the perfectly usable and well understood uid to control access? What does the next context concept provide you at all besides additional complexity?

The last piece of the puzzle is one that’ll need a lot of work, admittedly. Basically, the SELinux policy’s last goal is to control access to executables and kernel resources. For example, you don’t want bind to execute /bin/bash, ever, under any circumstance. It doesn’t need it. In this case, it could be handled (again) by ACLs; only let users run /bin/bash (role: user) and the few daemons you know need it (user: ). For something like a network socket, however, the management must be done in kernel. Linux already has this facility through “capabilities.” There just isn’t any clear way to define who gets which capabilities. The SELinux policy attempts to provide a mechanism for controlling these but, as with everything else SELinux related, it ends up relying on complex source files and all the misdesigns of the SELinux architecture.

The proper solution to the capabilities situation will probably be similar to SELinux, however. Some set of config files which specificy in detail which users/roles have access to which capabilities along with per-executable settings (perhaps a new system attribute, system.capabilities ?) that define what capabilities are granted. The capabilities of any process in the system would be the union of the executable’s capabilities and the user/role’s capabilities. So a user denied the capability to open remote TCP/IP connections would not be able to do so using any executable, even ones that have those capabilities like lftp. However, when the gconfd executable which hasn’t the capability to exec other executables is launched it will not have this capability ever, even tho the user (obviously) had the capability.

Big Picture:

Looking at SELinux’s architecture and my notes above, you can see the main underlying differences between the two approaches. SELinux puts all policy in one central place. File system access checks, for example, aren’t just between the file system and the process privileges - they’re checked against the SELinux policy. In the implementation I outline above, the file access checks would stay where they were before. Instead of putting the “who can access this file and how” into a separate SELinux module, the access information gets stored right in the file.

Another way to look at it is like SELinux being a third party. When two entities attempt to communicate (a file and a process, for example) they do not themselves decide what to do. Instead, SELinux jumps in and interrupts them while making the decision for them.

And that leads to the problems above. That third party can become confused (it thinks one of the two communicating entities is someone/something it isn’t, because the policy is out of sync with the information stored right on those two entities), that third party adds resource drain (yes, a system running SELinux is definitely slower by a rather disturbing amount, based on recent benchmarks I’ve seen on the Fedora lists), and that third party just increases the total complexity of the system (which in turns usually resuls in more bugs) and increases the amount of knowledge a sysadmin needs to be effective at her job.

Conclusion:

I said above that I’ve been waiting for something like SELinux to come along. A framework for real security measures that provides the facilities the very, very weak classic UNIX security and access control systems lack. Unfortunately, I’m still waiting.

Converted

Not too long after getting home, my little sister had asked by maternal-type parent for “the word processor CD.” I.e., a copy of MS Word that was probably illegal (if it was legitimately bought at all, it was definitely already installed on someone else’s computer). As usual, being the anti-software-pirate that I am, I rather pointed out the stupidity of using a vastly overpriced, underpowered, and illegal word processor. Of course, as usual, I was ignored. I don’t think I know *anyone* (except one person; just one) who actually believes that stealing is wrong. Games, music, business apps, art - you name it, everyone I know has stolen some of it. Usually while completely failing to grasp that they stole it. “Well, Will already paid for that CD, so its OK for us to make copies.” Um, no. You want the CD, you have to pay for you copy. You like that musician, right? Then fucking pay them for their work.

Anyways, back from the rant; said little sister tried to install the illegal word processor but, thankfully, it failed. She then returned in shame to ask about the “free” word processor I mentioned 5 minutes earlier. After checking what her needs were (just a word processor with spell checking and basic formatting) I downloaded and installed AbiWord on her machine. And she’s quite happy because, like I said to begin with, it’s very nice and a lot quicker than Word.

It was kind of funny, as we were installing it, when the GPL license agreement popped up in the AbiWord installed.

“Do you agree to the license, Sami?”"Ya.”"Do you even know what it means?”"Ya, you can’t copy it or anything ‘cuz its illegal.”"Nope. This is Free Software. You can copy it all you want, give it to friends, change it, do whatever you want.” “Why?”"Well, you like it a lot better when you’re allowed to do that, right?”"Ya.”"Well, so do the people who wrote it.”

I think that explanation is probably the first one I’ve come up with that actually got across to her. It’s not Free because it’s not as good or not worth anything. It’s Free because the people who wrote it hate the chains proprietary software forces on you, just like you hate the chains.

And Then I Saw Her

::sigh:: Always the same. “You’re a great guy, I love hanging out with you. But I’m not interested in you.” I really just don’t get it anymore. I think it’s time to buy a cabin in the mountains and just live there alone. At least then I wouldn’t have my hopes raised and shattered by meeting someone who is exactly what I’ve always wanted intellectually, ethically, morally, personally, and entertainingly, only to find out I’m not really what they wanted at all. “The game” sucks.

ZMP

Did some ZMP work today. The two main changes to the spec are the addition of the zmp.input command and the requirement that all implementations support the entire core package (zmp.*).

The former was added after reading a discussion on a MUD development site and how a completely incompetent, as usual, coder was bragging how silly we all were for bothering with newline terminators when he just read in the whole chunk of command and processed it. i.e., he just did a read(). Way to understand networking, or even understand the basic semantics of read(). Anyways, beacuase he was doing this to allow users to paste blocks of input with embedded newlines, I realized this feature could be useful. The zmp.input command basically just used a single argument which is interpreted as a normal line of input. Because its a ZMP command, however, the newline isn’t handled any differently than other characters. Therefor, if the client sends all player input as zmp.input commands, blocks of text containing newlines will just work. Of course, you might want to still explicitly strip those out if you don’t want them. (Which I wouldn’t; too easy to abuse the formatting other players will end up seeing.)

The second change, requiring all of the core package to be implemented, was basically for simplicity reasons. Not simple because now you have to implement all those, but simpler because you can rely on those commands always being present. So a client can just use zmp.input without having to check if its available. Since the core package will remain pretty small (just very minimal commands needed for the basics of MUDing), it shouldn’t be tough to follow.

Finally, I also put the ZMP spec in my arch repository. I rewrote the spec in a custom, simple XML markup and wrote some basic XSLT scripts for converting to HTML, text (actually to a certain HTML output optimized for text translation using Links), and the specific SHTML codes used on the AweMUD.net site.

PString

Yup, as I thought, not portable. The getcontext() stuff I used for finding the stack doesn’t work on OS X (as its not implemented), and the quick hack for locating the data segment doesn’t work either. Plus, as pointed out to me, that would fail anyhow when shared libraries are brought into the mix. (Which wasn’t discovered with my test cases, since no shared libraries were used.)

All in all, I’m going to need some library to find these for me. The libgc hackers think my proposal for the API additions was interesting, which is good. Now just to get them actually added. I feel I may end up having to do this on my own. Oh horror. ~,^

Event Model

Working a bit more on the AweMUD event model. Moving events to work on a queue basis. Basically, when an event is triggered it isn’t processed immediately, but added to a queue instead. This is step #1 in avoiding the possibility of an event loop locking up the system (step #2 would be for queue additions made during queue processing to not be processed until the next game loop). Additionally, this’ll also help in my long-term goal of getting a solid threading model in Scriptix to allow very nice, complex background/concurrent threads. Probably using a library like GNU/OSSP Pth. (Assuming the license is usable; havent’ checked that yet.)