Technical Ramblings

Chris: 2, Redland: 0

For quite a while, I’ve been having some problems with installing the latest version of the Redland RDF Application Framework. I’m sure that these were problems of my own: I know that my system is a little goofy, in part because I’ve got a combination of ebuild-built and non-ebuild versions of things. So, for a while, I’ve been unable to compile the lateset Redland, because whenever I’d try and compile, the system-wide redland-config would get in the way, and then Redland would try to link against the already installed libraries, etc. etc.

However, yesterday I started having issues with julie crashing, returning a glibc exception. I wrote a small test script, and found that the issue was occuring any time I was using two “AND” clauses in an RDQL query. I recompiled glibc with debug info this morning, and was amazed to find that my backtraces suddenly became much more informative: Instead of gdb returning ?? in ?? in my backtrace, it actually gave me function names and line numbers. I didn’t think that just recompiling glibc would have helped so much.

After I was able to do this, I sent mail to the Redland-dev list: Here’s my problem, here’s my backtrace, where do I go from here? That Email got several responses back and forth from Dave Beckett (maintainer of Redland) and Morten Fredrickson (maintainer of the MySQL storage backend). I ended up stopping all services which used Redland, uninstalling redland completely, and making the latest release successfully. In combination with discussion on IRC, I moved onto testing with valgrind, an open-source memory debugger for x86-GNU/Linux and ppc-GNU/Linux. Sending the data dump from that helped Dave to find the problem, which was related to the way that the mysql backend frees results.

One patch later, the problem was cleared up. There’s still the question of why it broke, which seems to be related to some broken statements in my database. However, it’s good to know that I helped to find a bug in Redland, rather than just finding something broken locally.

In the end, I did learn a few things:

1. Having glibc debug symbols on is a Good Thing.
2. If you can’t install the latest version, try uninstalling the previous version first.
3. valgrind is an app which may help you locate the source of bad free() calls
4. glibc has some extra sanity check memory detection: you can adjust its behavior via MALLOC_CHECK_. Levels of sanity are 0->3, with 0 being “display error message and kill application” and 3 being “do nothing”. (I found this via fedora core release notes.)

I also was able to compile new Redland bindings: this means that I fixed my Perl Query problem (which has been fixed for a while, but I couldn’t install new bindings), as well as adding Sparql support to my local Redland tools. Now, I just need to write something that uses it.

So, in the end, I helped squish a bug, and upgraded all my Redland related stuff on my system. That’s what I call a good day.

I have about 1500 images that I’ve taken over the last 1.5 years since I got a digital camera. These photos vary wildly in what they contain, from parties with friends to buildings to family photos to stuff around the house. There is a variety of images depicting people from all walks of life, from my college friends to my family now.

I want to search them.

I’ve become very convinced that the best way to store the information about these photos is in RDF: It’s the only data description framework that lets me extend and expand my descriptions to include everything I need. (There’s also the fact that I’m an RDF nut, so everything is a case for RDF.) Once the data is created, adding it to an RDF store and asking that store questions will not be difficult; the main problem is to get the data created.

There are some web based tools for photo/image annotation: mortenf uses a tool he created which spits out ntriples, and there is also the java app attached to the foaf codepiction project. However, neither one of these lets me describe my images very easily; There’s a lot of effort to go through for it.

My ideal photo annotation tool works like this:

• Provide a URL to a photo.
• This photo may or may not have EXIF data included. If it does, use the exif data to generate the appropriate RDF; something along the lines of the EXIF metadata created by Mashide’s exif2rdf web app would be good.
• Allow me to describe the image with a title and description: dc:title and dc:description.
• Allow me to describe the person who took the photo: foaf:maker. This part can work from the existing triple store I have; Let me enter a name, and then use that name, along with an mbox_sha1sum, to describe the person.
• Offer me a license list: a list of CC licenses to choose from.
• Allow me to describe “things” that are included in the photo: entering a term and checking it against wordnet, then using the right sense of the word.
• Allow me to describe “people” in the photo: enter a name or nickname, and then present me a list of options from my triple store, or the option to describe a new person. Once I choose a person, a name + IFP should be included in the output so that I can smush the data together.

I’d like for the entire interface to be text based. I don’t want a web or graphical app: with 1500 photos, that’s just too damn much. I want to be able to describe these things quickly and easily, so very little should get in the way, with as few keystrokes as possible. If I want to skip something, I can just hit enter to skip over it.

I’m lucky in some respects, because some of the hard work of gathering information about people is already done for me: I have a large triple store at hand that can help me with this project. I also have some limited experience in working with the Redland framework, so I may even have the experience I need to make this work. So, here’s how I see it working on the inside:

Create an interface along the lines of what Config::Tiny (in perl) does: Question [default] ? then allow for the user to either skip past (and use the default, which may be none in some cases) or to use their option: if their option, possibly provide a list of choices after that. For example, if I enter “Jessica” as a name, it should tell me “There is no name Jessica in the database: would you like to create a new user?” with a yes or no option; if no, it would go back to the previous step and let me enter a different name. Dump the data, as its created, to an in-memory Redland model. Once the photo description is complete, serialize the model to a file, which can then be run through cwm to clean it up for more permanant storage.

I’m probably going to actually get off my butt and write this pretty soon: I have a lot of interest in describing this data, in part because it’s another case where a nifty project can lead to a great demonstration of why and how RDF can work. I’m most interested in someone who may have suggestions on what kind of tools would be good for creating the interface: the last time I did any kind of user input other than command line was when I was working with cin>> statements in C++. I’m probably going to write it in Python (I feel that I really accomplish things much more quickly writing in that language than in either perl or PHP), so tips on how to create command line user interfaces in Python are appreciated.

A number of people have expressed interest in my posting some of my more technical ramblings as a seperate resource to the oftentimes personal weblog I’ve used in the past, my LiveJournal. Because I’ve always been one to aim to please, I have decided that I will seperate my more personal postings from my technical ones, although not because I mind people reading my personal life, but more because I want to ensure that people can read what they want and not feel discouraged, as well as ensuring that I can post in specific places and have people read it only if they want to.

Of course, this entire problem could be alleviated if LiveJournal ever got a decent categories system in place, under which people could subscribe or filter to certain categories, however, I do not think that this is a very likely occurance.

LiveJournal has long been my “home” on the web. It offers a number of features that I don’t understand the lack of in other tools: for example, the emails for comments act much like any other mailing list which takes place through email, something that I’ve never seen another blogging system do. However, as the author of this weblog, this is less likely to be an issue: there will probably be few “threads” back and forth, and even if there are, I will receive information on all the comments being left.

Perhaps one thing to do with this new blog is to make communication be a bit more like a mailing list; email subscriptions to posts, so that when they’re updated or a comment is left, people can tell.

I’ve chosen a relatively odd permalink structure for this weblog. In part, this is to prove that it can be done: Many people do not realize that WordPress offers the ability to completely change what your permalink structure is like. In part, it’s simply to shorten post URIs: with only %postid% and %posttitle%, I’m keeping it simple, but still unique from both the %postid% perspective, and informative to the human who may be dereferencing the URL manually.

Expect me to add categories, links, plugins, and possibly change the style over the next few days. If anything I change is something that you feel makes this weblog less usable, please let me know.

The weblog that I was using previously will continue to be available, although I will no longer update it. From now on, posts go to LiveJournal or to here, depending on their nature.