Technical Ramblings

I’m looking to find out whether anyone has any desire to see new applications based on RDF. What would you like to see that hasn’t been done yet? What do you think would be a cool demonstration?

This can be web apps, local apps, whatever. Most of my stuff these days are in Python, most of my RDF stuff is based around Redland (when it’s not dying because of something I broke) and most of my platform-specific knowledge is Linux based.

Given these constraints, what “cool thing” hasn’t been done to pieces yet? Would a redux of Plink be in order, perhaps with better tools for automatic information removal to avoid the situation of angry users? A web service allowing you to use julie’s database for queries? Mail filtering? Image searching by all kinds of fun stuff, a la MortenF’s application, only with a user-upload section?

I really am low on ideas at this point, so I’m interested in hearing what thoughts other people might have.

So, a long time ago, I posted about how I planned to update the FOAF that LiveJournal spits out to become more friendly, since I really didn’t know anything about RDF at all when I was first creating the format. So, I wanted to do some updates, to match things that I had discussed with members of the FOAF community over time.

However, I got a real job, lost my free time, blah blah blah, these things happen. The patch has sat around in various incarnations for at least 6 months, probably longer. Recently, someone asked me what had happened to it, and I responded that I had gotten tired of pushing for changes to FOAF, and given up. He said he had time to spare on pushing people, so I cleaned up my patch, and posted about it – a detailed rundown of the changes the patch makes, for evaluation by the community.

Since the community involved is larger than LiveJournal, I’m here to encourage absolutely anyone who might have an interest or knowledge in the arena to read through the explanation or the patch, and let me know what they think. I really would love to see this happen, but like I said, I’m not going to fight for it anymore: it took a lot of hard work to get the previous FOAF patch accepted, and I just don’t have the energy or free time to do that again. So, this one is going to depend on community feedback and demonstration of interest. If you think that LiveJournal’s FOAF data might be suboptimal, say something about it. That entry is a great place to comment and discuss. Open a dialouge with developers about what you’d like to see, because if enough people want to see it, someone may actually push it in.

I posted recently about Zeroconf, which allows for automated service discovery. One aspect of this which has already been exploited is the ability to have a local network of presence information, something like what iChat does to announce users who are online. However, it also passes along some simple FOAF information with it. I can’t remember who wrote the tool. If I recall correctly, it was shellac/Damien… ah, here we go. foaffinger.

Small HTTP server combined with zeroconf shares data.

Now, to expand this out a bit… now that I’m starting to know how to deal with Python a bit better, I might be able to do something with pyzeroconf and pyrple. I’d probably want to learn something about graphics too… wxPython requires an install. Are there any “built in” GUIs for Python? I don’t think so.

Anyway, local FOAF browsing of full data rather than just a subset would be nice. Not too hard, I don’t think, since Python makes almost everything simple. Of course, there is the problem that there’s no one that I share my network with… 😉

Ah well, I do have 2 macs, 2 linux boxes, and 2 windows laptops in the house now. (Wow! An even split. Nifty.) I can try it out somehow or another.

I’m kind of rambling. Usually I’d reserve this for noets, but I wanted a tie in from the Zeroconf post I just made.

My pet Python project is redlandbot, the most popular manifestation of which is julie, an irc bot. Basically, the service allows for adding data from a URL to the local model, and then running RDQL queries against that model. As the name implies, it’s built as a layer on top of Redland, using the Python language bindings.

One of the biggest complaints about using julie, or at least, one of my biggest complaints, is that it requires namespaces to be used explicitly in some cases. Or should I say, that *was* was one of my biggest complaints: I just fixed it.

From now on, julie will automatically add namespaces to a query. It checks if there already are any (by looking for ” using <"), then adds the rest of the predefined namespaces, trying to match syntax. There may be some commands which break because of multiple spaces between "using" and "<". If there are any, feel free to fix them by simply pulling the namespaces out of the command: they will be added automatically. This allows for more complex queries with less work, which is always a good thing. Any bugs can be reported to me. This has been on my wish/todo list for way too long, and it's good to have done. I should really work on fleshing out julie/redlandbot as a web service a bit more, I think. It'd be nice to be able to just set up the service on a webpage, and use the ircbot as a very high level transport between IRC and the web interface: this would allow me to move the bot around a bit more, without needing to install Redland to use it (occasionally a bit of a pain, due to dependancies, mostly because I suck at resolving them without root).

The response Sean, Cody and myself have had to RSS 1.1 has been mostly positive, and we’ve gotten a number of suggestions for ways to make it better. We’re going to be working over the weekend to implement these changes into the schema, and releasing a second draft on Monday sometime.

The response was, all in all, pretty good: We’ve had some good people on our side, offering encouraging comments on the specification. At the same time, there is the all too common “Oh no, not another RSS version” that you get in the case of anything new. I can honestly say that I think most of these comments misunderstand the goals that were being worked towards by RSS 1.1.

If you are not using RSS 1.0 right now, do not worry about RSS 1.1. It is designed only to fix the bugs that RSS 1.0 has, due to the fact that it’s 5 years old. If you try to drag me into politics: whether it be on whether another RSS format is good or bad, or on other things which are unrelated in a change from 1.0 to 1.1, you are misunderstanding the original intentions behind this format.

It is a possibility that given a better defined specification, using the current version of RDF rather than the one that was available 5 years ago, some people who are not using RSS 1.0 now may want to make the switch to RSS 1.1 (if it achieves support in the community). If you do, great. If you don’t, also great. I have no interest in converting those of you using RSS 2.0. I have no interest in telling you to stop using Atom. I want a technical improvment over RSS 1.0: and RSS 1.1 does that.

At the same time RSS 1.1 was designed to fix bugs with RSS 1.0, it was also designed to make it easy for aggregator developers to develop support for it. Complete backwards compatibility, although wonderful, would have been so limiting as to make any changes to the schema useless.

RSS 1.0, although it still had an existing working group, was not being developed further. To work in the working group atmosphere would most likely have stifled creation of the new specification. Although some may think this is a bad thing, I think that RSS 1.1 does fit the needs of a small group of people without putting an ardurous strain on the people who develop tools for it.

In the end, lots of people will be unhappy with any new syndication format. The motivations behind this move, however, are entirely technical, rather than political, so, as Sean said at one point, most political arguments against it “concern us not”. The technical questions, on the other hand, do concern us, which is why we’ll be working to craft a second draft as quickly as possible, to get further feedback from the community.

As a developer, for a long time, I believed RSS 2.0 to be significantly superior to RSS 1.0. It was a more popular format, more people had heard of it, but most importantly, it was easy to create. In comparison to RSS 1.0, creating RSS 2.0 documents was downright trivial. There was no need to have the goofy “rdf:RDF” tag at the top, with all those weird looking URIs for some reason no one could tell me. Clearly, at the time when I was thinking this, I was not aware of RDF or the benefits of it. As time went on, I realized the benefits of the RSS 1.0 way of doing things, most importantly, that the content is RDF.

However, RSS 1.0 is still a relatively confusing format to implement to a non-RDF aware developer. The “table of contents” listed in the rdf:Seq is redundant, and to a lot of people who are used to working in XML and not under the constraints of RDF, confusing. The documentation on all the RSS formats is quite weak. A table comparing what each one offers is hard, if not impossible, to find, especially since most such lists are biased towards the writer of said list.

As I’ve become more and more involved in developing tools relating to RDF and the Semantic Web, RSS 1.0 has become more and more of a sticking spot in my experiences with people. They are soured by their 1.0 experience: a feeling I can not blame them for, as RSS 1.0 is constrained in many ways by the circumstances under which it was developed.

Fed up with dealing with a less than optimal format that was truly leaving people feeling negatively towards RDF in general, I decided that I was going to work on something that would be better for RDF and RSS. I was just going to write it myself, but luckily for me, I had the help of a couple of friends.

sbp, and myself, along with many a helping hand from d8uv, have written a specification we are labeling RSS 1.1. From the Specification:

This specification is therefore made available by users of the RSS 1.0 format who wanted to update the specification to make use of the latest features of RDF in order to reduce the redundancy in the format, and the ambiguity in the specification, while at the same time implementing a series of bugfixes from the lessons learned in developing the other descendent of RSS 1.0, Atom.

This specification comes with a full Test Suite, a validator, and a number of other resources, from background information to implementation details in several different types of configuration.

We have formed a channel for discussion of RSS 1.1 related issues on Freenode: #rss1.1 on irc.freenode.net. If you are interested in using the specification, or would like to offer your opinions, you can stop by there to talk to us.

This specification is not designed as competition for Atom: it is designed only as a bugfix release for RSS 1.0, and is not designed to compete with any other existing efforts. There were a significant number of issues with RSS 1.0 that had been raised over the years, some of which were relatively important from the standpoint of a lazy developer such as myself. Hopefully others will feel a similar way.

Many of you who know me on IRC and other similar places will know that I have a wonderful partner-in-crime, Jessica. Together, we are rel:parentOf to Alicia and Julie. Julie, unfortunately, has a bad ear infection today, and is therefore getting ice cream for dinner, per doctor’s orders.

Julie decided that she wanted to come with on this trip, despite having a fever of 103. She said “I’m four, I can come!” Of course, this request was denied, at which point she repeated her assertion.

Repeated assertions reminded me, naturally, of RDF, so I expressed her statement in RDF: :Julie foaf:age “4”. (Which, verbally, is “Julie foaf:age 4”.) Our roomate looked askance at Jess and said “Ya know, he’s going to express your wedding anniversary in FOAF so he can’t forget.” Jess’s joking reply was “One of these days, I’m going to break up with him because he can’t stop whispering sweet RDF nothings into my ear.”

My response, naturally, was :ChrisAndJessBreakup log:implies “sad”.

(All terms, RDF and otherwise, in this entry are considered non-normative and non-validating. Do not use data from this entry for the basis of forming real RDF documents. I don’t understand rules, and I have no idea what log:implies does, so I probably used it wrong. But it was funny.)

Earlier, I mentioned how I was going to be switching to Flickr for Image storing. Although this is still the case, I’m waiting a bit until I do: I want to write some tools for generating RDF for me personally before I spend the money on it, I think. There’s already some work being done: Masahide announced a flickr2rdf service of his own, but it’s generalized, and I want to do something a bit more specific. I want to be able to describe people in the description, and have a tool which automatically extracts it. Not quite “Natural Language Processing”, something I’m sure that Arnia will chide me for, but something along those lines. Maybe even just using n3 in HTML Comments? Wonder how the API deals with that…

However, in the meantime, I’ve installed Gallery2, and I must admit, it’s very nice. I’ve been working in the module framework, and I’ve been able to extract exif information and include it in an RDF RSS Feed. Next step is to go back through my steps and look at all the changes I’ve done, then export it so it’s a usable format for the developers, and integrate it into their RSS 2.0 development. Triples, triples everywhere!

Once I’m sure my Gallery is in a usable state, I’ll release the work I’ve done on it. For the time being, I’m making sure that I have something I’m willing to switch to before I publicize it.

Hope everyone has a good weekend!

I’m an open minded kind of guy. There are a lot of services out there, and even though some of them aren’t open source, it’s possible that they may do what I want them to do. One of those services is Flickr, a photo sharing service.

Flicker does a lot of very nifty things: updates from anywhere, advanced annotations, including an extremely easy to use Javascript interface for annotating regions of images, and posting to blogs directly from the service. However, that’s not the coolest aspect of the service, in my opinion.

Flickr provides a relatively advanced, full featured, well documented API – a way to get pretty much any information you might want out of the site without screenscraping. (LiveJournal, in comparison, strongly discourages screenscraping, preferring that you use services listed on their bot-friendly services list. However, the data afforded through these interfaces is extremely limited to the point where it’s unusable for most advanced tasks.) Through this API, you can retrieve all the information you want about the people and photos available through the service.

This is especially interesting to me as an RDF nut because it means that I can use Flickr’s nice interface and handy annotation tools – and at the same time, I can convert the data, via the API, to an RDF format that I can use for all the things I’ve been describing in my Image Description posts.

The limits a free Flickr account places on you are kinda strict: relatively small upload limit, given that I prefer to store full size images in the photos I already have in my personal gallery. I’d immediately set Flickr aside months ago because there was no way I could use it to store all my images. However, upon review tonight I discovered that an annual Flickr account during their beta period is only about 45 USD. Included in this is:

• 1 GB monthly upload limit
• Unlimited Storage
• Unlimited Bandwidth

In addition to a few others, listed at their upgrade page.

It’s a case where I can build lots of tools and do lots of work myself, and get exactly what I want… or I can use flickr, pay a pretty minimal fee, and get 90% of everything I want with no effort, plus an additional bit of work to get that last 10%. I’ll probably still keep things locally (if for no other reason as a backup should flickr ever go poof), but move my primary photo gathering to be flickr based.

I think I know which way I’m going to go. Once I do, I’ll keep you all updated on the progress I have with RDF.

Typepad exports RSS 1.0, as well as FOAF. LiveJournal exports limited FOAF information because that’s really all I could squeeze out of it the first time around, when bandwidth and load time was a major concern. I wonder if with the possible buy out of LiveJournal by SixApart, some of the LiveJournal specific XML formats will change to RDF…

For example, the Latest Images from LiveJournal is an XML format… it might be interesting to see this as an RSS feed, especially RSS 1.0. Perhaps the creation of feeds for the LiveJournal photo gallery option…

I can see a lot of room to expand the semantic content that LiveJournal emits, and I wonder if Six Apart might be in the right place to do it. As far as I’ve seen, they’ve seemed interested in doing so with Typepad. Does anyone know if something similar runs through more of their work? I’d love to see more RDF coming out of LJ. It’s a large data source and so much of it goes untapped because getting anything into the development process is very difficult.

Mm, LiveJournal sized image annotation stores… advanced semantic web development at its finest. Now that’s something I’d love to see. Imagine being allowed to check which LiveJournal users were in a photo, and then having that data emitted as RDF… allowing searches over it, finding photos with more than a certain number of people, photos of places. Geolocation. The UI isn’t hard, it would just be a bit of hackwork to get it together.

If there’s one thing I hope for, it’s that this leads to quicker turnaround on LiveJournal development time. Whether it’s with volunteers or not, I’d be willing to double my annual cost if it meant I got to see new features within a year, rather than three. That kind of interface would bring such a boost to the semantic web…

Now I’m tempted to build an external tool myself, just to demo how cool it would be.