Technical Ramblings

A new crschmidt.net webservice:

lynxview, converting based on a domain name to a lynx -dump form. For when you want to show some windows user just how crap their website is with all the graphics turned off.

As a form of demonstration, check out crschmidt.net or Planet Mobile.

Currently, sites are cached eternally, so that the service can’t be used to DDoS some poor site.

Produced in part by a request from DanC on #swig earlier today.

So, for the first time this weekend, I’m on a network where outgoing mail on port 25 is blocked. How annoying.

I use a number of mail servers in a number of different ways. Typically, when on one of my Linux boxes (zeus or athena) I’ll send mail directly from those servers by using a localhost Postfix installation, and no smart or relay hosts. I don’t really see a need for my ISP to see my mail, and doing it this way is the default setup for most Linux distros that I’m aware of.

If I’m someplace that doesn’t have a mail server (ie the powerbook, creusa, or the mac mini, hermes), I use athena as a mail host, on which I have installed SASL authentication. Athena is set up to accept mail in a couple cases:

1. Mail from local network. This includes all the IPs in my block on Sagonet.
2. SASL Authenticated users: This users password authentication against the local mail database to check users who can login to the server to send mail.

As such, the server is protected against being an open relay (so long as I don’t get a spammer on the local machine, but I don’t think that’s going to be the case), and I like having it there as a backup for when other mail servers fail me. wedu’s mail uses POP before SMTP for authentication, which is all well and good, but can be a pain since the logs are reset at :45 past the hour, and if you try and send mail right after that, you get a nice “Relay denied” message.

In any case, I tried to send mail this morning via crschmidt.net… and got a timeout. Tried getting there from here, no go. Panicked a bit, since this is my main mail server, and if it’s down, that’s a bad thing. Tested it from zeus: no problems. Tested it locally: no problems. Tried going to another port 25… problems. So it’s on the Ameritech end. Great.

Set up an ssh tunnel: ssh -L 25:localhost:25 crschmidt@crschmidt.net. Set up a server in Mail.app as localhost port 25. Forward my mail. Sigh at Ameritech. Bitch in weblog. And the circle of life continues.

Yesterday, after some discussion regarding Bluemoon (currently offline, LiveJournal syndicated copy available at livejournal temporarily), the idea of a “Planet Swhack” was brought up: an aggregated collection of the weblogs of members of #swhack, much similar to the many other planets run by the Planet Planet software or like Planet RDF, run off the Chumpalogica aggregator.

So, yesterday, I set it up. AaronSw controls Swhack DNS, and wasn’t around at the same time as me at any point, so I set up as a temporary URI to demonstrate it. Picked up some bloggers, and set up the stylesheet to be the same as my other Planet, PlanetMobile. Tonight, as I was preparing to ask AaronSw to set up DNS for Planet Swhack, I noticed that jcowan‘s most reccent entry was messing things up. I looked into the issue a bit more and found out that Planet was using version 2.7 of Mark Pilgrim’s Universal Feed Parser, which barfed quite badly on the XHTML in his Atom content.

So, I looked into it a bit more, and found out that the “nightly tarball” of Planet has not been updated since October. So much for any kind of decent release schedule.

Looking at a mailing list thread on release scheduling, I realized that the issues I was having had been fixed, and set about to check out the latest code from their version control.

Except there’s no instructions on how to do that, just a repository name. And it’s GNU Arch, which I sure as hell don’t know. So, I go to install it… apt-get install tla, on my home machine… apt tells me:

Media change: please insert the disc labeled
‘Ubuntu 5.04 _Hoary Hedgehog_ – Preview i386 Binary-1 (20050310)’
in the drive ‘/cdrom/’ and press enter

Update: Since I get a large number of hits from Google for this issue: The way to fix this is to edit your /etc/apt/sources.list, and remove the first line in it that references the cdrom drive. (You can simply put a # in front of it.) Then type apt-get update. (You’ll have to edit and update as root – type sudo before the commands to do that.) If you need more help, feel free to comment. (2006-01-10)
… right. I’m a 15 minute bike ride from home. Not going to happen. So, I switch from zeus to athena and try it, and get tla installed. Then start looking for instructions on how to check out a repository.

Apparently, the industry standard term “check out” is not part of the Arch repository system. Eventually, I wandered into Logjam’s arch repository, which provides clean instructions for how to get the code out of it:

tla register-archive http://logjam.danga.com/arch/2004
tla get logjam@danga.com–2004/logjam–dev–4.4

I was able to check out the “shiny development branch” of the Planet code, and get it in place on the site, fixing all my issues with Atom and XHTML content. All is well in the world, and Planet Swhack is a go. Never let it be said that checking out code from an arch repository is intuitive though. Anyone who thinks it is is out of their tree.

Earlier, I posted about extracting SVG metadata with Redland. However, one of the problems with this is that there isn’t a whole lot of SVG out there, nor is there a whole lot of SVG with metadata out there.

One solution to this is the OpenClipArt Library – thousands of Public Domain SVG images with embedded metadata, totalling a heck of a lot of RDF information that could provide an interesting example of how RDF information can be used in real world scenarios.

However, the metadata provided by this library was, when I looked at it, broken RDF. I sent an email to the clipart list explaining the problems with their metadata, and received friendly and helpful replies letting me know that the data was generated with the SVG-Metadata perl library.

This weekend, I downloaded that code and began working on it, submitting a patch to the maintainer (who is also one of the founders of the Inkscape project, and works on OpenClipart), which was integrated today, improving their license support (now supporting all Creative Commons licenses) and their RDF output (such that it validates).

A new version has been released, uploaded to CPAN, and will soon be propogating its way to the CPAN archives. New SVGs uploaded to openclipart will contain metadata which is valid RDF, and Bryce is looking into regenerating the data on older SVGs as well.

More RDF. Better metadata. That’s something that I think I can live with.

How to Parse SVG Metadata, the Redland + Python way:

import urllib
import xml.dom.minidom as minidom
import RDF

m = RDF.Model()
p = RDF.Parser()
u=urllib.urlopen(“Location Of SVG File”)
svg = u.read()
doc = minidom.parseString(svg)
p.parse_string_into_model(m, doc.getElementsByTagName(“rdf:RDF”)[0].toxml(), “Location of SVG File”)
print m

In other words: Bring in the RDF and minidom modules, Create an RDF model and parser, download the SVG file to a string, parse the string into a minidom compatible variable, then look for RDF in the SVG file, parsing it into the model, and serializing the model.

Problems: What if someone uses something that’s not rdf: as the prefix?
Solutions: mattmcc offers that minidom supports getElementsByTagNameNS, so the parse line would become:
p.parse_string_into_model(m, doc.getElementsByTagNameNS( “http://www.w3.org/1999/02/22-rdf-syntax-ns#”, “RDF”)[0].toxml(), “Location of SVG File” )
resolving the Namespace issue.

Of course, since this is Redland, this is taken care of for you. Rather than doing it in this way, which is specific to SVG, we can scan for RDF in any XML doc. Simply:

import RDF
m=RDF.Model(); p=RDF.Parser()
p.set_feature(“http://feature.librdf.org/raptor-scanForRDF”, “1”)
p.parse_into_model(m, “URL Of SVG File”)

There are a number of other features you can use with a Parser. They are available via rapper -f help, but here’s a list: assumeIsRDF, allowNonNsAttributes, allowOtherParsetypes, allowBagID, allowRDFtypeRDFlist, normalizeLanguage, nonNFCfatal, warnOtherParseTypes, checkRdfID.

Naturally, Redland already does what I want it to do. Another pat on the back for Dave (and thanks to him for pointing it out).

Lately, I’ve been playing with SVG, since I finally got it to work decently well on two of the computers I regularly use. I was able to get it working on a Static FOAFNaut even, which is motivating me to actually write a few more tools in Redland to get FOAFnaut working better. I never realized that much of the speed problem with FOAFNaut before was that it was dynamically parsing RDF in Javascript, which is not fast, rather than something related to the actual SVG rendering, which is actually pretty quick.

With help from #svg on freenode, I’ve got SVG running with a prerelease version of an Adobe plugin on my Linux box, and I’ve had it for a while on Firefox on the mac. I’m really looking forward to the release of Firefox 1.1 now though: having built in SVG support will lead me to be able to try out some pretty neat stuff, and maybe pull a few more people over to Firefox in the fray (if the engine isn’t crap, at least).

SVG is, all and all, pretty cool. I’m probably going to add support for parsing RDF out of SVG files to julie once I get my DSL line problems fixed and start running her again. Yet another source of data… such nifty stuff to be done.

For those who don’t know: SVG is kind of like a standards-compliant version of Flash. It stands for Scalable Vector Graphics, and it lets you describe how to draw things in terms of curves and lines, rather than by specifying the pixels. This means that you don’t get blurriness at any size you look at it, unlike rasterized formats. It’s kind of like comparing Adobe Illustrator files to flattened Photoshop files, for those of you who are familiar with such things: one can be stretched at will and not look odd, whereas the other is just not going to react so well to that. There’s still some issues I’m having with them in the “embedded in web pages” way, but that may just me not knowing how to deal with stuff.

For Linux and Windows SVG authoring, there’s Inkscape, which seems to be a simply fantastic piece of work. Illustrator can also export to SVG, and I’m sure there are other tools which the lazyweb can share.

All in all: SVG is cool, and I hope to do some work with it in the near future. I’m happy to hear anything about success stories you may have had so far.

Hula – Open Source Groupware for the Corporate Soul.

There’s lots more out there on all aspects of this than I care to even think about, but some of the things that Nat has written about it in his weblog sound pretty cool. Personally, I have only one wish for this product: for it to become something that I can use in the workplace for task/appointment management. Currently, we’re in a 20% Windows/80% Mac world, but all the appointment management is done through Outlook and ACT!. The number of times I’ve heard the people who use it cursing ACT! is way too high, and it has the obvious problem that the rest of the office can’t use it.

I need to learn to accept that not everything is going to validate, and I need to learn that hacks and workarounds are a way of life. I need to accept that Javascript really is here to stay, and do more to learn how to make good Javascript applications, because really, they can make the entire user experience way better. Software isn’t about perfection, it’s about doing the simplest thing that could possibly work, and making things better for the user.

I wonder what I could start with… a project to learn some of the more advanced Javascript technology, while still keeping the interface usable for non-Javascript enabled browsers, allowing for linking, and so on. Google’s interfaces don’t work so well for this: Gmail, Orkut, and Google Maps all fail miserably without Javascript. I don’t like that.

Ponder ponder…

I posted recently about Zeroconf, which allows for automated service discovery. One aspect of this which has already been exploited is the ability to have a local network of presence information, something like what iChat does to announce users who are online. However, it also passes along some simple FOAF information with it. I can’t remember who wrote the tool. If I recall correctly, it was shellac/Damien… ah, here we go. foaffinger.

Small HTTP server combined with zeroconf shares data.

Now, to expand this out a bit… now that I’m starting to know how to deal with Python a bit better, I might be able to do something with pyzeroconf and pyrple. I’d probably want to learn something about graphics too… wxPython requires an install. Are there any “built in” GUIs for Python? I don’t think so.

Anyway, local FOAF browsing of full data rather than just a subset would be nice. Not too hard, I don’t think, since Python makes almost everything simple. Of course, there is the problem that there’s no one that I share my network with… 😉

Ah well, I do have 2 macs, 2 linux boxes, and 2 windows laptops in the house now. (Wow! An even split. Nifty.) I can try it out somehow or another.

I’m kind of rambling. Usually I’d reserve this for noets, but I wanted a tie in from the Zeroconf post I just made.

There’s not a lot of webpages out there on the neat things you can do with Zeroconf. For some reason, this surprises me. I feel like it’s a pretty interesting technology, but it doesn’t seem like anyone’s done much with it, despite the fact it’s been around for quite a while. For example, AaronSw wrote about zeroconf more than two years ago.

For those who may have missed the boat, the primary thing that I mean when I talk about Zerconf is automated service discovery (DNS-SD). This is the kind of thing that allows iTunes to see the other computers locally that are also sharing iTunes music. Apple has put a lot of work into making this stuff simple, to the point that it’s one of their more popular open source releases. (Their mDNSResponder code, that is.) I know that Gnome is putting a lot of effort into making things like this work, but I don’t run Gnome (I’m a minimalist) so I don’t know where that’s really at.

Earlier tonight, I was having some problems announcing services over Rendezvous, which is the only way that iTunes will attempt to talk to a DAAP share – there’s no way to tell iTunes “go to this IP address”. This is probably by design: this way I can’t open up someone’s music share that’s not announcing it locally. Kind of annoying for me, since I’d like to be able to play my music from home at work, which I would be able to do with DAAP otherwise After some trouble, I eventually got my machine to announce the iTunes share (which is done via mt-daap) on my Linux box, so I can now play that music. I also added a few other Services: HTTP and SSH. Still, I don’t really know what I can do with these. Why doesn’t putty or some other app capitalize on this by offering to search local hosts for SSH? Does no one really use local services that this could make easier?

My Zeroconf stuff comes from Howl, which offers a multi-platform open-source solution to providing Zeroconf services. It’s a decent library, with decent tools. It is a bit iffy on the user-level: The error messages are pretty unspecific. For example, if you try to announce a service before first running mDNSResponder, it will give an error message that it is unable to connect: but won’t tell you where or why, nor is it mentioned in any documentation (that I can find). The code itself seems solid, however, and is working well here. So, what cool ideas are there for Rendezvous that no one has done anything with? And who’s going to step up to the plate to start?

One example I can think of: set up Rendezvous to announce your local MySQL servers. Then, if you’re in an office where you do development, you can just pick from a drop down of development servers: you don’t have to depend on static IP addresses, you can just scroll through a list. You can even use the text fields of the Rendezvous to include a Guest login. Seems like something that would work well for CocoaMySQL. Of course, it doesn’t look like it’s been touched for more than a year, so it’s not likely to change in the near future. Just an idea.

What zeroconf does not do, which many people don’t seem to understand, is provide any real help in actually creating the connections. Lots of people, myself included, seem to think “Rendezvous” is this magical tool that does all the network communication for you. (That is, I used to think that way – I know better now.) Really, all it does is give you an IP address – nothing that a relatively tech savvy person couldn’t do on their own, at the application level. The communication actually takes place over the typical channels – regular socket communication.

So, although Zeroconf/Rendezvous is cool, it’s not an “end all/be all” to network communication. Just something to keep in mind.