Technical Ramblings

Earlier today, I tried Jython for the first time, because I’m doing some work that may involve interactions with Java libraries in the near future. Jython, which I’ve always avoided in the past due to an irrational fear of Java, is “an implementation of the high-level, dynamic, object-oriented language Python written in 100% Pure Java, and seamlessly integrated with the Java platform.” (I love projects that have great one-liners that I can copy paste.)

My goal for Jython was to do some work with the GeoTools EPSG registry code related to SpatialReference.org. Sadly, I didn’t get that working, but in the process, I learned that Jython now has a beta version which is up to Python 2.5 — much newer than the 2.2 that had previously been available.

With that in hand, I decided to see if I could get some of my other Python projects running under Jython. I’m the maintainer for both TileCache and FeatureServer — two pure Python projects. Theoretically, these projects should both work trivially under Jython, but I’ve always had my doubts/fears about this being the case. However, it turns out that my fears here are entirely unfounded.

I downloaded the FeatureServer ‘full’ release from featureserver.org: this includes the supporting libraries needed to get a basic FeatureServer up and running. I then tried to run the FeatureSever local HTTP server… and it worked out of the box. I was able to Load the layer, save data to it, query it, etc. with no problems whatsoever. Java has support for the DBM driver that FeatureServer uses by default, so out of the box, I was able to use FeatureServer with Jython without problems.

Next came TileCache. TileCache was originally built to support Python all the way back to 2.2, so I wasn’t expecting many problems. Getting it running turned out to be almost as easy: the only code modification that was needed was a minor change to the disk cache, because Jython doesn’t seem to support the ‘umask’ method. Once I changed that (now checked into SVN), Jython worked just as well with TileCache as it did with FeatureServer.

Clearly, there are some things which are less trivial. The reason that these libraries were so easy to use is because they were designed to be low-dependancy: TileCache and FeatureServer default paths are both entirely free of compiled code. Using something like, for example, GDAL Layers in TileCache, would be much more difficult (if it’s even possible).

However, this presents some interesting capabilities I had not previously thought of.

For FeatureServer, this means that it may be possible to write a DataSource which accesses SDE using the ArcSDE Java API, ESRI’s supported method for accessing their SDE databases. One of the purported “holy grails” of the GIS world is RESTful access to SDE databases via lightweight servers — Jython may provide a path to that, if someone is interested in it. (It may be that this has become a moot point with the release of the ESRI 9.3 REST API — I’m not really sure.) This may be a waste of time, but the fact that it *can* be done is interesting to me. Edit: Howard points out that ArcSDE read/write support exists in OGR 1.6, so this is a moot point; you can simply use OGR to do this without involving Jython/Java.

I think this might also speak to a possibility of having better answers available for people who want to use things like FeatureServer from Java platforms (though I don’t know enough about jython to be sure): the typical answer of “use GeoServer” is great, but to be able to provide something a bit more friendly would be interesting. Thankfully, the Java world is largely catching up to the advances made in TileCache/FeatureServer, so this is also less urgent than it has been in the past.

In the end, this was likely simply an interesting experiment. However, it’s nice to know that the capabilities to do things like this within Jython are improving, and that Jython continues to advance their Python. The 2.2 release being the ‘current’ one still is disappointing, but seeing a 2.5 beta available is an exciting development.

As I said, the current version of FeatureServer works out of the box with Jython, and I’ll be doing a TileCache release shortly that will work with Jython out of the box as well. It’s neat to see more possibilities for using these libraries I’ve spent so much time on.

First commit in 2008:
* FeatureServer: r412
* TileCache: r242
* OpenLayers: r5614

All just updates to copyright dates, but hey, they make me fee special.

San Diego Wildfires 2007

Lots of different data here:

• Landsat7: i-Cubed processed landsat7 WMS hosted by Telascience.
• USGS Urban Orthos, hosted by Telascience.
• 2006 DigitalGlobe imagery of the area, hosted by Telascience — built last night specifically for this project
• MODIS fire map (last 7 days), hosted by Telascience, put together by Martin Spott
• OpenStreetMap Hybrid tiles built by me and hosted by TelaScience
• Destroyed houses: WFS served by FeatureServer, initial population from KML file
• 2 KML files, from Google MyMaps: KPBS and LA Times
• Geotagged Photos tagged ‘wildfires’ from Flickr, converted to WFS by FeatureServer.

Gave my FeatureServer talk. What I walked through in prep and what I actually talked about were not so much alike. However, the talk was *extremely* well attended. I feel bad for Charlie, who had way way more preparation — and really, a better talk — and about half the room walked out right before it :/

Still, I think the talk went well, and hopefully people got some information about FeatureServer that they didn’t have before.

Anyone planning on making it to the FeatureServer presentation at FOSS4G? What would you like to hear about?

RESTClient: “RESTClient is a simple, Python + wxWidgets Desktop application for talking to RESTful web services. It is designed to fill a gap in existing offerings by offering support for GET/POST/PUT/DELETE, making it a useful tool when exploring RESTful web services which use a wider range of HTTP verbs.”

Simple. Stupid. It Works.

Having spent some time around SJ Klein the past week, I’m currently hacking on some mapping things for them. I’m also trying to find interesting activities that already exist to try them out: One of the ones I stumbled into is Kuku Anakula (Hungry Chicken). After playing it for a bit, I found it dissappointingly slow to respond to keyboard input. Since Python is a main component in OLPC, it’s written in Python, so I took it apart.

The result is that I refactored the code to make an order of magnitude fewer function calls, and in the process shaved about 50% of the CPU time off, so far as I can tell, when testing on my mac. I learned how to use hotshot, and have the original profiling data and the profile for my refactor. Getting the new version of the activity on the XO was very satisifying: the gameplay is significantly better, and changes the game from being more like a chore to more like a game.

Other things I did today for OLPC include building/installing OGR, and getting FeatureServer set up to run on it with a world borders shapefile. Still got a bit of work to do to build a world map/atlas with click-to-query attributes local on the box, but it’s not too far off.

If you didn’t see them via Slashdot, this set of photos of the OLPC in action is very cute.

Just got notice that I’ll be giving two presentations at FOSS4G 2007:

FeatureServer: A REST-based Server for Simple Features
With the number of tools for creating vector data online growing rapidly over the past year, users have spent more and more of their time creating annotations of existing maps. Unfortunately, managing this user generated data in a web browser can be difficult: the existing tools for storing geographic data on the server are largely based around the WFS-T specification, which is not the most conducive to the browser environment. FeatureServer provides an alternative mechanism for fetching and storing geographic data, using a REST-based interface that is friendly to browsers and other clients alike.

And with Howard Butler:

The Gift Economy Ain’t Free: Getting Help with Open Source Software
Have you ever been told to RTFM? STFW? Sent an email to a project’s maillist that languished for days without a response? This talk will give you ammo that you can use to bust out of the rut of frustration and non-response.

Woot.

I just wanted to thank all the well-wishers in comments on my previous post about the upcoming wedding. For some reason, I hadn’t actually gotten the comment notifications — or I just deleted them as noise — and so I’ve just gone back and approved the 5 comments I got. (And to Taral: the ‘highway’ called RED is actually the Red Line — a subway line. This is a case where local knowledge is assumed. :))

The week post wedding has been relatively peaceful, though our last wedding guests didn’t leave until Thursday, so we’re finally getting a bit of alone time this weekend. Bio-dad has the kids, so I plan to spend the weekend chilling out.

If you haven’t seen FlickrBrowse recently, you might want to check it out: It’s now using the new world wrapping support in OpenLayers, which is kind of nifty. I also added Twittervision support to FeatureServer a couple days ago — not the most useful thing in the world, since the only query type is ‘current location for $username’, but it was almost too easy *not* to do it: the code is only 12 lines.

Maps tell stories. They tell all sorts of stories, but one of the stories that they tell the best are processions: series of photos taken over a wide area over a period of time, but with the same principle actors.

The most important procession to me right now is the one that I’ve been working up to for four years, as of tomorrow. At the church shown (using the Boston Freemap TileCache, FeatureServer for feature translation from the Flickr API, and OpenLayers, of course, as the map interface) on the map I put together, I’ll be getting married tomorrow to the lovely Jessica Allan.

From Chicago to Champaign-Urbana to Manchester, NH to Cambridge, MA, with more late nights and late flights than I care to remember, more love and devotion than I can describe, and more acceptance of my tendencies to show utter obsession with anything I’m doing than I could possibly have expected, my relationship with Jess has flourished, and I couldn’t be happier to be tying the knot tomorrow.

Okay, so the relation this has to mapping or technical ramblings is tentative at best, but I still think it’s cool. 🙂