Technical Ramblings

TileCache 2.0 has been released. From the announcement:

TileCache 2.0 is a major change in that way that the code works internally: rather than one large module each for layers, services, and caches, these are now split up into individual files.

The benefit that this offers is large: it allows for much easier customization of TileCache layer classes, with support for dropping your own custom modules into your TileCache config for either Layers or Caches. Additionally, it makes support for new services simpler.

One of the new layer types is an ArcXML layer, and one of the new Service types is support for MGMaps, a mobile mapping client available on many different mobile phones. One of the things that’s *not* mentioned in the relase is that TileCache now has limited support for KML SuperOverlays, allowing you to use browse your worldwide dataset in Google Earth. (A handy thing to have, if you ask me.)

Second announcement: TileCache in SVN now has support for storing tiles in Amazon Web Services S3 — simply tweak your tilecache.cfg, and enter:

[cache]
type=AWSS3
access_key=your_access_key
secret_access_key=your_secret_access_key

save the file, and you’ll be caching in S3.

(Also, in development, I have accrued a total of 4 cents of cost — but all of those are rounded up numbers, I’m nowhere near actually having to pay any ‘real’ money yet on any of their four metrics.)

Included as part of the Cache layer is a handy set of management tools for your S3 cache:

Usage: AWSS3.py [options] action    
    action is one of: 
      list_locks
      count_tiles
      show_tiles
      delete <object_key> or <list>,<of>,<keys>
      delete_tiles

Options:
  -h, --help  show this help message and exit
  -z ZOOM     zoom level for count_tiles (requires layer name)
  -l LAYER    layer name for count_tiles
  -k KEY      access key for S3
  -s SECRET   secret access key for S3

There’s probably a lot more that should be done with regard to the S3 cache layer, but for users for whom outsourcing data management is an acceptable cost, you now have the ability to use S3 more easily.

Enjoy!

OpenLayers has the ability to read and write geographic data in several different formats: KML, GML, GeoRSS (Simple and GML), GeoJSON, WKT. One of the things that hasn’t thus far been exploited as a result of this is the use of OpenLayers as a format conversion tool… until today.

OpenLayers Data Conversion is a new service that allows easy conversion from any of the formats that OpenLayers supports to any of the formats that OpenLayers supports. Simply paste your data, choose your format, and hit convert. Additionally, this tool supports reprojection to and from Spherical Mercator, the internal coordinate system used in Google Maps.

This code isn’t quite in trunk of OpenLayers yet, but in the meantime, you can grab the example from an svn sandbox, including the prebuilt lightweight OpenLayers build for format conversion.

From the mailing list:

Tomorrow Ordnance Survey (Britain’s national mapping agency) is letting a group of developers preview a new mapping API named OS OpenSpace. We aim to launch this to the public in January. We have used OpenLayers as a base for the API and have really enjoyed the opportunity to work with such a comprehensive library.

The email goes on to say that they’ve added a number of features to OpenLayers… but offers no indication that there is likely to be any likelihood of sharing the changes back with the OpenLayers community.

Of course, this doesn’t really come as surprising: a number of aspects of the effort are somewhat misguided. Very low request limits, “we own all your derivative work”, etc. However, there is one thing that is interesting, brought up on the OpenStreetMap Mailing List:

4.4 In the event that you or any User creates Derived Data, such Derived Data shall be owned by us, save that if any Derived Data is created which is a severable improvement (as defined by Commission Regulation (EC) No 772/2004, known as the Technology Transfer Block Exemption) of the Ordnance Survey Data then such Derived Data shall be owned by the person or entity creating the same.

The interesting bit is the “severable improvements” bit, which Richard Fairhurst (a member of the OSM Foundation) delves into a little deeper:

In practice, then, I read that to mean that if you use the Sustrans webmapping to find out where the routes go, this information is solely copyright Sustrans (who might be more willing to give permission) and not OS, even though it’s delivered through the medium of a derived work. So, if you had the wiggly lines already (whether mapped by GPS or NPE), you could tag them as NCN routes if Sustrans were ok with that.

Certainly brings an interesting development into the world of ‘derived works’ based on OS Mapping for the free data crowd.

Regardless, I must admit some level of frustration with users who bother to email and say “Thank you”, but not to contribute any effort into further development of the software they’re using. I suppose it’s better than ignoring the project entirely, but some of the things mentioned in the email are things I would love to have in the trunk — and there wasn’t even a consideration given to the mailing list as to whether we were working on some of these things.

I wonder if the OpenLayers community is scary in some way. We really do try to be open and welcoming. Maybe I’m just too much of an ass to people on the mailing list, but I try not to do that to people who actually want to help…

One of the things that users often want to do within MapServer or other tools is to hook up to existing caches of high resolution aerial imagery, like VirtualEarth, Google, etc. to get an aerial basemap that is as high resolution as possible, preferably for free. This applies to cases other than serving web maps as well: for example, the EVS Islands blog has been using Google Earth imagery to derive datasets from for some time now, and is now acting as if there is a product that Digital Globe is selling which could fill this need.

Of course, many users also want a pony.

In some cases, the imagery you see in Google Earth is something you can’t even buy for use online — so with all the money in the world, you might find yourself short a high resolution dataset, or so you might think.

It turns out that for some areas of the world, there is a lot of *public* imagery available. Instead of depending on commercial providers, and the restrictions they entail, you can turn this free datasource into a source for your own maps. OpenAerialMap has begun the task of collecting and collating these images for you, so you don’t have to have your own WMS to see the latest data from the USGS dataset in your maps. (It’s still in its early stages, but it’s getting there.) Now, I understand that in the less enlightened parts of the world, this is not yet the case, but we’re getting there, slowly but surely.

But what happens when you don’t want a map, but instead want an image that you can include in your print-resolution magazine spread?

Open Source Software happens, that’s what.

GDAL has the ability to read images from a remote WMS server, and treat them as it does any other data source: which means that it can take the images, and convert them to any other format, from JPEG2000 to GeoTIFF. And it doesn’t even require any system dependencies for Windows or Linux: Just use Frank Warmerdam’s excellent FWTools. FWTools wraps up all the tools you need to work with geographic formats, be they raster or vector, into one neat package.

1. Grab FWTools: Linux, Windows
2. Unpack/extract it.
3. Run the install script in the root directory.
4. Grab http://openaerialmap.org/static/gdal_wms.xml and save it to the root of your FWTools installation.
5. Open a command line prompt in the FWTools directory.
6. Type: bin_safe/gdal_translate -projwin -93.246226 44.892179 -93.195672 44.866816 -outsize 9424 4728 -of JPEG gdal_wms.xml airport.jpg

End result? A 10000 by 5000 mosaic of the Minneapolis-St. Paul International airport, using data loaded from Public Domain USGS aerial imagery — some from Ramsey County, some from the Twin Cities overflight, both higher than 1M resolution (.15 and .3 respectively) within the past year and a half — and the speed to create it is essentially as fast as your connection to the net.

You can try this at home, and once you have the image, you can do *whatever you want with it*: The data involved is all public domain, and there is nothing to stop you from doing anything with the data. You can convert it to another format, derive vectors from it, print it out on a giant poster board, etc. No attribution or other sourcing necessary, because the data is public.

OpenAerialMap is all about the sharing of data that you should be able to get access to anyway, and as you can see here, in some cases the data that you wish you had you have access to already — you just didn’t know it yet.

Google’s imagery vs. the imagery in OpenAerialMap:

Which one is which?

The answer is probably obvious from the OAM Main Map, once you know where to look…

This is a case where I actually like OAM’s imagery better than GE’s. I’m pretty sure both are sourced from MassGIS, but GE’s is from 2003, and OAM’s is the more recent 2005 imagery.

One of the things I forgot about in my previous post: TileCache now has “superoverlay” support for some limited cases, and OAM is the first real test case of that. You can check it out by downloading OAM KML and offer any feedback to me.

I’ve actually found myself very disappointed with this support: It seems that Google downloads all the tiles in series, with no parallelization. The end result is that it takes *forever* to actually get down to an area, though I think I’ve worked out a solution that will make that much better (at the cost of sometimes dropping out the OAM images before the next layer is ready to display).

So, if you’re a Google Earth user, and want to see what OAM is all about, grab the KML above and start spinning the globe around.

One of the cool things about building a map on top of quality Open Source products is the ease with which one can extend the possible uses of your software.

GDAL 1.5 has gained the ability to read a WMS-C style remote tile cache directly: this means that any tool which uses GDAL as an image provider can read remote tile caches directly. The one that matters the most to me is MapServer: MapServer has the ability to load up a simple config file — http://openaerialmap.org/static/gdal_wms.xml — and use that in a mapfile: Using With MapServer.

Not running the bleeding edge GDAL? That’s fine: Instead, you can use the WMS that’s hosted by the OAM project: Using With WMS. This means that any WMS client — including those fancy schmancy ESRI tools that speak WMS. (There are at least some of them, I think.)

Your application only speaks in WorldWind? Well, we can take care of that for you: With a WorldWind metadata file, you should be up and running in no time. (In my experience, unfortunately, this is true in ossimPlanet, which has no problem with the file, but WorldWind itself seems a bit touchy about it once you zoom past the first couple levels. But I can only do so much.)

Using the new plugin-friendly TileCache that I rewrote last night, I was able to add support for the Mobile GMaps client — Using With MGMaps — in just 20 minutes. Similarly, a user reported success using the tiles with Maemo Mapper.

Of course, it can be used with OSM — either as a tile backdrop, or in the OSM editing application (JOSM): Using With OSM — due to the above plus TileCache’s TMS support.

As a result of the TMS support, it should be possible to load into Google Maps as well, though I’m really too lazy to test that one.

Overall, with just a few small tools — GDAL, MapServer and TileCache — it’s possible to build support for a wide variety of services from a single dataset.