Archive for the 'TileCache' Category

New Mailing List: tiling; Feedback On WMTS

Posted in FOSS4G 2010, OSGeo, TileCache on September 9th, 2010 at 03:07:15

In the past, for tiling, we discussed tiling on an EOGEO list. In the meantime, OSGeo has grown up, EOGEO has moved on, and it seems that there isn’t a very good home for future tiling discussions.

As a result, I have added a tiling list to the OSGeo mailing list server.

Tiling List @ OSGeo

Projects that I hope to see people joining from: TileCache, Tirex, Mapproxy, GWC, others, etc.

This list will be discussing general tiling ideas — how to cache tiles, how to manage caches, how to work with limited caches, where to put your tiles, things like S3, etc. etc. If you are at all interested in tiling — not at the level of a specific application, but in general — please join the list.

Additionally, if you are interested in discussing providing feedback to the OGC regarding the WMTS spec — especially if you are an implementer, but also if you are a user — I would encourage you to join the standards list at OSGeo:

http://lists.osgeo.org/mailman/listinfo/standards

Several people have expressed interest in coordinating a response to the OGC regarding the spec, and we would like to work together on this list to coordinate.

Jython + TileCache/FeatureServer: It Just Works

Posted in default, ESRI, FeatreServer, FeatureServer, spatialreference.org, TileCache on December 14th, 2008 at 10:37:04

Earlier today, I tried Jython for the first time, because I’m doing some work that may involve interactions with Java libraries in the near future. Jython, which I’ve always avoided in the past due to an irrational fear of Java, is “an implementation of the high-level, dynamic, object-oriented language Python written in 100% Pure Java, and seamlessly integrated with the Java platform.” (I love projects that have great one-liners that I can copy paste.)

My goal for Jython was to do some work with the GeoTools EPSG registry code related to SpatialReference.org. Sadly, I didn’t get that working, but in the process, I learned that Jython now has a beta version which is up to Python 2.5 — much newer than the 2.2 that had previously been available.

With that in hand, I decided to see if I could get some of my other Python projects running under Jython. I’m the maintainer for both TileCache and FeatureServer — two pure Python projects. Theoretically, these projects should both work trivially under Jython, but I’ve always had my doubts/fears about this being the case. However, it turns out that my fears here are entirely unfounded.

I downloaded the FeatureServer ‘full’ release from featureserver.org: this includes the supporting libraries needed to get a basic FeatureServer up and running. I then tried to run the FeatureSever local HTTP server… and it worked out of the box. I was able to Load the layer, save data to it, query it, etc. with no problems whatsoever. Java has support for the DBM driver that FeatureServer uses by default, so out of the box, I was able to use FeatureServer with Jython without problems.

Next came TileCache. TileCache was originally built to support Python all the way back to 2.2, so I wasn’t expecting many problems. Getting it running turned out to be almost as easy: the only code modification that was needed was a minor change to the disk cache, because Jython doesn’t seem to support the ‘umask’ method. Once I changed that (now checked into SVN), Jython worked just as well with TileCache as it did with FeatureServer.

Clearly, there are some things which are less trivial. The reason that these libraries were so easy to use is because they were designed to be low-dependancy: TileCache and FeatureServer default paths are both entirely free of compiled code. Using something like, for example, GDAL Layers in TileCache, would be much more difficult (if it’s even possible).

However, this presents some interesting capabilities I had not previously thought of.

For FeatureServer, this means that it may be possible to write a DataSource which accesses SDE using the ArcSDE Java API, ESRI’s supported method for accessing their SDE databases. One of the purported “holy grails” of the GIS world is RESTful access to SDE databases via lightweight servers — Jython may provide a path to that, if someone is interested in it. (It may be that this has become a moot point with the release of the ESRI 9.3 REST API — I’m not really sure.) This may be a waste of time, but the fact that it *can* be done is interesting to me. Edit: Howard points out that ArcSDE read/write support exists in OGR 1.6, so this is a moot point; you can simply use OGR to do this without involving Jython/Java.

I think this might also speak to a possibility of having better answers available for people who want to use things like FeatureServer from Java platforms (though I don’t know enough about jython to be sure): the typical answer of “use GeoServer” is great, but to be able to provide something a bit more friendly would be interesting. Thankfully, the Java world is largely catching up to the advances made in TileCache/FeatureServer, so this is also less urgent than it has been in the past.

In the end, this was likely simply an interesting experiment. However, it’s nice to know that the capabilities to do things like this within Jython are improving, and that Jython continues to advance their Python. The 2.2 release being the ‘current’ one still is disappointing, but seeing a 2.5 beta available is an exciting development.

As I said, the current version of FeatureServer works out of the box with Jython, and I’ll be doing a TileCache release shortly that will work with Jython out of the box as well. It’s neat to see more possibilities for using these libraries I’ve spent so much time on.

Using TileCache with Google Maps, Virtual Earth

Posted in Google Maps, Locality and Space, TileCache, Virtual Earth on April 18th, 2008 at 06:15:00

In order to use TileCache to cache tiles for use in the Google Maps API, you need three things:

  • A WMS server which supports the spherical mercator projection
  • A properly configured TileCache pointing to it
  • A small snippet of code to add a custom TileLayerOverlay to your Google Map
  • A small snippet of code to add a custom TileSource to your Virtual Earth Map

WMS Server

My experience is with MapServer, so that’s what I’m going with here.

MapServer uses proj.4 for its reprojection support. In order to enable reprojection to Spherical Mercator in MapServer, you must add the definition for the projection to your proj.4 data directories.

On Linux systems, edit the /usr/share/proj/epsg file. At the bottom of that file, add the line:

<900913> +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 
               +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs

After you do this, you must add the projection to your wms_srs metdadata in your map file:

map
  web
    metadata
      wms_srs "EPSG:4326 EPSG:900913"
    end
  end
  # Layers go here
end

This will allow you to request tiles from your MapServer WMS server in the Spherical Mercator projection.

Configuring TileCache

Your TileCache configuration will need to point to your WMS installation, using the parameters suggested for Spherical Mercator in the default tilecache.cfg.

[google-tiles]
type=WMS
url=http://labs.metacarta.com/wms/vmap0
layers=basic
extension=png
bbox=-20037508.3427892,-20037508.3427892,20037508.3427892,20037508.3427892
maxResolution=156543.0339
srs=EPSG:900913

Here, you can see that I’ve used the MetaCarta Labs vmap0 WMS. If you’re using a standard MapServer WMS, you might have a url more like:

url=http://example.com/cgi-bin/mapserv?map=/mapdata/mapfiles/vmap0.map

Setting up Google Maps

Finally, you must make a GTileLayerOverlay for your tiles.

 
    var myTileLayer = new GTileLayerOverlay(new GTileLayer(null,null,null,{ 
      tileUrlTemplate: 'http://example.com/tilecache/1.0.0/google-tiles/{Z}/{X}/{Y}.png?type=google', 
      isPng:true})); 
    var map = new GMap2(document.getElementById("map_canvas"));
    map.setCenter(new GLatLng(0,0), 0);
    map.addControl(new GSmallMapControl());
    map.addOverlay(myTileLayer);

The ‘type=google’ flag on the end of the URL tells TileCache to use the “Google-style” 0,0 in the upper left corner.

Once you’ve done this, you should have a TileCache layer on top of your Google Maps base layer. You can see an example of this setup, just for proof that I’m not putting you on. 🙂

Virtual Earth Javascript

“But I don’t like Google!” you say. “I want VE!” Well then, why aren’t you using OpenLayers already? I mean, that’s what it’s for, right? 🙂

More seriously, VE isn’t much more difficult:

    vemap = new VEMap('myMap');
    vemap.LoadMap(new VELatLong(0, 0), 0 );
    //Add layer
    var tileSourceSpec = new VETileSourceSpecification("mclabs", null, 1, 
       [new VELatLongRectangle(new VELatLong(-86,-180),new VELatLong(86,180))], 
       1, 16, function (tileContext)
{
   if(tileContext != null && tileContext != "undefined")
   {
      var key = tileContext.ZoomLevel+ "/" + tileContext.XPos + "/" + tileContext.YPos + ".png?type=google";
      var path = "http://example.com/tiles/1.0.0/google-tiles/" + key;
      return path;
   }
}, 0.8, 100 );
     vemap.AddTileSource(tileSourceSpec);
    
     var tileLayer = new VELayerSpecification(VELayerType.VETileSource,"mclabs","mclabs");
     vemap.AddLayer(tileLayer);

And, for your viewing pleasure: an example of the same tileset in use.

TileCache in Debian

Posted in Debian, Locality and Space, TileCache on January 6th, 2008 at 13:07:51

I took the last week of the year off from work, and I spent most of the time doing a push to get some of the software that I help develop into Debian.

Debian is a Linux distribution that is built and maintained almost entirely by volunteers: it is the basis of the very popular Ubuntu operating system used on desktops, and is also used heavily for both desktops and servers at MetaCarta. Getting a package into Debian’s main repository is an indication that it meets the projects (relatively stringent) guidelines for inclusion into the repository.

The first package that I did this for was TileCache. TileCache is a relatively simple to install utility, but it has significant benefits from being installed directly on the system — such as the ability to place the configuration file into the main configuration files locations, etc. In addition, TileCache is a relatively widely used piece of software — meaning it could have benefits for a number of users.

New packages are typically packaged by a user, then a ‘sponsor’ is searched for. A sponsor is a Debian Developer — someone with the rights to upload packages to Debian — who will take on the review of the packaging materials to ensure that the meet Debian’s guidelines.

Thanks to a thorough and patient review by Paul Wise, from the DebianGIS list, TileCache was able to be put into Debian in less than a week’s time, and is now available to users of Debian “Sid”/unstable. Barring any problems with TileCache showing up, in 10 days, TileCache will be migrated to “Lenny”/testing. Additionally, I believe that this means that TileCache may be included in the next Ubuntu release. (I’m not sure on that — I don’t know the exact way that Ubuntu pulls its packages in.) You can see information on the TileCache package on the TileCache Debian package info page.

As a result of this, I’ve also packed the python-memcache library, maintained by tummy.com, ltd, which will make it easier for TileCache users to use the Memcached cache class.

All in all, my initial experiences with Debian have been very positive, and I’m looking forward to continuing to work on building more of the software I build into Debian packages for wider distribution. Indeed, cleaning up things for Debian has resulted in a lot of other improvements to the code as well: meeting guidelines for documentation, packaging, etc. has resulted in adding documentation that didn’t otherwise exist, and cleaning up what did exist.

You can track some of what I’m now maintaining or working to get into Debian via my maintainer page, as well.

early commits

Posted in FeatureServer, Locality and Space, OpenLayers, TileCache on January 1st, 2008 at 03:51:35

First commit in 2008:
* FeatureServer: r412
* TileCache: r242
* OpenLayers: r5614

All just updates to copyright dates, but hey, they make me fee special.

TileCache: Two announcements

Posted in Amazon Web Services, Locality and Space, TileCache on December 22nd, 2007 at 17:57:51

TileCache 2.0 has been released. From the announcement:

TileCache 2.0 is a major change in that way that the code works internally: rather than one large module each for layers, services, and caches, these are now split up into individual files.

The benefit that this offers is large: it allows for much easier customization of TileCache layer classes, with support for dropping your own custom modules into your TileCache config for either Layers or Caches. Additionally, it makes support for new services simpler.

One of the new layer types is an ArcXML layer, and one of the new Service types is support for MGMaps, a mobile mapping client available on many different mobile phones. One of the things that’s *not* mentioned in the relase is that TileCache now has limited support for KML SuperOverlays, allowing you to use browse your worldwide dataset in Google Earth. (A handy thing to have, if you ask me.)

Second announcement: TileCache in SVN now has support for storing tiles in Amazon Web Services S3 — simply tweak your tilecache.cfg, and enter:

[cache]
type=AWSS3
access_key=your_access_key
secret_access_key=your_secret_access_key

save the file, and you’ll be caching in S3.

(Also, in development, I have accrued a total of 4 cents of cost — but all of those are rounded up numbers, I’m nowhere near actually having to pay any ‘real’ money yet on any of their four metrics.)

Included as part of the Cache layer is a handy set of management tools for your S3 cache:

Usage: AWSS3.py [options] action    
    action is one of: 
      list_locks
      count_tiles
      show_tiles
      delete <object_key> or <list>,<of>,<keys>
      delete_tiles

Options:
  -h, --help  show this help message and exit
  -z ZOOM     zoom level for count_tiles (requires layer name)
  -l LAYER    layer name for count_tiles
  -k KEY      access key for S3
  -s SECRET   secret access key for S3

There’s probably a lot more that should be done with regard to the S3 cache layer, but for users for whom outsourcing data management is an acceptable cost, you now have the ability to use S3 more easily.

Enjoy!

TileCache Talk

Posted in FOSS4G 2007, Locality and Space, TileCache on September 26th, 2007 at 13:09:51

Schuyler gave a great overview of TileCache, how to use it, and what it does earlier today. It was good that I didn’t have to give it. 🙂 Came together really nicely. From the talks, sounds like a few people are interested in it, and more than a few people are already using it.

TileCache is a weird piece of software, because it’s very good at what it does — per user, it seems like I get significantly fewer questions for TileCache than OpenLayers or something else. This is good, but means that my insight into the number of TileCache users is way lower than into OpenLayers.

Still, good to see the high interest in TileCache, and a great presentation by Schuyler which is great for people getting into it.

Update on Near Term Goals for TileCache

Posted in Locality and Space, TileCache on July 20th, 2007 at 14:56:00

Threshold bug: Fixed. 1.9-rc1,2 released, and a user even confirmed that it fixed his threshold problem (yay)! This will probably be the last 1.9 release, and includes significantly improved error reporting.

Refactor for drop in Layers: done, in the ‘refactor’ branch. I think I’ve maintained backwards compatibility at the ‘using a config file’ level, even, though it does need an additional option for mod_python support, so it will definitely be a 2.0. I also added drop in support for caches.

Add ArcXML layer: Done, though I have no clue how people actually use ArcXML, so this support may actually be total crap. I got it to work with the MassGIS site though. (Would love to talk to an actually ArcIMS/ArcXML user and figure out what I should be doing that I’m not.) I bet you’ve never seen ArcIMS look so fast.

Add KML Region Support: Not done, and not really too excited about it. I plan to get the 1.9 final out the door, then release the 2.0 without it, and maybe loop around for the KML region support later.

Overall, not bad.

Near term goals for TileCache

Posted in Locality and Space, TileCache on July 18th, 2007 at 19:29:05
  • Fix threshhold bug, release 1.9, call it done
  • Refactor to make support for dynamic loading of layer types easier (a la FeatureServer)
  • Add ArcXML Layer (so that I can skip having to do it in OpenLayers)
  • Add KML Region support

First Attempt at IronPython

Posted in default, IronPython, Locality and Space, Python, TileCache on June 20th, 2007 at 06:53:26

My first attempt to do something useful with IronPython:

>>> import urllib
>>> urllib.urlopen("http://crschmidt.net/")
Traceback (most recent call last):
  File httplib, line unknown, in getreply
  File httplib, line unknown, in getresponse
  File httplib, line unknown, in __init__
  File System, line unknown, in set_ReceiveBufferSize
  File System, line unknown, in SetSocketOption
WindowsError: Invalid arguments

Note that I’m working on OS X, and my exception is a WindowsError. Fancy.

(I was inspired by Bill Thorp’s efforts to get TileCache working on IronPython: Round 1, Round 2. However, I’m not all that inspired now.)

Still, it is kind of cool that IronPython just ran — I didn’t expect that to work. Maybe there’s something to this mono business after all.