Technical Ramblings

Notes about Processing, from processing.org, and other related things.

“Drawing images is not about telling which pixels to turn on or off, it’s about telling to draw lines.”

In addition to processing being pretty, you can draw the entire map into any unit you want. Drew a bunch of lines out in state plane projected unit system, and then cropped image.

Drawing all of the streets in King County plus all of transit lines

4d expansion of transit data — time out of the map on photo

Processing is great at generating static images; looping over a dataset and drawing lines is what processing is for.

Demoing a bunch of demos of processing from Tom Carden; cabspotting, travel time tube map, etc. etc.

Processing.js: Processing in JS, by John Resig. Uses “Processing.js is like developing Processing on a 400mhz celeron”

Source code to London Tube Map is available; Processing makes source code available by default, Google for “powered by processing” for examples.

Stamen typically does Prototype in Processing, final client in Flash.

Processing is Java; it should be possible to pull in Java libraries; can import .jar files, can drag in data (images, etc.); loading data is a one liner.

NodeBox: “Processing in Python”

Obsessing.org: Processing.js GUI.

Same processing source code in browser and applet using Processing.js

Context Free Art: here

The default mode of processing is:
* Setup
* Draw function that’s called as much as possible

Going to publish cabspotting example to wherecamp wiki

Brandon demos OpenStreetMap data rendered on the fly with processing data.

NodeBox demo using a springgraph. Proxmiity fixes of local bluetooth data.

Processing is 4-5 years old now

Collaboration is typically “Copy an example, create your own, keep going”

Tonight, starting around 7:30, there is a plan to descend on The Dubliner, in Noe Valley, San Francisco, for beer and hackery. If the plan changes, I’ll update twitter. If you want to join us, please:

• Comment here
• Email me (crschmidt@crschmidt.net)
• or Text or call me (603.264.2294)

I’m off to Wherecamp tomorrow. I’ll be in town and have no plans yet as of Friday, and I’ll be at wherecamp all weekend, planning to spend the night hacking at the Googleplex.

Who’s going to be there? Anyone interested in meeting up on Friday, or doing any hacking ahead of time or during?

In order to use TileCache to cache tiles for use in the Google Maps API, you need three things:

• A WMS server which supports the spherical mercator projection
• A properly configured TileCache pointing to it
• A small snippet of code to add a custom TileLayerOverlay to your Google Map
• A small snippet of code to add a custom TileSource to your Virtual Earth Map

WMS Server

My experience is with MapServer, so that’s what I’m going with here.

MapServer uses proj.4 for its reprojection support. In order to enable reprojection to Spherical Mercator in MapServer, you must add the definition for the projection to your proj.4 data directories.

On Linux systems, edit the /usr/share/proj/epsg file. At the bottom of that file, add the line:

<900913> +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0
               +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs

After you do this, you must add the projection to your wms_srs metdadata in your map file:

map
  web
    metadata
      wms_srs "EPSG:4326 EPSG:900913"
    end
  end
  # Layers go here
end

This will allow you to request tiles from your MapServer WMS server in the Spherical Mercator projection.

Configuring TileCache

Your TileCache configuration will need to point to your WMS installation, using the parameters suggested for Spherical Mercator in the default tilecache.cfg.

[google-tiles]
type=WMS
url=http://labs.metacarta.com/wms/vmap0
layers=basic
extension=png
bbox=-20037508.3427892,-20037508.3427892,20037508.3427892,20037508.3427892
maxResolution=156543.0339
srs=EPSG:900913

Here, you can see that I’ve used the MetaCarta Labs vmap0 WMS. If you’re using a standard MapServer WMS, you might have a url more like:

url=http://example.com/cgi-bin/mapserv?map=/mapdata/mapfiles/vmap0.map

Setting up Google Maps

Finally, you must make a GTileLayerOverlay for your tiles.

    var myTileLayer = new GTileLayerOverlay(new GTileLayer(null,null,null,{
      tileUrlTemplate: 'http://example.com/tilecache/1.0.0/google-tiles/{Z}/{X}/{Y}.png?type=google',
      isPng:true}));
    var map = new GMap2(document.getElementById("map_canvas"));
    map.setCenter(new GLatLng(0,0), 0);
    map.addControl(new GSmallMapControl());
    map.addOverlay(myTileLayer);

The ‘type=google’ flag on the end of the URL tells TileCache to use the “Google-style” 0,0 in the upper left corner.

Once you’ve done this, you should have a TileCache layer on top of your Google Maps base layer. You can see an example of this setup, just for proof that I’m not putting you on.

Virtual Earth Javascript

“But I don’t like Google!” you say. “I want VE!” Well then, why aren’t you using OpenLayers already? I mean, that’s what it’s for, right?

More seriously, VE isn’t much more difficult:

    vemap = new VEMap('myMap');
    vemap.LoadMap(new VELatLong(0, 0), 0 );
    //Add layer
    var tileSourceSpec = new VETileSourceSpecification("mclabs", null, 1,
       [new VELatLongRectangle(new VELatLong(-86,-180),new VELatLong(86,180))],
       1, 16, function (tileContext)
{
   if(tileContext != null && tileContext != "undefined")
   {
      var key = tileContext.ZoomLevel+ "/" + tileContext.XPos + "/" + tileContext.YPos + ".png?type=google";
      var path = "http://example.com/tiles/1.0.0/google-tiles/" + key;
      return path;
   }
}, 0.8, 100 );
     vemap.AddTileSource(tileSourceSpec);

     var tileLayer = new VELayerSpecification(VELayerType.VETileSource,"mclabs","mclabs");
     vemap.AddLayer(tileLayer);

And, for your viewing pleasure: an example of the same tileset in use.

Using Client Side Storage for Mapping — a proof of concept which saves locally drawn features either via HTML5 Offline Storage or the same from Google Gears.

After a long, long haul, r6945 tags OpenLayers 2.6 as a final release.

The OpenLayers Development Team is proud to announce release of OpenLayers 2.6. As of this final release, the OpenLayers 2.6 release closes 294 outstanding tickets. This is the largest of any OpenLayers release to date.

Client side reprojection! Smooth commercial layer panning! KML styling support and pretty, auto-sizing popups! Improved styling support! And lots more.

Six months in the making, this is the best OpenLayers release ever. (And only two RCs! A new record. Here’s hoping that’s not just because we had no testers… ;))

Use 2.6 today!

Just a reminder: If you use OpenLayers in your applications, Now Is The Time to test out 2.6 and make sure that it doesn’t break anything. We’re currently in RC, and we’re getting pretty close to cutting a final release if no one complains.

The RC2 Announcement has more info on how to get set up with OpenLayers 2.6.

OpenLayers 2.6 adds some cool mechanisms for doing your styling on the client side: allowing you to use attributes of your data to create styling information on the fly.

A couple days ago, there was a post about using GeoJSON for thematic mapping. The conclusion of that post puts the blame in the wrong place: “Conclusion: GeoJSON has a lot of potential, but is currently not suitable for world maps due to browser restrictions.” was the conclusion, but this is the case for all formats: nothing here is specific to GeoJSON. It also stated that Firefox is the best browser for in-browser vector display: this is also wrong, as both Safari and Opera do significantly better with SVG rendering than Firefox or IE.

I’ve moved beyond that, though, and wanted to look at various ways to style the data.

The original map was essentially only styled by using data built into the GeoJSON for storing attribute color. Clearly this is not ideal: embedding styling information with the data is great when you want to control the user experience, but seperating it allows application developers more control. Luckily, with the upcoming release of OpenLayers (OpenLayers 2.6), you can do this styling in the browser.

This choropleth map uses the same data as the example in the Thematic Mapping blog post, but instead of taking the style rules from the data, creates a graduated color set. (I don’t know what this is actually called: Thematic mapping isn’t my gig.) Looking at the code, it’s easy to see the color ranges: 0 -> 10, 10-> 20, 20->50, 50->100. (The theme is separated into a separate function for readability.) No base map: we don’t need one for the visual effect to be reasonably pronounced. We get a worldwide, colored map of internet users (in 2005), with attribution and the ability to hover over a country to see its statistics.

But wait, there’s more!

OpenLayers 2.6 has reprojection support: the ability to change the projection of data. So, we can reproject the map into mercator. This is actually a useful educational map: you can see that different projections show significantly different percentages of the world as being covered with high-percentage of internet users.

Anyway, threw it together, and thought it was cool.

A couple people have asked me over the years what I plan to do with GSMLoc.org. I think that every time I’ve answered, I’ve wished I could say “I have great plans!”… but I don’t.

Until now, I didn’t have a good dump of the data made automatically available. This was silly. I’ve now created a cronjob that will, at midnight, create a gzipped csv dump of the data.

I make no claims as to the quality of this data, and place no restrictions on it, though credit to gsmloc.org or the gsmloc project would be cool.

Sorry that I’ve been so lazy about not making this available. I may still clean up the website, and I’ll gladly link to other projects who are doing the same thing, and I’m sorry that I’ve been such a bad steward over the past couple years.

Jo Cook writes about ESRI’s crackdown on licensing:

In what can only be described as a noble act of self-sacrifice, ESRI have told us that as an educational charity we are no longer allowed to have an educational discount for using their software and, not only that, our license codes will cease to work at the end of this month.

This is why Open Source software is so important. So you think you have a stable relationship with your vendor? Maybe you think that you’ve come to a great licensing agreement that you’re happy with? Remember that so long as you’re working in an environment where someone else controls the tools you use, you’re not able to make your own rules.

Why does Open Source software matter? If you’re not using it, you’re handing over control of your use of your tools — possibly important ones — to people who aren’t under your control. In the end, that lack of control may end up hurting you far more than you’d expect.