Technical Ramblings

With Steve Lime’s help on my previous post on the topic, I’ve been able to turn my previously somewhat dissappointing map into something that is definitely coming out way ahead of where it was. (Click the image to see a bigger version of the same area).

First trick: Removing ‘antialias true’/’transparency alpha’ lines from my mapfile, as Zac and Steve both mentioned on the previous post. This actually did most of the work. Looking back, had I actually removed them all, that would have done all of the work, but I missed one, so my polygon fills were still not working. Thankfully, Steve is a great man, and committed fixes for that to trunk, so now I’ve got real pretty images serving up out of the Boston Freemap. With these changes, the label problems aren’t nearly as pronounced as I’d seen them be on other maps.

I definitely feel like this is a huge step in the right direction.

Old:

New:

Pay extra attention to the T lines, which are one of the more obvious failings of the GD rendering.

Many many thanks to Steve for his help in turning the rather disappointing map into something that I’m really very happy with. I’m glad to be proven wrong — in exactly the way I hoped I would be — on the problems with MapServer’s AGG rendering. I’m looking forward to converting more maps in the near future.

So, I’ve just made some updates to spatialreference.org:

• Added comments to EPSG reference pages
• Added human readable wkt output
• Added pagable listing of EPSG codes
• Searching of spatial reference titles
• Searching of WKT

With this, I think I have removed my need to ever have to touch /usr/share/proj/epsg again. The reasons I touched it in the past were usually to search based on title or to add a new projection. Once the new GDAL/OSR is out and installed where I need it, I will be able to use SR.org as references rather than needing to edit proj/epsg to add my own.

Also, for the record, I love Django. This has been the easiest web coding of my life. I spent yesterday porting a half-baked app that was spread over 4 different SVN directories into one Django app. Took me a day, but now instead of a bunch of half-baked stuff that can only be put together by me, I can actually hand the code to someone else and have them run it. (It’s still a bit hard to get up and running on some platforms, because it depends on PostGIS, Python bindings for Postgres, MapScript, OGR, and GDAL. Which are non-trivial on non-Linux platforms.)

It’s great. Thanks to hobu for getting me started with it.

The MapServer 5.0 beta is out. For a while, people had been mentioning the AGG support, and I even saw a pretty great demo put together by DM Solutions that demonstrated the functionality, and I had been impressed.

I was disappointed to learn recently that the fonts are still going to continue to be rendered by GD for the 5.0 release: really, the biggest case where I care about smooth lines is in fonts, because it makes the difference between the ability to use an 8px font, and the corresponding increase in number of labels, and being forced to use a 10px/11px font. (Small fonts become less and less readable at smaller sizes without sub-pixel rendering.) However, I was hoping that the rest of the support would be sufficient that I would be happy with switching the Boston Freemap over to using AGG instead of GD as the renderer.

This morning, I grabbed MapServer, and figured out how to build it after some stumbling. MapServer’s default of “Don’t include anything” is probably a great idea for most people, but for me, it definitely was at stumbling block. Just an example: I was trying to set up –with-agg. I knew I had installed it, and files were in /usr/include/agg2 and /usr/lib/libagg.a. But there’s only one directory! So I tried just pointing to /usr/lib, and got:

error: “could not find agg_rendering_buffer.h or libagg.a in /usr/lib.”

Well, obviously agg_rendering_buffer.h wasn’t there, but libagg.a was! I got there eventually by reading the configure script. It actually wants you to point to /usr, which would have been clearer if it asked for ‘AGG *prefix*’ instead of just ‘AGG directory’. I’m sure this stuff is obvious to people who roll their own, but I love my debian, damnit! 😉

I then ran into an error in building mapagg.cpp, #2188, which I was able to fix by commenting out the offending lines, and an error in building php_mapscript, which I solved by removing that from the build.

I eventually got it built, and bumped into what I eventually filed as a bug: #2187: WMS Request for DRIVER AGG/PNG claims ‘invalid output format’. FrankW was kind enough to help me fix mapwms.c so I could continue to play.

I now have an agg demo of the Boston Freemap. However, I find it very disappointing. The lines actually look *more* jagged in some cases, and for fonts without an outlinecolor, the text is almost unreadable. Additionally, it seems that the blue of my water has changed to be a weird grey color. You can compare against 5.0’s GD rendering and 4.10’s GD rendering — the two differ only by what likely amounts to rounding errors.

The agg rendering is simply not up to snuff. I’ll admit I’ve put more than my fair share of work into getting the Boston Freemap to look good with previous versions of MapServer, but this definitely seems like a step backwards.

I’d love to be told that I’m missing some obvious “Make Output Pretty” option, but if there is one, the AGG RFC doesn’t mention it.

I’d hoped to have a brilliant demo of the power of MapServer. Instead, I’ve got disappointment and pain. It’s a shame that it has to end this way… 😉

The mapfile for the Boston Freemap is, of course, available, if someone feels like trying to understand better why the problems exist. The data is also available, though it’s likely easier to just get it direct from MassGIS.

So, as a result of Custom Projection 6: Google Projection, I got pinged by SharpGIS on IRC, and told that the projection I had there was wrong.

He helped me understand what he was using, and I let him know it still wasn’t helping me. So I set up a Virtual Earth + TileCache demo, and he helped me find the missing piece, in the proj4 FAQ:

In this case, and many other cases using spherical projections, the desired approach is to actually treat the lat/long locations on the sphere as if they were on WGS84 without any adjustments when using them for converting to other coordinate systems. The solution is to “trick” PROJ.4 into applying no change to the lat/long values when going to (and through) WGS84. This can be accomplished by asking PROJ to use a null grid shift file for switching from your spherical lat/long coordinates to WGS84.

+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs

The key here was the +nadgrids=@null, which I’d never seen anywhere before.

The end result?

• Fixing my OSM Tiles
• WMS and TMS overlaid in OpenLayers, as described in an email
• A slightly better understanding of what I didn’t understand for so long.

So. Nice. To not feel lost anymore.

Three cheers for sharpgis! Hooray hooray hooray!

From a post by Howard Butler to gdal-dev:

Chris Schmidt and I have been working on a website that reflects the EPSG spatial reference codes database and user-contributed codes in a variety of output formats. The EPSG code database is never as up to date as you would like, and we frequently tell people to hack their /usr/share/proj/epsg database file with made up codes to support alternative projections. If we could consume URL-based spatial references with OSR, all that would be required is someone adds the custom coordinate system definition to the site and then everyone could refer to it at that location (hopefully indefinitely).

As for the website itself, if it has uptake and people find it useful, we will work to put it somewhere it can live for a very long time — for example OSGeo. The source code (it’s written in Django) will be made available so anyone can stand up their own version of the site.

I’ve been having a lot of fun: this is my first real foray into Django, and I wish I’d done it long ago.

Threshold bug: Fixed. 1.9-rc1,2 released, and a user even confirmed that it fixed his threshold problem (yay)! This will probably be the last 1.9 release, and includes significantly improved error reporting.

Refactor for drop in Layers: done, in the ‘refactor’ branch. I think I’ve maintained backwards compatibility at the ‘using a config file’ level, even, though it does need an additional option for mod_python support, so it will definitely be a 2.0. I also added drop in support for caches.

Add ArcXML layer: Done, though I have no clue how people actually use ArcXML, so this support may actually be total crap. I got it to work with the MassGIS site though. (Would love to talk to an actually ArcIMS/ArcXML user and figure out what I should be doing that I’m not.) I bet you’ve never seen ArcIMS look so fast.

Add KML Region Support: Not done, and not really too excited about it. I plan to get the 1.9 final out the door, then release the 2.0 without it, and maybe loop around for the KML region support later.

Overall, not bad.

• Fix threshhold bug, release 1.9, call it done
• Refactor to make support for dynamic loading of layer types easier (a la FeatureServer)
• Add ArcXML Layer (so that I can skip having to do it in OpenLayers)
• Add KML Region support

Just got notice that I’ll be giving two presentations at FOSS4G 2007:

FeatureServer: A REST-based Server for Simple Features
With the number of tools for creating vector data online growing rapidly over the past year, users have spent more and more of their time creating annotations of existing maps. Unfortunately, managing this user generated data in a web browser can be difficult: the existing tools for storing geographic data on the server are largely based around the WFS-T specification, which is not the most conducive to the browser environment. FeatureServer provides an alternative mechanism for fetching and storing geographic data, using a REST-based interface that is friendly to browsers and other clients alike.

And with Howard Butler:

The Gift Economy Ain’t Free: Getting Help with Open Source Software
Have you ever been told to RTFM? STFW? Sent an email to a project’s maillist that languished for days without a response? This talk will give you ammo that you can use to bust out of the rut of frustration and non-response.

Woot.

A long time ago, I played with Povray. Now, with a recent revamp of my website, I’ve decided to take another poke at playing with it. I’ll post my work here in the Povray category as I progress.

My first piece is Reflection:

A reflective sphere over a checkerboard sphere. I liked the look of the colored shadows — the mixing of the primary colors in each of the shadows resulted in the purple/orange/green shadows, even though the light sources are all primary colors. The sphere is white, with a high phong value (sheen) and relatively high reflectivity.