Technical Ramblings

The Symbian webkit browser is pretty nice, overall. It’s fast, it renders well, and other than the navigation options wrapped around it by a Nokia cell phone UI, it’s a relatively pain-free experience as a user.

As a web mapping developer, these things do not apply.

When attempting to build functionality to allow tap-to-pan on the map, Symbian’s event handling is just different enough to be unusably different from the point of view of a Javascript hacker.

When loading a page, the first tap on an element fires a mouseover event and a mousemove event — but no mousedown, mouseup, or click events. This behavior is tied to the first click on a page, but means that the common behavior of “This browser uses tap if mouseover/move/down/up/click happen at the same location at the same time” doesn’t apply here. (That behavior is used, for the record, by Opera, Firefox, and IE — though admittedly, IE’s mouseover brokenness breaks that particular aspect of it.)

Unlike most devices, mousemove happens when putting a finger down on the screen — not when you pick the finger up. However, no matter how fast you ‘tap’, the minimum amount of time until you get the mousedown/up/click events is 400ms. (This is presumably because the browser is waiting for a ‘double tap’ to indicate a zoom; OpenLayers implements much the same delay for detecting double taps.) Unfortunately, by firing the mousemove first, it is long enough before the other events that the behavior essentially mimics the browser.

These two mouse behaviors make Symbian stick out like a sore thumb in any comparison of modern mobile browsers. Unfortunately, it makes them look very *similar* to some desktop browser behavior, making it hard to implement useful mobile-specific functionality without user-agent sniffing.

For the past two days, I’ve been surrounded by 5-6 devices at all times, trying to figure out what to do about the *rest* of the browsers out there who might want to use OpenLayers.

After a lot of grimacing, grumbling, and fighting with these devices, I’ve come up with a couple of demos of the functionality I’ve been working towards: Tap-to-Pan, recentering map on a tap event. The biggest problem that I faced was trying to determine when a tap was a tap, rather than a click. Mobile browsers have tried hard to support ‘legacy events’ — mousedown, up, etc. when using their various touch or tap functionalities. (In fact, this very approach is what allows for Map Navigation on the Kindle.) However, as a result it can be difficult to determine the difference between a desktop client, where a single click should not center the map, and a mobile browser without touch support.

After much gnashing of teeth, I’ve got a solution that seems to work for:

• Firefox (built-in) on Maemo (N900)
• Firefox4 Beta on N900
• IE on WP7
• Opera Mobile on Symbian touch devices

You can see it in action on YouTube:

Tap-to-Pan on Nokia E7 (Opera)

Tap-to-Pan on IE7 on WP7

(Still to be done: Symbian’s built in browser on the E7. Unfortunately, mouse events are handled just differently enough that it’s very difficult to establish the difference between that browser and a desktop.)

It’s not ideal navigation, but as you can see, it allows one to move around the map on these devices, something that would be very difficult without any other sort of dragging support.

If you’d like to test it yourself, you can go to the tap-to-pan test page and try it out. Note that the code currently has an alert() in it to tell you whether it detects you’re on a tap browser or not; this alert happens in response to your first ‘click’ on the map. If you find that this is returning ‘false’ on an OS/platform/browser that you expect to be returning true, feel free to drop by #openlayers on irc.freenode.net, or send an email to the mailing lists with information on your platform so I can look into whether supporting it is practical. 🙂

Mobile browsers generally implement mouse events in a somewhat ad-hoc way, which is understandable, but this one is pretty confusing to me: Internet Explorer on the WP7 platform appears to return a clientX and clientY for the mouseover event which is changed depending on the current scale of the page. (mousedown, mouseup, and mousemove are unaffected.)

On most platforms, when you first click on a div, you will get a ‘click’ and ‘mouseover’ event which are in the same location. Getting these events in the same location at the same time is one of the few ways that you can determine you are actually getting a ‘tap’ from a mobile browser, as opposed to a ‘click’ from a desktop browser.

Unfortunately, if you have a page which IE has scaled — which it does by default — the mouseover event is modified; when zoomed out (the default), it looks about half of the click location, whereas when zoomed all the way in, it can be exactly double the click location.

You can test this on this URL in the OpenLayers sandbox; note that the mouseover event is only set the *first* time you click on an element in IE as well, but that’s not an uncommon behavior.

Here’s hoping that someday the browser on WP7 is a bit less… ‘different’. Perhaps someday it will feel less like I’m working with… well, IE.

I’m currently attending the OpenLayers Mobile Development Code Sprint in Lausanne, and finding myself completely frustrated by lack of any kind of reasonable touch / drag events for non iOS/Android browsers.

I was hoping that more than four months after the last update to QuirksMode’s table of touch events that it would be possible to report that browser had improved, but it appears that my hopes were misplaced.

IE on WP7, released in November? No touch events. (There is some talk about them existing in some various Microsoft presentations, so maybe this will come in whatever version of IE they ship on the next release of WP7… but who knows when that will be.)

Symbian’s browser on the E7? No touch events. Opera Mobile on any platform? No touch events. Firefox browser on Maemo? No touch events.

OpenLayers now has support for touch events in the browser — and more than 60% of mobile users won’t be able to take advantage of it.

Last year, in February, the author of the QuirksMode report said that “Any browser that does not support the touch events by the end of 2010 is out of the race.” As far as support for rich mobile mapping experiences go: I absolutely agree. It’s time to get your browsers in shape.

Edit: As a followup to this, someone on Twitter mentioned that “Firefox 4 Beta Adds Multi-touch Support http://on.mash.to/h4suCB”. Great! I even went ahead and flashed my N900 to PR 1.2 so I could actually use it. And what did I find out? No touch support. Further research shows that not only is there no touch support on anything other than Windows 7 (!!), *Mozilla created their own events for touch*. So instead of using the somewhat standard touchstart, Mozilla uses MozTouchDown, MozTouchMove, MozTouchUp. Ah well. More heartbreak.

The HTC HD7, with Windows Phone 7, is a damn cool phone.

I don’t say this because I’m some kind of corporate shill: I gave a talk last year at Boston’s BarCamp which was titled “Why I love my N900 — and you probably wouldn’t.” Despite working for Nokia, I realize that their products are not for everyone, and comparing Symbian to some of its competitors shows some obvious flaws.

Windows Phone 7, on the other hand, is a really fun operating system.

This phone is fast. Lightning fast. Blazing fast. There’s no ‘loading’ screens for seconds while an app starts up. Every time you press a button, something happens. On sheer ‘toy’ factor alone, the phone and its OS are very impressive. The first reaction that you have when you use the OS is “Wow”, all over.

The integration between apps and the OS is pretty nice. I took a picture, and I could share it via Flickr, Facebook, or Twitter, through their existing apps. Facebook news updates integrate into the contacts application, as do others (like Windows Live) though right now it seems that this is done at the OS level and not at the app level. Facebook integration into contacts includes everything you’d expect, pictures and so on — including phone numbers, something I never got from the N900 ‘facebook’ (Jabber) integration.

The calendar app is slick; the ‘Agenda’ view gives me all the information that I want, quickly and easily, and the home screen includes a clock and my next calendar appointment, making it so I don’t even need to

The browser is fast, like the rest of the system. Unlike most prior experiences in IE, the browser is competitive with most other mobile browsing experiences. (It doesn’t seem to have any touch event support, but the only thing I’ve ever used that does is iOS, so I’m biased here.)

Wifi handling is good; it switches easily to Wifi when it sees a saved access point, with no obvious problems.

The phone app is solid: getting to it is easy, answering calls has a good UI, and the voice quality is solid. Getting to the phone easily is something that the N900 definitely did not give me, and I was surprised by how much it annoyed me.

Voice search/activation is cool, and actually worked pretty well. (Getting to it is stupid — holding down the “Windows” button turns it on rather than, say, “Search” — but it works pretty well.)

The Marketplace application is nice; support for updates, and a reasonable UI for quickly finding what you’re looking for. (The search leaves something to be desired; I oftentimes find myself searching in ‘Apps’ and getting back music/videos first, which isn’t really what I want. But when I know what I’m looking for, it works relatively well.) Compared to the Ovi Store which I’m used to, it’s a breath of fresh air: again, fast, easy to find things, and it has lots of content. Of the top 10 free apps on the android market, 7 of them are also in the Microsoft Marketplace — everything from Kindle to Facebook to Twitter is available in the existing marketplace.

Live tiles are neat. Seeing them on TV/advertisements, I didn’t really care. But they actually work well, include information that’s helpful and useful at a quick glance. The homescreen UI also makes it easy to get easy access to the applications, contacts, etc. that I want — and doesn’t have an arbitrary limit on a number of home screens like other devices do.

Text messaging is easy to use. Text all over, in fact, is really pretty — the entire UI is just easy on the eyes, which helps a lot.

Pictures are easy. Picture quality is solid — as much as I expect from any point and shoot.

Despite the lack of ‘multitasking’, the app ‘stack’ is well managed — much better than iOS, which is always centered around the home screen. I can open an email, have it take me to a link, follow it to a YouTube video in the YouTube app, and then hit back to go all the way back to the original email.

Overall, I feel like the WP7 OS running on the HTC HD7 really feels like a solid competitor in the next generation of smartphones. It’s fast, clean, and smooth all over, and for everything from sending email to watching movies on Netflix, it’s a fun phone to use, and lets me get done everything that I want to do quickly and easily.

Now if only I could get cell reception in the damn office.

Earlier this weekend, I was pointed to a post about MBTiles, a new portable map tile distribution format. After a bit of hemming and hawing, I realized what it actually was, and realized that there wasn’t really much to hem and haw about.

This ‘new’ format is really nothing new, nor is it actually a ‘file format’ in the strictest sense of the word. What it is instead is:

1. An OS X binary that takes a bunch of files in directories on disk and writes them into a sqlite database.

Some of you may remember the old commercials for the iMac: “Step 1: Plug in. Step 2: Get connected. There is no step 3.” In this case, there isn’t even a step two. There’s nothing else here: not a specification document, not a reader of any kind, not even a description of what this magical ‘file format’ is.

Okay, so it’s not much of anything, but it’s not really a *bad* idea, it’s just that… it’s not the way I’d go about it.

First of all, if you actually want people to actually use something, you kind of have to have a reader and a writer. A promise of an iPad app improvement someway down the line along with some vague handwavey “saved 90% of your disk space!” statistics might be good for a flash in the pan in the Twitter-world echo chamber, but if you want success, you gotta do a bit better than that.

So, to outline what the MBTiles ‘file format’ actually seems to be: It’s a sqlite database with two simple tables.


CREATE TABLE metadata (name text, value text);
CREATE TABLE tiles (zoom_level integer, tile_column integer, tile_row integer, tile_data blob);

The metadata is not required to contain anything (so far as I can tell; possibly some reader tools might require it).

Now, a smart reader might notice that there is nothing very complex about this: any programming langauge that can interact with sqlite can create or access the MBTiles data — a very good thing. (It’s possible that not publishing this simple fact is because the Development Seed/MapBox folks plan to extend it and don’t want to actually make it so that other people are using it; dunno.) However, excepting that, as it is, there’s no reason (that I can see) that the code to create a cache should be in C!

For prototyping, or if your goal is to create and develop a ‘standards’ thing, you really want to be working in a language which is more widely understood and easier to prototype. I realize that this is a judgement call on my part, but for things where you encourage people to check it out and use it, you should be working in a language with a wider audience. For example, using the mb_tiles_importer on my tiles produced by TileCache gives me a database that has 4 rows… even though my directory only has two tiles in it, and 2 of the 4 rows are entirely empty. If the code were in Python, I might take a look and offer some feedback, but with it being an OS X only binary, or even a thinly documented C script, there’s almost nothing I can do to figure out what’s going on or help.

Add to this the fact that although this format has a writer, it has no open source reader of any kind that I can find. There’s some chat about various MapBox related software reading it, but no separation of this from MapBox — and with no description of the format, there’s not an excuse that it’s designed for people to write their own clients…

That said, I’ve gone ahead and written support into TileCache for the ‘format’ such as it is; I’m not convinced it’s the ideal thing to do, but the core concept of delivering a single file in the form of an SQLite database for tile data is a pretty solid goal.

Overall, the idea is reasonably sound: Delivering tiles in a single file is important, and sqlite is a nice, lightweight format for that that’s accessible from most C-based languages. Writing a quick cache format to read these things in TileCache was easy enough — because, as I said, there really isn’t much there. I didn’t write write support, because doing so seemed like it could be a waste of the MapBox folks want to ‘own’ this format (Hello, GeoPDF, how are you today) and are still developing it, and the only way that I was able to even do what I did was using a tool that I had to grab from a Github link I got over Twitter (and doesn’t appear to work right).

B+ for the idea. It’s a bit iffy on the implementation, but the core goal is sound. However, the way that it’s approached is a somewhat typical approach that I see lately: Publish first, actually create the thing you’re publishing about later. That type of attitude is the kind of thing that drives me — as a creator who puts a lot of time and thought into community interaction first and foremost — absolutely bonkers.

Clean it up, make it a spec, and describe some of the benefit and utility in a way that’s not tied directly to MapBox, and I can see this actually becoming a pretty regular thing for distributing files around. I can definitely see the value and benefit — with some metrics at larger scale — of doing this kind of thing for distributing larger tilesets. I just don’t want to fall into the Admiral Ackbar problem: “It’s a trap!”

OpenLayers is approaching the ‘Brave New World’ of a major API breakage for the first time since OpenLayers 2.0 in October of 2006.

Things that have changed in OpenLayers since then:

• Support for vector drawing.
• The existence of ‘spherical mercator’
• reprojection of objects
• More than a dozen new layer types
• The existence of TileCache, FeatureServer, and RESTful HTTP APIs for geodata
• A tile-serving service from OpenStreetMap (also, for OpenStreetMap: 4 major API changes)
• Over four dozen additional contributors to the source code
• Over 2000 trac tickets
• 10 2.x releases

Overall, is it any shock that it’s time for a pretty major change?

In order to facilitate rapid development, we’ve shifted development of OpenLayers ‘3.0’ to github; you can follow along (or fork you own) on the OpenLayers github project. To be honest, git scares the crap out of me; every time I’ve used it, I have consistently lost data as a result of not understanding the tool and using the wrong command. However, I fully realize I am a fuddy duddy who needs to get over his problems at this point. 🙂

Looking forward to seeing the future of the world where OpenLayers is even more awesome than it already is!

As of April 9th, MetaCarta has been acquired by Nokia, and I am now an employee of Nokia working on local search in the Ovi services. (Woohoo!)

When using the Python ‘boto’ library for accessing Amazon Web Services, to enable logging to a file at the ‘debug’ level, simply use the logging module’s configuration:

import logging
logging.basicConfig(filename="boto.log", level=logging.DEBUG)


Place this line near the top of your script, and logging will take place to a file in your current directory called “boto.log”.

I’m sure that this is obvious for most people who use the Python logging module, but this is new code to me, and it took me a fair bit of looking to find out how to enable logging; hopefully other people find it more easily now.

KML is linked. It is self-descriptive, and can rely entirely on following of links to obtain more information, whether that is styles or additional data.

However, the most common way of packaging KML is as KMZ — which is sort of like packaging an HTML page inside a zip file with all of its component parts. When this is done, web-based tools — like the Javascript support in browsers — lose all access to the data other than through a server side proxy (and even that isn’t a trivial thing to achieve). Styling information and related parts are not stored as separate resources on the web. The information available in the KML has suddenly become just another application-specific format.

If this were uncommon, it wouldn’t be such a shame; it’s certainly possible to distribute data like this for use cases where it is necessary, including offline use and other use cases. However, this is not a limited situation — in fact, more than 80% of KML made available on the web tends to be primarily available as KMZ. This packaging of KML leaves much to be desired, and limits the use of such data in web-based tools.

The web already has ways to compress data — gzip-based compression is common on many web servers (a tradeoff of CPU time for bandwidth), and works fine in all KML clients I’m aware of (including Google Earth and Google Maps). This lets your data exist on the web of resources and documents, rather than in a zipped up bundle.

My interest in this matter should be obvious: I work with mapping on the web. Ideally, I work with tools that don’t require server-side code — every piece of server side code you have to build is another heavy requirements placed on the users of any software. Browsers, as a common platform across which developers can code, are a worthwhile target, and trapping your data in KMZ hides it from browsers.

Free your KML! Publish on the Web! Don’t use KMZ!