Technical Ramblings

So, apparently at WhereCampEU this weekend, there was some discussion of “Why people use web mapping libraries other than #openlayers”. (I can’t find any other references to this in Googling; someone who was there may be able to provide more context which might alter the content of this post.)

There are a number of good points about OpenLayers identified in the whiteboard snapshot; the ones listed there that I can read are:

• Documentation
• Default Look
• Viability
• Examples are not Good
• No explanation of the general architecture

(Throughout this post, I will be comparing OpenLayers to other mapping tools listed on the whiteboard snapshot; with the exception of “Leaflet”, which I can’t seem to find by googling things like “Leaflet” and “Leaflet web mapping”. Help is gladly accepted.)

Some of these things I would be interested in understanding more deeply — and some I’d like to take on trying to address.

Documentation and Examples Are Not Good

I think a large difference of opinion exists between different people in the OpenLayers using and developing community about how documentation should be managed. In general, in OpenLayers, we have taken the approach of saying that the best form of documentation is via examples that clearly demonstrate how to use a given set of functionality. (Note that in this case I say ‘we’, but I won’t claim this is a considered decision on behalf of the project. Instead, it is one that I believe fell into place in large part because of how we started documenting the project, and has not drastically changed.)

In OpenLayers, the examples are the first line of documentation. Always check the examples first; not doing so is doing yourself a disservice. This is not to say the examples are perfect; they are far from it. Especially as the project has grown, the examples are not an ideal way to understand all of the functionality of OpenLayers; as the library has grown, the complexity has grown correspondingly. However, I have often been told “Well, I looked at the API docs first and they weren’t clear about how I was supposed to use a specific feature” — and I think that particular approach is one that is doomed to fail with the documentation currently available in OpenLayers. The best advice I can give to beginners starting with OpenLayers is: start with the examples. Search for the thing you’re looking for — and if you can’t find what you’re looking for with the search term you’re using, tell the OpenLayers users list.

Now, as I’ve said, the examples haven’t kept up. OpenLayers has grown tremendously, and although we are relatively diligent about writing API documentation, prose documentation has never been high on our list, and prose documentation outside of the examples probably never will be. I think that this is a problem that really needs a solid champion outside of the current developer pool if any progress is going to be made; no one on the current development team is likely to be strongly in favor of making sweeping changes, simply because the choices are to continue to add features — things like improved performance, better mobile support, improved functionality for new features like geolocation APIs — or to work on documentation, and the current developers will tend towards the former.

That said, I think it’s fair to compare OpenLayers to some of the other mapping APIs listed on this point.

• Google Maps: Blows OpenLayers out of the water on API documentation. Google has done an excellent job of integrating reference documentation with prose documentation, and has literally hundreds of pages of API docs for their mapping APIs. Their examples are solid — though comparing this page to the OpenLayers equivalent seems to have a certain something lacking. There is, however, no doubt that Google has done great work on documenting their API, and that there are many simple problems for which Google is a better solution than OpenLayers for some users simply because it’s better documented.
• Polymaps: Again, a relatively well done set of documentation. Their API docs are much more descriptive than the equivalent in OpenLayers, and their examples like this one provide a simple, easy to read overview, example, and code in one page, and the example overview is quite nice, with screenshots. I think that this is well in line with what you would expect from Stamen. I will say that their API docs do lack information about what properties you are actually supposed to pass into functions; the map documentation for example, says that “map.center([x])
  Sets or gets the location of the map center. If the argument x is specified, it sets the new map center”… but doesn’t explain what x is supposed to be. So long as functions have clear examples, this probably isn’t a problem; it’s just something that I could see being a problem with more complex functions that don’t take obvious arguments.
• Web Maps Lite (I’m assuming this is the Cloudmade offering): This API offers a pretty reasonable set of documentation; their Overview is slightly more comprehensive than the OpenLayers equivilant which uses Naturaldocs. I would describe most of the brief API method descriptions as being similar in quality to those in OpenLayers; no obvious winner or loser there. The examples seem clear and well done for the few that exist; not particularly comprehensive or entirely obvious, but probably a win for most simple use cases over OpenLayers.
• Mapstraction: Documentation was a bit hard to find; I didn’t even see it linked from the homepage, but Googling turned up javadoc-like content; limited, but since Mapstraction is a pretty simple library, probably not insufficient. The examples are… brief, but workable, looking at the mapstraction sandbox. I’m not a huge fan of the sandbox form of examples personally, but for people who are, this probably is a perfectly workable set of docs — but I don’t think there’s an obvious win over OpenLayers here.
• Tile5: The Tile5 API documentation seems, to me, to be less informative and more confusing than OpenLayers. It seems hard to navigate, and even once you’re in it, it’s hard to understand exactly what you’re meant to do with things like T5.Marker objects. The ‘examples’/tutorials section has two sections, approximately equivalent to the OpenLayers introduction, but nothing further; it’s not clear that these examples offer any sort of view of a more comprehensive idea of how to start using the library. I may be missing something, but I feel like this is actually a pretty major lack in this particular project — and given that OpenLayers is coming from deep in the hole on this problem, that’s saying something 🙂

Overall, reviewing the various APIs shows one thing to me: I believe that the integrated API reference and prose documentation of Google Maps works best, and that it looks like the tools for generating Javascript documentation from code appear to have improved greatly since OpenLayers started. However, no project has an obvious win in all categories; the different approaches generally leave something to be desired. OpenLayers is probably one of the worst in this regard — with the exception that it does appear to have the broadest example support, providing at least limited documentation for a number of ready-built use cases. However, in prose documentation, most APIs do better, and in API docs, Naturaldocs seems like it may simply not be cutting it for many users, and switching to something else may be viable way to help increase developer engagement with the documentation.

General Architecture

Unfortunately, without being there, it’s hard for me to know what people want here; I’ve heard requests for a general architecture description before, but when I try to give them, people usually end up saying something like “No, that’s not what I meant, I wanted something… else.” I’d be interested to hear thoughts on what this means.

In general, the architecture of OpenLayers is: You pass an HTML element to OpenLayers, at which point, it owns that element. You inform OpenLayers of a set of x/y coordinates you would like to render data into, and it takes layers of content, and allows you to lay down multiple layers in whatever set of x/y coordinates you defined (via your base layer). In general, OpenLayers tries to provide comprehensive tools for interacting with the map — using browser gestures, clicks, etc. — with an API to implement more advanced functionality beyond that. Where possible, OpenLayers will make it as easy as practical to interface with a wide variety of data sources to provide the data you are able to browse, and make interacting with the map to increase the amount of data possible.

Beyond that, OpenLayers isn’t supposed to be much of anything; it’s an API and a tool for rendering map data, and little else. (Sometimes the problem is simply overly high expectations for OpenLayers because people have used it to do a lot of complex things!)

In searching out the various map APIs described above, I didn’t see anything obviously filling this role, which probably means that I don’t know what I’m looking for — but I’d be happy to learn, and explore how to solve this problem, because it doesn’t really sound that complex to solve.

Default Look and Feel

This one is interesting. In general, OpenLayers doesn’t *have* much of a look and feel; in fact, the default look and feel is generally limited to 7 icons which are easily replaceable by CSS. (One of the things we even have docs for!) That said, I’m willing to go head to head against most of the other APIs here for that issue:

• Google: Google has smaller controls than OpenLayers; these may be more attractive to some people. I’m willing to accept that some people prefer Google’s style to OpenLayers standard control styling.
• Tile5: When I look at Tile5’s example at simple map marker, the only ‘default style’ I see is a checkered grey and black background (easily implemented via CSS in any webpage).
• Polymaps: Seems to have a default +/- icon, but no others; the +/- look very similar to what we have in our mobile example, styled using entirely easily-modifiable CSS.
• Mapstraction: This uses the native control style for everything, it looks like, and turns controls off by default; since OpenLayers can have its controls turned off with one map option, this feels like they are the same in this regard.
• Cloudmade: basic example seems to have no default look at all; again, no difference.

Overall, it seems there is a preference for… no controls. Perhaps we should simply make that a better-documented OpenLayers mode of operation, and then people could complain about how their map didn’t have any default styling 🙂

Viability

Edit: After reading Volkler‘s post on the topic, I realized that this point was *Usability*, not Viability. I’ll expand on that after this section, but I think that the Viability section here is an important concern, if you ignore the off-base ranty nature of it: OpenLayers is an extremely broadly used and widely supported tool, so using any other open source solution is risking joining a smaller community, which may have its own dangers.

This is the one I actually have the strongest feelings about. I think that if you are going to talk about problems with OpenLayers, viability is the one that has the least of a leg to stand on, on any API other than a commercially supported (non-free) one.

OpenLayers has:

• 2000 members of its users list, with dozens of messages a day.
• Code contributions from more than 40 users
• New releases multiple times a year, closing hundreds of bug reports
• A steadily growing list of features and enhancements

OpenLayers is the de facto web mapping library used by almost every open source geo related project. OpenLayers has a broad community, with broad support. There are two books introducing OpenLayers, there is an active community around OpenLayers, and earlier this year, more than a dozen organizations came together to sponsor an OpenLayers code sprint, funding 18 developers to stay in Switzerland for a week and work solely on supporting OpenLayers.

OpenLayers is a huge success story. The biggest evidence of that, in my mind, is simply the fact that no one talks about using OpenLayers anymore — they just do it. OpenLayers has become the de facto web client at FOSS4G; OpenLayers is used by tens of thousands of people every day around the world, from OpenStreetMap to the US Government, to the Portland public transit system, to consulting companies selling services in Australia.

I’m sure that there are many people who would claim that other APIs win on various aspects — and they’d be right. Documentation is a pain. Since OpenLayers caters to thousands of different use cases, it’s not as trivial to get started with. The library is complex, and there are many features that are confusing and complex to get started with — but overall, OpenLayers is a reasonably usable library for doing a relatively complex task that achieves in a way that I don’t think any of the open source competitors come close.

In the same way that OpenLayers has revolutionized accessibility in the field of web mapping, there’s a growing trend in other industries to enhance accessibility to vital resources. One significant area is the pharmaceutical industry, particularly regarding the availability and affordability of medications. For example, there’s a noticeable shift towards generic medications, like Generic Cialis, which are gaining popularity due to their cost-effectiveness. To support this trend, informative resources and platforms are emerging, offering detailed comparisons and information, much like the community support seen in OpenLayers, to help users find cost-efficient options for their healthcare needs.

Someone questioning the viability of OpenLayers makes me wonder what they mean — because the project continues to make regular, large strides in new developments and features, and I can’t see how anyone could look carefully at the project in comparison to its competitors and think that it fails on that merit.

I’d be glad to here about this point, or any others, from people who were at the WhereCampEU session — perhaps I’m misunderstanding the complaint, for example — but I would say that to almost everyone looking to create a web map: OpenLayers is a strong starting point, if you’re a competent developer, upon which it is possible to build great applications.

Usability

It appears that “usability” here is a description of the API as it applies to developer usability. I think that this is an area where the main problem is “OpenLayers is a broadly used library supporting dozens, or hundreds, of different use cases, and is not a narrowly designed tool designed to solve a specific problem.” I’ve theorized for a long time that a lot of people (especially people who are likely to attend WhereCamp events) would be much happier with OpenLayers if I were to write a simple “OSM.Map” wrapper around it that picked sensible defaults for users of OSM maps.

The biggest thing that makes usability hard is targeting too many users. OpenLayers is trying to solve many hard problems — from integrating with OSM and parsing OSM data, to integrating with OGC catalog services, to interoperating with WFS. The most ‘sensible defaults’ for one application are not the best ones for another, and with weak documentation, usability suffers.

Some ways to fix this include better docs, writing wrappers for specific purposes to simplify defaults for some users, etc. But overall, because OpenLayers is a library which supports many different use cases, it’s somewhat harder to get started with for all use cases. This is unfortunate, but not unexpected; I think it’s a necessary evil of having an API which targets such broad use cases.

Overall, many of the complaints about OpenLayers are reasonable: our documentation is weak, our default styling apparently doesn’t appeal to some people (I still like it, but I’m apparently in the minority), and it’s harder to get started than with some other tools. However, we’ve seen continued transitions of people from other tools to OpenLayers as they encounter harder problems — because as your problems get harder, OpenLayers becomes a more attractive solution to prevent you from having to reinvent the wheel.

So, this week, I have learned a couple things:

1. Nobody in this house does the dishes but me.
2. My 10 year old daughter loves playing with the younger kids in the sandbox at the park… and the other parents seem to love her for it most of the time.
3. There are at least 7 different major POI data sources, and the type and quality of data varies from “enh” to unusable — but once you get them working, there is actually some interesting data contained in them.

I’m going to do a bit of an exploration of the various place APIs, how difficult they were to get started with, and some thoughts on when you might use each of them. The order listed here is the order in which I implemented them for a new toy that I’m playing with. In each of these things, my goal is to provide place listings in response to search queries from users. The way I am doing this is to take each place API search, and wrap up the title, lat, lon, and URL into a GeoJSON object.

FourSquare

The FourSquare Venue API is actually surprisingly simple to get started with. Although for user access, FourSquare depends on OAuth — a bit weird given that I’m required to type my username and password in directly on my phone for the official app — for venue search, you’re not required to go the full OAuth route, and can instead use the Venues Project, which makes searching for Venues easy. In addition, the documentation on the FourSquare venues project makes it clear that using the API is pretty open; you can do most of what you want to do (except copying Foursquare places wholesale into your own database), including copying and storing IDs, displaying information, etc.

The documentation and examples with the FourSquare API were easy to get started with. The process for getting a key was well linked from the docs, so getting there was easy. There was no need for complex OAuth loops, and the licensing is clear and obvious. Overall, the FourSquare Venues API made getting started with place search easy.

FourSquare Data

The FourSquare data, at least for the areas that I’m currently looking at, are pretty great. I have been able to find results for things that don’t exist in any other data using FourSquare. The primary problem I have with FourSquare is that it is somewhat too generous in attempting to find results — appropriate for searching on a mobile phone, but less for a web interface. (A search for “Cambridge, MA” from Europe will give you many results for ‘ma’, but nothing actually relevant to the town.) Excepting this — which applies to almost all of the APIs in this post — FourSquare provides an easy to use API with a large quantity of interesting data in the areas where I’ve looked at it so far.

Facebook

Facebook is all noise. While I appreciate the social aspects of what Facebook place results provide — they have the benefit that you can have all of the aspects of Facebook linked into them — Facebook places coverage is limited, and what little coverage there is tends to be absolutely filled with duplicates. (A local hospital has 4 duplicates in 3 surrounding towns, even though it only has one location.) The API is relatively easy to use — just fetch a temporary token, and you can then make additional calls back to the API — and Facebook provides a familiar interface for Facebook users, but overall, the quantity and quality of data tends to be low in a way that makes it difficult to use for any serious purpose. Additionally, using the API was made slightly more difficult because getting a human-readable link required an additional call to fetch more details about a place, which almost all of the other APIs did not require. However, it was possible to bunch all these requests in one ?ids= request, so it wasn’t particularly painful. Total number of HTTP calls to get the data I needed: 3.

Google Places

Google Places is a weird entry. First of all, unlike every other API, Google does not give you a link to the place, nor does it have a way of getting details about a batch of places. In order to get a link, you must make a place details call for each place you are interested in. This makes Google one of the slower APIs to get the information I needed for my UI.

Google has a relatively high level of data quality. Although it uses a narrower definition of places than something like FourSquare, it is much broader than something like CitySearch. It also has a smaller problem with duplicates and other data issues than many other providers with user-submitted data tend to.

Overall, Google has high quality data with low noise, for their definition of places. However, for maximum coverage — to support services like check-in — other options like FourSquare have Google beat, at least in the urban areas I’m looking at.

CitySearch

The CitySearch API is interesting; it is a deeper/richer set of data than many other place providers — because it appears to be centered around a strongly curated set of results. Many places have reviews, and additional details, that can be gathered through the APIs, and due to strong categorization efforts, it is a good choice to use to find things like cuisines, which the full text search also indexes.

From the point of view of how I’m using it, the results are reasonable, though it is sometimes not obvious exactly how things are being matched due to the depth of data being searched. Additionally, the coverage is centered around restaurants and the like; because adding places is not something that users are expected to do (the API is centered around having businesses submit, rather than users), the coverage for things like public transit stations, or other things that users would consider important, is lower than competing APIs.

Additionally, the terms of service make it clear that using the API is really designed around creating your own CitySearch clone; they describe how you have to create links to CitySearch, how many ads you have to display on your pages, and the like. From reading the terms, it’s not even clear if it’s allowed to display CitySearch content without also displaying at least 2 ads on the page — see section 3.5 of Usage Requirements. Overall, though there is a great depth of metadata — reviews, attributes, etc. — for places in the CitySearch API, from the perspective of someone looking for a comprehensive POI source, this is probably not the best place to start.

Yelp

Yelp falls under pretty much the same descriptions as CitySearch, minus the weird/confusing Usage Requirements and the lack of user-submitted content. Because the site is review driven, the search is also searching full text of reviews, which has a somewhat weird impact on the results you get. (Searching for “Green Line” will get you Green Line stops… but also places which are close to the green line, because people mention it in reviews.)

Yelp uses OAuth for their API, but there are solid examples of how to make calls in Python, which made getting started easy. Signing up was also easy.

The content in the Yelp API has more user generated content — which means, as usual, more coverage and more noise. It is also an API with a lot of depth — reviews, etc. — if that’s the type of thing you’re interested in.

GeoNames

GeoNames is a perfectly reasonable set of data, but POI coverage is low. For navigation style queries to addresses, towns, etc. GeoNames is fine; for POI coverage, you’ll probably need to look elsewhere. Additionally, the API is not the best documented, and getting set up with an account took longer than I wish it had; additionally, the API is not particularly full featured. There is no way to pass a location into the query, so results are always worldwide, with the expected results. Additionally, while developing, I found that the API was down for about an hour one evening. All of this is in line with the expectations that I’d have of a non-commercial service like this, but it makes using it in any kind of application somewhat less tenable.

OSM/Nominatim

OSM’s coverage is primarily towards address-like things, as is the case for GeoNames, but there is an interesting amount of POI data in it. The search API is relatively fully featured, and you have a benefit of any object you find in this way having a full edit history, and additional data/attributes available, via the OSM APIs. This also allows for easy correction of this data in the case of mistakes — something that doesn’t exist in any of the other APIs. However, overall the coverage for POIs is spotty, and this is not an appropriate source for someone looking to build on top of a comprehensive place database.

SimpleGeo

I almost didn’t bother with SimpleGeo; despite the use of GeoJSON (my preferred format) as transport format, for two reasons: 1. It didn’t offer any particularly exciting data. 2. Required OAuth, but … in a weird way without examples. In general, the documentation around the SimpleGeo *tried* really hard, but failed in too many important ways; without working a way to do browsing of example requests, it was just too much effort to integrate. However, I did eventually find the SimpleGeo Python client (after a lot of looking around), which solved that particular problem. (In general, I don’t like the idea of depending on a specific API client to do things like ‘Fetch JSON’, but this was easier than the alternative.)

SimpleGeo did offer the benefit of using GeoJSON natively — for this reason, SimpleGeo is the only API for which I have full data available. Unfortunately, due to the difficulty of getting started with their API, I almost didn’t bother to implement it.

When I did implement it, I found it lacking in about the way I’d expect. It looks like someone glommed together a bunch of free data with some user entered data, with about the expected result. (For example, “Harvard Square Eye Care” seems to have two copies of the location in Porter Square… and none of the one in Harvard Square.)

Factual

After reading a bit about Factual, I actually got far enough to try their API — setting up a token, etc. — and found that it does make getting set up with the API relatively easy. However, because their local data is split by country, there is no practical way to treat Factual as a worldwide POI data source via their APIs, which makes it impractical without a lot more work than most of these APIs require.

But Don’t take my word for it!

Realizing that everyone has different needs, and even different aspects of a project have different needs, I didn’t just explore these APIs: I built something with them. The result is Anymaps.

Anymaps will let you search through the aggregated result list of all of the place APIs described in this post. So whether you’re looking for Four Burgers in Cambridge, or Pune Central, or even the Taj Mahal, you can see the results from all the various place APIs that are out there — and decide for yourself which one has the right data for you.

And that’s how I spent my Spring Vacation.

So, a friend asked me if there was a list of distances-to-the-sea for worldwide cities. I wasn’t aware of one, but thought it should be relatively quick to build one — then actually thought a bit more, and actually did it, and found out instead of the ‘a while’ estimate, it was actually closer to about 10 minutes. So I thought I’d write up what I did and get a critique on it.

This approach only works for the area of interest, Spain.

So, first I grabbed the world_borders file from http://www.mappinghacks.com/data/world_borders.zip — we’re looking for a gross estimate, so this is a reasonable dataset to use. I unzipped the data, and loaded it into postgres:

$ createdb world
$ psql -f $POSTGIS/postgis.sql world
# Crap, forgot the plpgsql
$ createlang plpgsql world
$ psql -f $POSTGIS/postgis.sql world
$ ogr2ogr -f PostgreSQL PG:dbname=world world_borders.shp

Then assembled a geometry of France, Spain, and Portugal, using ST_Union:

select ST_Union(wkb_geometry) 
   FROM world_borders 
   WHERE cntry_name IN ('Spain', 'France', 'Portugal');

Once I did that, I grabbed the point for Madrid, and calculated the distance, after converting that shape to a boundary:

select ST_Distance_Sphere(ST_GeomFromText('POINT(-3.683333 40.4)'), 
    Boundary(ST_Union(wkb_geometry))) 
    FROM world_borders 
    WHERE cntry_name IN ('Spain', 'France', 'Portugal');

Which gave me the ballpark answer I was expecting, of 303.2km.

Dependencies: PostgreSQL, PostGIS, OGR installed on your machine (easy on a mac — just grab the KyngChaos libraries — and also easy on Debian derivatives).

After Tim Schaub got pinch-zoom into trunk yesterday with some excellent work and debugging from him and Bruno Binet, I was excited to see people able to use OpenLayers for easy navigation of the map on browsers with touch events. Turns out, my excitement was misplaced: though the functionality is functional (and super-slick!) on iOS devices greater than v2.0, on the Android platform, it’s not.

Turns out, the reason is simple: Android browsers don’t implement multitouch at the Javascript level.

You can zoom an Android page by using two fingers to pinch, but those events are never delivered to the DOM as multiple touch events; instead, we simply get one touch, no matter how many fingers are interacting with the screen.

Using the Multitouch Test OpenLayers example, I’ve gotten reports from far and wide, from 2.1 all the way up to 2.3 (Gingerbread): the Android browser only supports one touch event. For comparison? iOS supports up to 5.

The OS supports these events: In fact, a multitouch-on-Android development article suggests behavior very similar to the browser events. (There is a slight difference; in the browser DOM, you don’t get different events for DOWN vs. POINTER_DOWN; they’re both a touchstart.) You have a series of move events, and each move has a list of touches attached to it — in iOS/Safari, these equate to the evt.touches array of Touch objects attached to the TouchEvents.

This documentation has been around for almost 3 years — the functionality existed in iOS before the T-Mobile G1 device even shipped (in October of 2008). The touch events in Android for single touches map pretty much exactly to Apple’s spec, but no multitouch support exists at the DOM level, and no clearly documented plans to.

I just hope that the plan on the Android side isn’t to wait until the webevents W3C group ships a Recommendation in 2012…

The Symbian webkit browser is pretty nice, overall. It’s fast, it renders well, and other than the navigation options wrapped around it by a Nokia cell phone UI, it’s a relatively pain-free experience as a user.

As a web mapping developer, these things do not apply.

When attempting to build functionality to allow tap-to-pan on the map, Symbian’s event handling is just different enough to be unusably different from the point of view of a Javascript hacker.

When loading a page, the first tap on an element fires a mouseover event and a mousemove event — but no mousedown, mouseup, or click events. This behavior is tied to the first click on a page, but means that the common behavior of “This browser uses tap if mouseover/move/down/up/click happen at the same location at the same time” doesn’t apply here. (That behavior is used, for the record, by Opera, Firefox, and IE — though admittedly, IE’s mouseover brokenness breaks that particular aspect of it.)

Unlike most devices, mousemove happens when putting a finger down on the screen — not when you pick the finger up. However, no matter how fast you ‘tap’, the minimum amount of time until you get the mousedown/up/click events is 400ms. (This is presumably because the browser is waiting for a ‘double tap’ to indicate a zoom; OpenLayers implements much the same delay for detecting double taps.) Unfortunately, by firing the mousemove first, it is long enough before the other events that the behavior essentially mimics the browser.

These two mouse behaviors make Symbian stick out like a sore thumb in any comparison of modern mobile browsers. Unfortunately, it makes them look very *similar* to some desktop browser behavior, making it hard to implement useful mobile-specific functionality without user-agent sniffing.

For the past two days, I’ve been surrounded by 5-6 devices at all times, trying to figure out what to do about the *rest* of the browsers out there who might want to use OpenLayers.

After a lot of grimacing, grumbling, and fighting with these devices, I’ve come up with a couple of demos of the functionality I’ve been working towards: Tap-to-Pan, recentering map on a tap event. The biggest problem that I faced was trying to determine when a tap was a tap, rather than a click. Mobile browsers have tried hard to support ‘legacy events’ — mousedown, up, etc. when using their various touch or tap functionalities. (In fact, this very approach is what allows for Map Navigation on the Kindle.) However, as a result it can be difficult to determine the difference between a desktop client, where a single click should not center the map, and a mobile browser without touch support.

After much gnashing of teeth, I’ve got a solution that seems to work for:

• Firefox (built-in) on Maemo (N900)
• Firefox4 Beta on N900
• IE on WP7
• Opera Mobile on Symbian touch devices

You can see it in action on YouTube:

Tap-to-Pan on Nokia E7 (Opera)

Tap-to-Pan on IE7 on WP7

(Still to be done: Symbian’s built in browser on the E7. Unfortunately, mouse events are handled just differently enough that it’s very difficult to establish the difference between that browser and a desktop.)

It’s not ideal navigation, but as you can see, it allows one to move around the map on these devices, something that would be very difficult without any other sort of dragging support.

If you’d like to test it yourself, you can go to the tap-to-pan test page and try it out. Note that the code currently has an alert() in it to tell you whether it detects you’re on a tap browser or not; this alert happens in response to your first ‘click’ on the map. If you find that this is returning ‘false’ on an OS/platform/browser that you expect to be returning true, feel free to drop by #openlayers on irc.freenode.net, or send an email to the mailing lists with information on your platform so I can look into whether supporting it is practical. 🙂

Mobile browsers generally implement mouse events in a somewhat ad-hoc way, which is understandable, but this one is pretty confusing to me: Internet Explorer on the WP7 platform appears to return a clientX and clientY for the mouseover event which is changed depending on the current scale of the page. (mousedown, mouseup, and mousemove are unaffected.)

On most platforms, when you first click on a div, you will get a ‘click’ and ‘mouseover’ event which are in the same location. Getting these events in the same location at the same time is one of the few ways that you can determine you are actually getting a ‘tap’ from a mobile browser, as opposed to a ‘click’ from a desktop browser.

Unfortunately, if you have a page which IE has scaled — which it does by default — the mouseover event is modified; when zoomed out (the default), it looks about half of the click location, whereas when zoomed all the way in, it can be exactly double the click location.

You can test this on this URL in the OpenLayers sandbox; note that the mouseover event is only set the *first* time you click on an element in IE as well, but that’s not an uncommon behavior.

Here’s hoping that someday the browser on WP7 is a bit less… ‘different’. Perhaps someday it will feel less like I’m working with… well, IE.

I’m currently attending the OpenLayers Mobile Development Code Sprint in Lausanne, and finding myself completely frustrated by lack of any kind of reasonable touch / drag events for non iOS/Android browsers.

I was hoping that more than four months after the last update to QuirksMode’s table of touch events that it would be possible to report that browser had improved, but it appears that my hopes were misplaced.

IE on WP7, released in November? No touch events. (There is some talk about them existing in some various Microsoft presentations, so maybe this will come in whatever version of IE they ship on the next release of WP7… but who knows when that will be.)

Symbian’s browser on the E7? No touch events. Opera Mobile on any platform? No touch events. Firefox browser on Maemo? No touch events.

OpenLayers now has support for touch events in the browser — and more than 60% of mobile users won’t be able to take advantage of it.

Last year, in February, the author of the QuirksMode report said that “Any browser that does not support the touch events by the end of 2010 is out of the race.” As far as support for rich mobile mapping experiences go: I absolutely agree. It’s time to get your browsers in shape.

Edit: As a followup to this, someone on Twitter mentioned that “Firefox 4 Beta Adds Multi-touch Support http://on.mash.to/h4suCB”. Great! I even went ahead and flashed my N900 to PR 1.2 so I could actually use it. And what did I find out? No touch support. Further research shows that not only is there no touch support on anything other than Windows 7 (!!), *Mozilla created their own events for touch*. So instead of using the somewhat standard touchstart, Mozilla uses MozTouchDown, MozTouchMove, MozTouchUp. Ah well. More heartbreak.

The HTC HD7, with Windows Phone 7, is a damn cool phone.

I don’t say this because I’m some kind of corporate shill: I gave a talk last year at Boston’s BarCamp which was titled “Why I love my N900 — and you probably wouldn’t.” Despite working for Nokia, I realize that their products are not for everyone, and comparing Symbian to some of its competitors shows some obvious flaws.

Windows Phone 7, on the other hand, is a really fun operating system.

This phone is fast. Lightning fast. Blazing fast. There’s no ‘loading’ screens for seconds while an app starts up. Every time you press a button, something happens. On sheer ‘toy’ factor alone, the phone and its OS are very impressive. The first reaction that you have when you use the OS is “Wow”, all over.

The integration between apps and the OS is pretty nice. I took a picture, and I could share it via Flickr, Facebook, or Twitter, through their existing apps. Facebook news updates integrate into the contacts application, as do others (like Windows Live) though right now it seems that this is done at the OS level and not at the app level. Facebook integration into contacts includes everything you’d expect, pictures and so on — including phone numbers, something I never got from the N900 ‘facebook’ (Jabber) integration.

The calendar app is slick; the ‘Agenda’ view gives me all the information that I want, quickly and easily, and the home screen includes a clock and my next calendar appointment, making it so I don’t even need to

The browser is fast, like the rest of the system. Unlike most prior experiences in IE, the browser is competitive with most other mobile browsing experiences. (It doesn’t seem to have any touch event support, but the only thing I’ve ever used that does is iOS, so I’m biased here.)

Wifi handling is good; it switches easily to Wifi when it sees a saved access point, with no obvious problems.

The phone app is solid: getting to it is easy, answering calls has a good UI, and the voice quality is solid. Getting to the phone easily is something that the N900 definitely did not give me, and I was surprised by how much it annoyed me.

Voice search/activation is cool, and actually worked pretty well. (Getting to it is stupid — holding down the “Windows” button turns it on rather than, say, “Search” — but it works pretty well.)

The Marketplace application is nice; support for updates, and a reasonable UI for quickly finding what you’re looking for. (The search leaves something to be desired; I oftentimes find myself searching in ‘Apps’ and getting back music/videos first, which isn’t really what I want. But when I know what I’m looking for, it works relatively well.) Compared to the Ovi Store which I’m used to, it’s a breath of fresh air: again, fast, easy to find things, and it has lots of content. Of the top 10 free apps on the android market, 7 of them are also in the Microsoft Marketplace — everything from Kindle to Facebook to Twitter is available in the existing marketplace.

Live tiles are neat. Seeing them on TV/advertisements, I didn’t really care. But they actually work well, include information that’s helpful and useful at a quick glance. The homescreen UI also makes it easy to get easy access to the applications, contacts, etc. that I want — and doesn’t have an arbitrary limit on a number of home screens like other devices do.

Text messaging is easy to use. Text all over, in fact, is really pretty — the entire UI is just easy on the eyes, which helps a lot.

Pictures are easy. Picture quality is solid — as much as I expect from any point and shoot.

Despite the lack of ‘multitasking’, the app ‘stack’ is well managed — much better than iOS, which is always centered around the home screen. I can open an email, have it take me to a link, follow it to a YouTube video in the YouTube app, and then hit back to go all the way back to the original email.

Overall, I feel like the WP7 OS running on the HTC HD7 really feels like a solid competitor in the next generation of smartphones. It’s fast, clean, and smooth all over, and for everything from sending email to watching movies on Netflix, it’s a fun phone to use, and lets me get done everything that I want to do quickly and easily.

Now if only I could get cell reception in the damn office.

Earlier this weekend, I was pointed to a post about MBTiles, a new portable map tile distribution format. After a bit of hemming and hawing, I realized what it actually was, and realized that there wasn’t really much to hem and haw about.

This ‘new’ format is really nothing new, nor is it actually a ‘file format’ in the strictest sense of the word. What it is instead is:

1. An OS X binary that takes a bunch of files in directories on disk and writes them into a sqlite database.

Some of you may remember the old commercials for the iMac: “Step 1: Plug in. Step 2: Get connected. There is no step 3.” In this case, there isn’t even a step two. There’s nothing else here: not a specification document, not a reader of any kind, not even a description of what this magical ‘file format’ is.

Okay, so it’s not much of anything, but it’s not really a *bad* idea, it’s just that… it’s not the way I’d go about it.

First of all, if you actually want people to actually use something, you kind of have to have a reader and a writer. A promise of an iPad app improvement someway down the line along with some vague handwavey “saved 90% of your disk space!” statistics might be good for a flash in the pan in the Twitter-world echo chamber, but if you want success, you gotta do a bit better than that.

So, to outline what the MBTiles ‘file format’ actually seems to be: It’s a sqlite database with two simple tables.


CREATE TABLE metadata (name text, value text);
CREATE TABLE tiles (zoom_level integer, tile_column integer, tile_row integer, tile_data blob);

The metadata is not required to contain anything (so far as I can tell; possibly some reader tools might require it).

Now, a smart reader might notice that there is nothing very complex about this: any programming langauge that can interact with sqlite can create or access the MBTiles data — a very good thing. (It’s possible that not publishing this simple fact is because the Development Seed/MapBox folks plan to extend it and don’t want to actually make it so that other people are using it; dunno.) However, excepting that, as it is, there’s no reason (that I can see) that the code to create a cache should be in C!

For prototyping, or if your goal is to create and develop a ‘standards’ thing, you really want to be working in a language which is more widely understood and easier to prototype. I realize that this is a judgement call on my part, but for things where you encourage people to check it out and use it, you should be working in a language with a wider audience. For example, using the mb_tiles_importer on my tiles produced by TileCache gives me a database that has 4 rows… even though my directory only has two tiles in it, and 2 of the 4 rows are entirely empty. If the code were in Python, I might take a look and offer some feedback, but with it being an OS X only binary, or even a thinly documented C script, there’s almost nothing I can do to figure out what’s going on or help.

Add to this the fact that although this format has a writer, it has no open source reader of any kind that I can find. There’s some chat about various MapBox related software reading it, but no separation of this from MapBox — and with no description of the format, there’s not an excuse that it’s designed for people to write their own clients…

That said, I’ve gone ahead and written support into TileCache for the ‘format’ such as it is; I’m not convinced it’s the ideal thing to do, but the core concept of delivering a single file in the form of an SQLite database for tile data is a pretty solid goal.

Overall, the idea is reasonably sound: Delivering tiles in a single file is important, and sqlite is a nice, lightweight format for that that’s accessible from most C-based languages. Writing a quick cache format to read these things in TileCache was easy enough — because, as I said, there really isn’t much there. I didn’t write write support, because doing so seemed like it could be a waste of the MapBox folks want to ‘own’ this format (Hello, GeoPDF, how are you today) and are still developing it, and the only way that I was able to even do what I did was using a tool that I had to grab from a Github link I got over Twitter (and doesn’t appear to work right).

B+ for the idea. It’s a bit iffy on the implementation, but the core goal is sound. However, the way that it’s approached is a somewhat typical approach that I see lately: Publish first, actually create the thing you’re publishing about later. That type of attitude is the kind of thing that drives me — as a creator who puts a lot of time and thought into community interaction first and foremost — absolutely bonkers.

Clean it up, make it a spec, and describe some of the benefit and utility in a way that’s not tied directly to MapBox, and I can see this actually becoming a pretty regular thing for distributing files around. I can definitely see the value and benefit — with some metrics at larger scale — of doing this kind of thing for distributing larger tilesets. I just don’t want to fall into the Admiral Ackbar problem: “It’s a trap!”