Technical Ramblings

After Tim Schaub got pinch-zoom into trunk yesterday with some excellent work and debugging from him and Bruno Binet, I was excited to see people able to use OpenLayers for easy navigation of the map on browsers with touch events. Turns out, my excitement was misplaced: though the functionality is functional (and super-slick!) on iOS devices greater than v2.0, on the Android platform, it’s not.

Turns out, the reason is simple: Android browsers don’t implement multitouch at the Javascript level.

You can zoom an Android page by using two fingers to pinch, but those events are never delivered to the DOM as multiple touch events; instead, we simply get one touch, no matter how many fingers are interacting with the screen.

Using the Multitouch Test OpenLayers example, I’ve gotten reports from far and wide, from 2.1 all the way up to 2.3 (Gingerbread): the Android browser only supports one touch event. For comparison? iOS supports up to 5.

The OS supports these events: In fact, a multitouch-on-Android development article suggests behavior very similar to the browser events. (There is a slight difference; in the browser DOM, you don’t get different events for DOWN vs. POINTER_DOWN; they’re both a touchstart.) You have a series of move events, and each move has a list of touches attached to it — in iOS/Safari, these equate to the evt.touches array of Touch objects attached to the TouchEvents.

This documentation has been around for almost 3 years — the functionality existed in iOS before the T-Mobile G1 device even shipped (in October of 2008). The touch events in Android for single touches map pretty much exactly to Apple’s spec, but no multitouch support exists at the DOM level, and no clearly documented plans to.

I just hope that the plan on the Android side isn’t to wait until the webevents W3C group ships a Recommendation in 2012…

The Symbian webkit browser is pretty nice, overall. It’s fast, it renders well, and other than the navigation options wrapped around it by a Nokia cell phone UI, it’s a relatively pain-free experience as a user.

As a web mapping developer, these things do not apply.

When attempting to build functionality to allow tap-to-pan on the map, Symbian’s event handling is just different enough to be unusably different from the point of view of a Javascript hacker.

When loading a page, the first tap on an element fires a mouseover event and a mousemove event — but no mousedown, mouseup, or click events. This behavior is tied to the first click on a page, but means that the common behavior of “This browser uses tap if mouseover/move/down/up/click happen at the same location at the same time” doesn’t apply here. (That behavior is used, for the record, by Opera, Firefox, and IE — though admittedly, IE’s mouseover brokenness breaks that particular aspect of it.)

Unlike most devices, mousemove happens when putting a finger down on the screen — not when you pick the finger up. However, no matter how fast you ‘tap’, the minimum amount of time until you get the mousedown/up/click events is 400ms. (This is presumably because the browser is waiting for a ‘double tap’ to indicate a zoom; OpenLayers implements much the same delay for detecting double taps.) Unfortunately, by firing the mousemove first, it is long enough before the other events that the behavior essentially mimics the browser.

These two mouse behaviors make Symbian stick out like a sore thumb in any comparison of modern mobile browsers. Unfortunately, it makes them look very *similar* to some desktop browser behavior, making it hard to implement useful mobile-specific functionality without user-agent sniffing.

For the past two days, I’ve been surrounded by 5-6 devices at all times, trying to figure out what to do about the *rest* of the browsers out there who might want to use OpenLayers.

After a lot of grimacing, grumbling, and fighting with these devices, I’ve come up with a couple of demos of the functionality I’ve been working towards: Tap-to-Pan, recentering map on a tap event. The biggest problem that I faced was trying to determine when a tap was a tap, rather than a click. Mobile browsers have tried hard to support ‘legacy events’ — mousedown, up, etc. when using their various touch or tap functionalities. (In fact, this very approach is what allows for Map Navigation on the Kindle.) However, as a result it can be difficult to determine the difference between a desktop client, where a single click should not center the map, and a mobile browser without touch support.

After much gnashing of teeth, I’ve got a solution that seems to work for:

• Firefox (built-in) on Maemo (N900)
• Firefox4 Beta on N900
• IE on WP7
• Opera Mobile on Symbian touch devices

You can see it in action on YouTube:

Tap-to-Pan on Nokia E7 (Opera)

Tap-to-Pan on IE7 on WP7

(Still to be done: Symbian’s built in browser on the E7. Unfortunately, mouse events are handled just differently enough that it’s very difficult to establish the difference between that browser and a desktop.)

It’s not ideal navigation, but as you can see, it allows one to move around the map on these devices, something that would be very difficult without any other sort of dragging support.

If you’d like to test it yourself, you can go to the tap-to-pan test page and try it out. Note that the code currently has an alert() in it to tell you whether it detects you’re on a tap browser or not; this alert happens in response to your first ‘click’ on the map. If you find that this is returning ‘false’ on an OS/platform/browser that you expect to be returning true, feel free to drop by #openlayers on irc.freenode.net, or send an email to the mailing lists with information on your platform so I can look into whether supporting it is practical. 🙂

Mobile browsers generally implement mouse events in a somewhat ad-hoc way, which is understandable, but this one is pretty confusing to me: Internet Explorer on the WP7 platform appears to return a clientX and clientY for the mouseover event which is changed depending on the current scale of the page. (mousedown, mouseup, and mousemove are unaffected.)

On most platforms, when you first click on a div, you will get a ‘click’ and ‘mouseover’ event which are in the same location. Getting these events in the same location at the same time is one of the few ways that you can determine you are actually getting a ‘tap’ from a mobile browser, as opposed to a ‘click’ from a desktop browser.

Unfortunately, if you have a page which IE has scaled — which it does by default — the mouseover event is modified; when zoomed out (the default), it looks about half of the click location, whereas when zoomed all the way in, it can be exactly double the click location.

You can test this on this URL in the OpenLayers sandbox; note that the mouseover event is only set the *first* time you click on an element in IE as well, but that’s not an uncommon behavior.

Here’s hoping that someday the browser on WP7 is a bit less… ‘different’. Perhaps someday it will feel less like I’m working with… well, IE.

I’m currently attending the OpenLayers Mobile Development Code Sprint in Lausanne, and finding myself completely frustrated by lack of any kind of reasonable touch / drag events for non iOS/Android browsers.

I was hoping that more than four months after the last update to QuirksMode’s table of touch events that it would be possible to report that browser had improved, but it appears that my hopes were misplaced.

IE on WP7, released in November? No touch events. (There is some talk about them existing in some various Microsoft presentations, so maybe this will come in whatever version of IE they ship on the next release of WP7… but who knows when that will be.)

Symbian’s browser on the E7? No touch events. Opera Mobile on any platform? No touch events. Firefox browser on Maemo? No touch events.

OpenLayers now has support for touch events in the browser — and more than 60% of mobile users won’t be able to take advantage of it.

Last year, in February, the author of the QuirksMode report said that “Any browser that does not support the touch events by the end of 2010 is out of the race.” As far as support for rich mobile mapping experiences go: I absolutely agree. It’s time to get your browsers in shape.

Edit: As a followup to this, someone on Twitter mentioned that “Firefox 4 Beta Adds Multi-touch Support http://on.mash.to/h4suCB”. Great! I even went ahead and flashed my N900 to PR 1.2 so I could actually use it. And what did I find out? No touch support. Further research shows that not only is there no touch support on anything other than Windows 7 (!!), *Mozilla created their own events for touch*. So instead of using the somewhat standard touchstart, Mozilla uses MozTouchDown, MozTouchMove, MozTouchUp. Ah well. More heartbreak.

The HTC HD7, with Windows Phone 7, is a damn cool phone.

I don’t say this because I’m some kind of corporate shill: I gave a talk last year at Boston’s BarCamp which was titled “Why I love my N900 — and you probably wouldn’t.” Despite working for Nokia, I realize that their products are not for everyone, and comparing Symbian to some of its competitors shows some obvious flaws.

Windows Phone 7, on the other hand, is a really fun operating system.

This phone is fast. Lightning fast. Blazing fast. There’s no ‘loading’ screens for seconds while an app starts up. Every time you press a button, something happens. On sheer ‘toy’ factor alone, the phone and its OS are very impressive. The first reaction that you have when you use the OS is “Wow”, all over.

The integration between apps and the OS is pretty nice. I took a picture, and I could share it via Flickr, Facebook, or Twitter, through their existing apps. Facebook news updates integrate into the contacts application, as do others (like Windows Live) though right now it seems that this is done at the OS level and not at the app level. Facebook integration into contacts includes everything you’d expect, pictures and so on — including phone numbers, something I never got from the N900 ‘facebook’ (Jabber) integration.

The calendar app is slick; the ‘Agenda’ view gives me all the information that I want, quickly and easily, and the home screen includes a clock and my next calendar appointment, making it so I don’t even need to

The browser is fast, like the rest of the system. Unlike most prior experiences in IE, the browser is competitive with most other mobile browsing experiences. (It doesn’t seem to have any touch event support, but the only thing I’ve ever used that does is iOS, so I’m biased here.)

Wifi handling is good; it switches easily to Wifi when it sees a saved access point, with no obvious problems.

The phone app is solid: getting to it is easy, answering calls has a good UI, and the voice quality is solid. Getting to the phone easily is something that the N900 definitely did not give me, and I was surprised by how much it annoyed me.

Voice search/activation is cool, and actually worked pretty well. (Getting to it is stupid — holding down the “Windows” button turns it on rather than, say, “Search” — but it works pretty well.)

The Marketplace application is nice; support for updates, and a reasonable UI for quickly finding what you’re looking for. (The search leaves something to be desired; I oftentimes find myself searching in ‘Apps’ and getting back music/videos first, which isn’t really what I want. But when I know what I’m looking for, it works relatively well.) Compared to the Ovi Store which I’m used to, it’s a breath of fresh air: again, fast, easy to find things, and it has lots of content. Of the top 10 free apps on the android market, 7 of them are also in the Microsoft Marketplace — everything from Kindle to Facebook to Twitter is available in the existing marketplace.

Live tiles are neat. Seeing them on TV/advertisements, I didn’t really care. But they actually work well, include information that’s helpful and useful at a quick glance. The homescreen UI also makes it easy to get easy access to the applications, contacts, etc. that I want — and doesn’t have an arbitrary limit on a number of home screens like other devices do.

Text messaging is easy to use. Text all over, in fact, is really pretty — the entire UI is just easy on the eyes, which helps a lot.

Pictures are easy. Picture quality is solid — as much as I expect from any point and shoot.

Despite the lack of ‘multitasking’, the app ‘stack’ is well managed — much better than iOS, which is always centered around the home screen. I can open an email, have it take me to a link, follow it to a YouTube video in the YouTube app, and then hit back to go all the way back to the original email.

Overall, I feel like the WP7 OS running on the HTC HD7 really feels like a solid competitor in the next generation of smartphones. It’s fast, clean, and smooth all over, and for everything from sending email to watching movies on Netflix, it’s a fun phone to use, and lets me get done everything that I want to do quickly and easily.

Now if only I could get cell reception in the damn office.