Technical Ramblings

Russ posted about a pretty cool Flash Lite application that was developed: a way to look at the NYC traffic cameras using your phone. It’s an extremely cool app – if I lived in New York, I’d buy Flash Lite just to be able to use that application. One thing that Russ mentioned was the development time for the project: 20 hours of development time, when something in Java would have been way larger.

Well, I’m a Python man, not a Flash man, so I can’t get much out of this yet. However, I do think it’s a cool use case: so I did a little research, found out where the data that the Flash app uses was coming from, and did a little hacking. The result? TrafficCam version 0.1, in Python. This little app took me 45 minutes to develop a fully functional prototype: this included taking the HTML from the NY Transportation site, building it into a Python file, creating a user interface, and downloading and displaying the image in the built in phone image viewer.

Not only did I do all this in 45 minutes, I did it without even having a phone to test with. Passing it off to the owner of a 6600 and a 6630, both say it works just fine, as is.

(Note that I think it probably doesn’t, but in ways that aren’t visible: There has been no testing done yet.)

So, although Flash is great for pretty apps – the Python app is *nothing* like the Flash app, which is a great user interface and something that’s really fun to use, even in a browser – but Python can be really great for *quick* apps, especially on the phone.

Update: With another 35 minutes of work, I now have fully functioning tabbed lists, one for each borough. So, with a total of less than an hour and a half of development, I have an application which allows you to download any of the traffic cam images and view them on your phone. It’s no flash, but I call that pretty damn impressive.

Around the time that Nokia Python was released, someone mentioned that it would be very interestig to see a replacement for the built in Nokia Notepad program. The Notepad program has many limitations: you can’t fit much text on a screen, there’s no easy way to browse or link notes you’ve written, all the typical limitations that you see when writing in a small paper notepad.

It was discussed that a small, wiki-like implementation might be a way to improve the usability of the notepad. Allow people to link between pages or nodes, let them type what they want and link it together. This would greatly improve the usability of having a phone along with you: if you can take notes on it that you can actually usee, there’s less chance you’ll lose them than paper, and you’re far more likely to be able to use something that’s connected than just write one note to yourself.

For a while, I didn’t develop anything on the phone, mostly due to technical issues related to my development process. However, after the recent Python SIG meeting, I decided that the next time I saw people, I wanted the phone to do something useful. So, I took on the challenge: a wiki-like scratchpad.

I started last night, and didn’t get too far before I ran into some symbian related problems: I couldn’t get anydbm to save anything. The file would open, it would stay open, but it wouldn’t sync, and it wouldn’t close. This morning, I finally looked at some code, and realized – wait. I need a full path. And once I got that far, things started looking up. In the hour I had this morning before work, I hacked out the basics: open a node, write to it, save, exit. Paavo on #mobitopia pointed out some other issues: the exit key handler, for one.

Then, over lunch, I created a webpage, and fleshed it out a bit more: adding a feature such that when you hit the action button, the word underneath the cursor is opened as a new node. This was about another two hours of work, spread over lunch and 10 minute breaks from work throughout the day. This included writing a program that catches keypresses and displays their associated keycodes, including learning a bit more about lambdas than I used to. I then had a working wikipad, and have since been adding minor cleanup tweaks.

In the past hour, I’ve created a beta, which allows uploading the wiki database to a webpage as POST data, from which it can be done with what you wish. (I’m currently storing the data in print_r() form, just for the sake of doing so.) From here, it’s pretty easy to imagine how this could allow reimporting the data.

So, the next step is, of course, making this work in a collaborative sense: allowing multiple people to edit one wiki via their phones. However, for the time being, it’s a great little scratchpad app, that’s only likely to get greater. You can get the code from the WikiPad Homepage.

Recently, a number of local Python users have assembled some form of organization, to the point that there is actually now relatively regular meetings of these groups of people (before other Linux Users Group gatherings, thus far). With the recent Nokia Python announcement, there’s been some renewed interest in my mobile python work, so I’m hopefully going to get some of that into shape over the weekend for a demonstration to the group on Monday, assuming I can make the meeting.

For those of you who have an interest in Python: What do you think would be interesting to a bunch of Python coders as a demonstration? Is there something that’s particularly spiffy that I could show off, or convert from being a command line application to being a cell phone application? Note thave I’m thinking relatively simply here: I only have a relatively limited memory space to work in, and I only have a small subset of modules to work with, and I’m in Python 2.2.

So, what do you see as being interesting topics/programs to demonstrate to the world the power of Python on the cell phone?

I’m really looking forward to getting together with a bunch of like-minded hackers, and racking their brains on what I can do better. I’ve never really had a good development process before, but Python developers seem to have one, especially the ones that I’ve seen discussing things on the mailing list. I’m used to LiveJournal’s spaghetti code, or writing in PHP which is typically not so well tested. It’ll be interesting to enter conversations with a group of more “formal” developers than myself.

Just looking for thoughts on what I should be working on.

One of the nifties things I have on my cell phone is a bluetooth capability: the ability to communicate with other devices nearby at relatively high speeds over a wireless protocol. Bluetooth is a very useful tool for development: I don’t have to worry about USB cables or anything else, and I can talk equally well to my Linux desktop with an abicom BT adapter as I can to my Powerbook with its built-in bluetooth.

One thing that no one has mentioned yet about the new Python release is the Bluetooth console. It took some doing, but I finally got it connected to my Linux desktop, and found an app that will let me connect to the port. Now, I basically have a way to tell my phone what to do over Bluetooth:

[crschmidt@peanut ~]$ sudo cu -l /dev/rfcomm0 
Connected.
Python 2.2.2 (#0, Dec  2 2004, 18:12:32) 
[GCC 2.9-psion-98r2 (Symbian build 540)] on symbian_s60
Type 'copyright', 'credits' or 'license' for more information.
Type 'commands' to see the commands available in this simple line editor.
>>> 

This has sped up my application development significantly: with this in place, I can start to experiment with different code at a rapid rate: without sending almost identical files to the phone several dozen times to test them, nor typing on the 3650’s keypad to enter my code. I just type into a normal tty, and it acts exactly like a local python interpreter.

This is also possible, and documented as such, using TCP/IP over GPRS instead of Bluetooth. However, the speed restrictions of that cause it to be much less practical. It’s like typing into ssh over bluetooth: sure, it’s okay, but you almost never want to do it if you can avoid it.

Here’s what I ended up doing to get it working:
– Get a working bluez install. Bluez is the standard Linux protocol stack, and is built in to most recent kernels.
– Test that you can talk to the phone, using hcitool scan, hcitool info.
– Register an “SP” (serial port) service with sdpd. sdptool add --channel=3 SP
– Ensure that sdpd is running
– Set up an rfcomm port to receive the communications: rfcomm listen /dev/rfcomm0 3
– (On Phone) Open Python, then bt_console.py
– On the computer, you should see:

[crschmidt@peanut ~]$ sudo rfcomm listen /dev/rfcomm0  3
Waiting for connection on channel 3
Connection from 00:60:57:41:86:C2 to /dev/rfcomm0
Press CTRL-C for hangup

– Using taylor-uucp, type cu -l /dev/rfcomm0
– Welcome to the phone!

If you want to test to make sure it’s working, you can do something simple like:

import appuifw
appuifw.note(u'Howdy!', 'info')

from there, refer to the excellent Nokia documentation for more tips and tricks on what you can do. I think this is definitely a great example of what power the distribution has, and I’m surprised that more people haven’t been writing about it. Has anyone besides me (and Nokia employees) gotten this working?

With the advent of a language I can program on my phone, my first goal was to learn the language, and my second goal was to rewrite something that I think is an almost-great service, but missing a couple key features. With that in mind, I headed to work, first with my FOAF app. My work after that concentrated on a locative service based around cell towers, and learning the user interface aspects of the Python interface.

Last night, I got my first victory in both: I wrote a decent (although minimal) user interface, which was able to communicate information about my location based on cell towers I was near. You can see the start of the work at http://crschmidt.net/cell/ . As I just described it to another mobile user, it’s going to be “Like CellSpotting, without the suck.” Using services for anything is completely unneccesary, especially with how simple python makes it to build user interfaces. I don’t know how hard it is with the Symbian interface, but I want to make it better.

My code is pretty simple so far, but it works. I will be continuing work on it tonight, but I’m already seeing other people start similar work, and I believe in shared efforts. So, the code is available: the .py.txt is the Nokia python program, and the .php is the server-side script I’m using.

I will be working on changing the way things work a lot, but I just want to show some of the cool things that can be done via the exposed Python interfaces on the mobile platform. Note that the actual Nokia code is only a dozen lines: which builds a nice interface, allows you to edit the form, and to send the data off. There’s not much funtionality yet: not even a way to update your location without exiting and reentering the program. However, it’s a start, upon which others can build.

An update to my previous post about bad documentation:

I’m blind.

Quite clearly described on page 18 in the API documentation is the popup_menu feature. I’m not sure how I missed it, as I thought I went over the entire UI section 2 or 3 times looking for it, but it’s right there, plain as the nose on my face. Thanks to Jukka Laurila for pointing this out to me.

Secondly, the additional documentation referenced in the references section of the API doc are included in the distribution: I had lost them in the sea of documents in the folder I unzipped into, since the zip file that I downloaded didn’t have the files inside it stored in a seperate folder. (Yet the Examples zip file inside that folder did). I still think that at least mentioning that they are included would be a good idea, for people who don’ot realize it.

So, my two issues with documentation are mostly resolved: both were due to me not seeing what was right in front of me.

However, the error with hitting return while on a word in t9 is definitely reproducible, so I don’t think that one is something I’m just making a mistake on.

Nokia has earned more than I’ve given them credit for. Nice job, and thanks again.

Although I love being able to develop things quickly and easily on my 3650, there’s a few major sticking points for me that I don’t understand. I know that my phone is considered an “untested” phone for this application: it’s almost the oldest Series 60 around, so I don’t expect everything to work nicely, or even at all. I was totally psyched when I learned that I was on the accepted list. Yet there are some things that bug me.

Lack of Documentation: Nokia has done an admirable job at documenting what their Python does and does not do, and has done pretty awesome, except for a couple things. First of all, there is completely undocumented UI funtionality in the Examples code that they give out. (Specifically, for the pop up choices: appuifw.popup_menu). I don’t know how much it bugs me that I looked through their entire PDF documentation 3-4 times looking for this feature described and found nothing. For a great project otherwise, to skip things like this when you’re releasing a public version is just a bit annoying. Additionally, 3 of the 6 references from the API reference don’t go anywhere: they’re just titles of publications, with no additional information. I believe two of these files are included with the distribution (I can’t check right now: The batteries of the computer I downloaded it to are dead), howeverm, to not even mention that and only list a title is again, quite silly.

Clear Lack of Testing: Again, as I said, the 3650 is not a “well supported” phone in this endeavour. However, there is a relatively major crasher bug in the Python distribution: when running in the Interactive Shell, if you press the “return” key equivilant while still on a word while using predictive text… it crashes.

That’s right, if you’re using a built in feature of the interface, it crashes. For no apparent reason. Doing something that most python programmers will do first, before anything. (The interactive shell in python is impressive, and is a good “sandbox” type environment to test small scripts out.) Granted, on a cell phone, the interactive shell users are going to be relatively slim: who wants to type on a phone when you can just type on a computer and send it over? However, to leave a standard python feature so broken is just silly.

Other than these two minor niggles, I’m very impressed, and hope to see the available UI interfaces grow over time: I’d like to be able to have as much control over user interface tools as Symbian native apps do, and I think Nokia would like to see that as well. I’m especially happy with the “location” module, which allows you to retrieve Network, Area and Cell IDs: I can now write my own tool to do MiniGPS-type cell storage, as well as uploading data to MeNow.

A list of standard modules which are included:
anydbm, atexit, base64, bdb, binascii, cmd, code, codecs, codeop, copy, copy_reg, cStringIO, dis, errno, exceptions, __future__, httplib, imp, keyword, linecache, marshal, math, md5, mimetools, operator, os, pdb, quopri, random, re, repr, rfc822, socket, sre, string, StringIO, struct, sys, thread, threading, time, traceback, types, urllib, urlparse (urlsplit only), uu, warnings, whichdb, xreadlines, as well as a “location” module for determining GSM cell info and “messaging” for sending SMS messages. Additionally, there is a UI Specific module, appuifw, the e32 Module, which offers some Symbian related utilities, e32db, a relational database module, and the e32dbm module, which offers an API to the Symbian RDBMS.

All in all, an impressive an useful subsection of the modules which most people use. The socket library has been extended to include a number of bluetooth functions, which apparently work quite well (I wouldn’t know, haven’t tried them yet.) I’m very pleased. It’s fun. But I think nokia could have waited just a little longer to ship this, to make sure that some things weren’t broken.

Today, Nokia released Python for Series 60 based phones. I’d been hearing about this stuff for months, but it had been in closed beta, so I hadn’t paid much attention to it. In addition, when this stuff was being discussed originally, back in April, I’d never had a hand in any Python coding: I was a Perl and PHP hacker only.

However, when mattb asked today who would be the first to write an RDF parser that would run on the phone, I asked what he was talking about, and received a pointer to Nokia’s announcement. I immediately started playing. (A bad thing, given that I had work today.)

So, in 5-10 minute increments, plus hacking after the kids went to bed, some help from sbp in the form of an ntriples parser written in Python, some help from Dave Beckett and Redland in the form of an easy RDF/XML->ntriples conversion, I now have a working (although clunky) FOAF application for the Series 60 phone running Python.

So, without further ado, I explain to you the steps behind the process:

First, the user runs Python. This is installed the way any other application is. Once Python is installed, the user must send two files to their phone: an ntriples parser and the actual FOAF Service script. The ntriples script is installed as a “library”: this means that it is installed on the include path, so that it can be imported as a module. The FOAF script is installed as a script.

The user opens Python, and chooses to run a script. They choose the script that they installed, and run it. A standard query box opens, and asks the user for a URL. Here, they enter the URL of a FOAF file. Once they do, the file is passed through an rdf/xml->ntriples web service: this converts most FOAF files to a format the phone can understand. An example URL for this webservice is http://crschmidt.net/semweb/xml2nt.cgi?uri=http://crschmidt.net/foaf.rdf : It is created using Redland, and source is available at http://crschmidt.net/semweb/xml2nt.cgis . It uses the Redland Python bindings. You can feel free to use this web service: however, if you’re going to be exceeding 10 hits/minute, please let me know so I can find someplace better to host it.

Much of the time for the script to run is simply fetching this content: my FOAF file generates 70k of ntriples, which is then parsed into a simple list of triples. Some post processing sorts the data, and looks for a personal profile document to whom the document can be attributed. If it is found, then the script prints out a list of contact information for that person, including email and contact IDs.

Many thanks to mattb for pointing out the Python release, sbp for his invaluable ntriples parser and his help getting it working, and to other people who muttered encouragement throughout the day.

The source is available at my Symbian Semantic Web Page. More images are available in my Symbian Hacking gallery.