Technical Ramblings

Over the past couple months, I’ve been importing a large chunk of data, largely from Zami, into the Open Guide to Boston. Tonight, the guide crossed the 1000 place mark, and I decided that showing everything on one map was no longer making much sense.

The result? Boston Mashup – pick the categories you want, and they’ll be dropped onto a map, with a different color for each type of marker (up to 7 – I don’t have that many colored markers yet).

I’ve still got more I want to do with it, but it works, and works pretty well: much better than I expected it to. Thanks to perigrin for helping me work around my Perl ineptitude and solve the problems I needed to.

Not really much to say about it other than that: try it out, let me know what you think. I’m happy with it, at least as a starting point, so I think that’s probably a good sign.

Attended my old Linux Users Group in NH last night: there was no scheduled speaker, so it was mostly just a “hang out and talk” type of meeting at Martha’s, in Nashua, NH.

Here’s a summary of how the meeting went from my point of view:

Had fun with all the MerriLUG folks last night. I also got to experience driving out of the Boston area during rush hour last night for the first time, which was significantly less fun.

Some things which were discussed:

* Results of the recent quarterly meeting, and location of the next one
* General questions on Linux:
** Why won’t my screen turn off when the computer goes into standby?
** How can I do load testing of MySQL and network traffic?
* Raffling off of books from Ken
* How Vendor/Client relationships are like teenage sex: They’re hormone driven and have no basis whatsoever in reality.

I also led some discussion on Ning. The reason I brought it up was in part because it is following the model of the open source world so much more closely than many other
services out there:

* Ning provides a “hosted” PHP framework
* All code, by default, is “open source” — can be viewed by anyone with an account.
* Users can “clone” applications: take the current application and customize it to their own liking from the same code
* Data is stored in a universal content store, and data is (by default) accessible to all applications across the server. So, my application “gnhlugbookshelf” can also read from the “restaurantreviewswithmaps application
* Income comes in via advertisements sold on the sidebar of the applications, as well as premium services (more space, removal of view source links, and the like)

Most of the time when this is described, it’s described as an “experiment” – can a company make money solely off ads to run their servers? Can premium services pay for all this? My experience with LiveJournal says yes: LiveJournal makes all its money off premium services (no ads), and they gross several million a year. However, their employees are paid much less than Ning’s are, so who knows.

I also mentioned the creation of GNHLUG Bookshelf, an application which “aims to store the suggestions and recommendations of the New Hampshire Linux Users Group on technical books that are the most useful of the bunch.”

Lots of interesting discussion on the business model behind Ning, where it could go from here, and how service is really where the money is these days. Give away the code: sell the service. RedHat learned from this model, and others are doing so too – Ning is simply a widescale demonstration of “give away the code”.

There are aspects of Ning which aren’t given away: the code that runs the playground itself is not open source, but the applications that run on it are. The “secret bits” are still probably important enough to Ning that open sourcing would give away a competitive advantage, but the PHP bits aren’t, since they’re easily reproducible.

Another thing that came up is how much control you have over code you write for Ning and taking it elsewhere. This is a question that has come up in developer discussion before, about how to take your application code elsewhere. What it comes down to is that Ning provides a lot of functionality: shared content store, tagging, user auth, etc., that doesn’t exist anywhere else. There’s no clone of the functionality which you can drop in and replace with something else. There’s nothing to stop people from mirroring the API and creating a way to drop Ning apps into your own webserver, it just hasn’t been done yet. So although you can take the code with you – you own it – it doesn’t do you much good without a lot of work to reproduce the functionality that Ning already provides for you.

In my spare time (hah!) I’ve been hacking on an app on Ning, called PeopleFinder. The goal is to be a geolocation app, supporting a variety of things social. One of my many goals for a long time has been to create and use some kind of tool which allows me to generate MeNow data with no effort.

I’ve finally got it.

With the help of Geocoder, I can now type in an address, and have it automatically update my location on the website, including showing my position on a GMaps interface. The PeopleFinder API is still a bit weak, but using it, I can write a Client for Series 60 Python, which allows you to update via a GPS lat and long, or via an address.

I’ve created a token system for authentication via the API — when logged in, you simply hit the token link, grab the token, and put it into the client (in the “token” variable).

Eventually, once I write the functionality, it is my hope that I will be able to provide more features via this API — the ability to look for people nearby you, get their information, send them messages, and so on. However, right now this is just a 2-3 hour hack, and I’ve got piles of work to do – but this is so cool I had to share it.

As far as I know, this is the first example of a Ning App exporting a semi-usable API to remote clients. In part, this may be due to previous limitations on Ning’s end which have recently been resolved: looking at the phone client code, you’ll see the ?xn_auth=no parameter, which allows you to skip cookie authentication. It’s a pretty nice solution to the problem in my opinion — it solves what I need while not interfering with the rest of Ning.

So grab your phone, grab a client, and update your address on the go — then, the next time I see you in the neighborhood, I’ll drop by and give you a wave.

Something I was previously unaware of:

The reason for the two different variations of “and” and “or” operators is that they operate at different precedences.

PHP’s && vs. and have different precedence? Who thought this was a smart idea? What end or purpose could it possibly serve?

Yet another one of the things that just makes me smack my head. It does explain a few things — why I couldn’t do variable setting as I might in Perl, a la:

$var = $test || $default;

In this case, the test occurs first, setting $var = 1; Had I used:

$var = $test or $default;

All would have been fine.

How can someone think this is sane?

Most of my work on Ning thus far has been in editing other apps, a large portion of it in testing and QA work. Since the launch, I’ve been taking a couple days “off” — still playing with ning, but at my own pace rather than at a “Almost time to release” pace.

Today, I built my first complete and clonable app from scratch.

Planet Ning is a planet style aggregation tool, supporting RSS 2.0 fields and the ability to rate any RSS item. This app took a total of about 1.5 hours to put together, a large part of that simply playing with design to get it to look semi decent. It uses the RSS alpha component for RSS parsing, XNC_HTML for the page bottom navigation, XNC_Rating for ratings, and the “bot” option to load new posts.

Pretty cool little app. One spot of trouble I ran into: XNC_Rating is designed to have only one form field per page. I overrode just one function to fix it though, testament to the flexibility of the code behind some of the XNC components. This is also a testament to Object Oriented code, and PHP 5 in general: in PHP4, I’m sure this would have been much harder, but PHP5’s exceptional class support, along with exception handling, make it so much more fun to work with than PHP4 was for me.

You can feel free to clone the app and play with it: it’s not ‘complete’ by any means, but it definitely works.

For the past 4 weeks or so, I’ve been working on a project known previoiusly as 24 Hour Laundry.

Now, it’s no longer 24HL: Welcome Ning.

A development playground with all kinds of neat and nifty toys, Ning is attempting to do to application and code sharing what other apps have done to photos, bookmarks or other arenas. Allowing people to clone, mix, and create new apps.

There’s a lot of cool things here, and I’ve got a pretty bad headache, so I’m not going to be able to cover all the things that I would like to here, but here’s some of the cooler things about the site:

* System wide content store. Public content which is created can be accessed by any application. This content store is well abstracted, and has a content creation and query system. You don’t have to worry about scaling up: You can leave that to the professionals in the backend. At the same time, you can collect data from all the other apps in the playground. You want to create a book reviews site? First, grab everything that’s known as a Book from the site, and then use the built in classes for ratings and comments to build a discussion board. The possibilities for content mix and match are really spectacular. However, if you don’t want others touching your data, you can mark it as “private” and use it only in your app – but why would you want to?
* Built in classes for lots of things. Build a calendar. Interact with Flickr. Make a GMap. Talk to Amazon. The code’s all done for you, you just use it. Bookshelf makes extensive use of the Amazon classes, Restaurant Reviews With Maps uses Google Maps to show where you’re going — Bay Area Hiking Trails shows you how to get there.
* RSS feeds of content. The Ning Pivot is a really cool way of looking at the content flowing by, but not only can you watch it, you can watch it flow by.

There’s about a half doezn other really nifty things here that I can’t even think of at the moment because it’s 5am and I’ve been walking like a Zombie for two weeks to get this stuff complete.

But the coolest thing is:
* All data added is placed under CC By-SA license. (If you don’t like this, ning isn’t for you.)
* All app code is completely open, and you can make it your own in 2 seconds.

Screw Ruby on Rails: who needs a 2 minute app, when you can write a 2 second app? All depends on how fast you can click.

If you run into problems with ning, feel free to drop them here: You can never fix all the bugs before release, but I think that the team working on Ning has done an absolutely incredible job with all the work they’ve put together here. I’ll pass them on as best as possible.

There’s a lot of other stuff I want to write — one that others here might find interest in is how similar Ning’s content store is to RDF, and why I think that there’s no functional difference. Of course, Marc and I got into a nice “discussion” on that one on IRC the other night, so maybe I’ll wait til I’m a bit less exhausted and can adequately express my points on the topic. 🙂

On Thursday night, I attended the first meeting of what looks to become a great group of people, at the first PySig meeting, in Manchester, NH. The meeting release has some more information on it, and the mailing list is a great place for discussion of it for people who may be interested.

It was interesting to speak to a mixture of people ranking from 20 years in the computer industry to people like me with one or two, and note that they were all interested in learning Python – and that I definitely had a thing or two up on most of them in this day and age. For example, in a room of 20 participants, several of whom have been working with Python for years, I was the only person who had actually had work posted to Daily Python URL.

During the meeting, I wrote some code (a clunky registration page) which we then did some initial code review on. I demonstrated a trick I learned from sbp, including a “View Source” part of my application so people could see exactly what it was doing.

I’m falling back in love with Python quite quickly. This is nice for a number of reasons, not the least of which is the start of a bit more free time as of this week. I’ll explain that in a later post.

Mostly, I’d like to put out the word to anyone that attended that I had a great time, and I’d love to see more people in the area attend. So far as I can find, there is no group of people in the Northeast that has as much coverage as the pysig.org website: there’s been a couple postings around Boston, but nothing big enough that they have a website. I’d be glad to find out I’m wrong, however, so feel free to tell me so. 🙂

Matt Croyden mentioned the other day that there is a new Python for Series 60 Alpha release. Reading through Erik Smartt’s post on the topic, I realized that this offers a number of the features that I had wanted built in in the original release: Camera access, Address Book and Contact APIs, and other similar things.

I had put off working on Symbian Python work for a while, but with the new release, I think I’m going to put some more effort into it: use of the new APIs will make things easier (like automatically uploading pictures taken to flickr, one of my original goals) and makes me want to get hacking again.

Congratulations to the Python-on-Symbian team, and I’m looking forward to starting work with the new alpha release.

After some thinking this morning, I converted the current PHP-based crschmidt.net templating system to a Cheetah Python template. This means that some more of my tools can move to being Python powered, rather than PHP powered.

“So what?”

Currently, the interface to Redland that I have available in PHP is significantly less good than Python. It’s coded by yours truly, and it’s basically only designed for my use cases, so every time I want to use something new, I have to go and code it, or use a closer-to-native C-style interface translated into PHP. Neither of those are particularly enjoyable.

Python is a much more comfortable language for me to use. It is more intuitive for me. It feels more natural, not to mention the fact that I keep forgetting semicolons in my PHP code. It has an awesome binding for Redland, which is one of the things that I’ve been working with most over the past while.

In the past, all my scripts had been either 1. PHP or 2. Python with no site theming. Hopefully the new Cheetah template will help make me create some more tools in Python, which is the language I feel most comfortable in.

With that in mind, I’ve created a new crschmidt.net web service: an RDF Validator. A number of times, I have found that the official RDF validator will puke, but won’t give much of a reason why. This tool uses Redland, which has a tendancy to return what I consider better error messages on worse RDF. It’s designed as a one-off example of the new templating system, and should not be considered indicitive of most of the expected output of such scripts. Just a first attempt at getting myself into more code.

I’ve been doing some playing with goofy Javascript stuff lately to try to get my head wrapped around it, since I’m going to be needing to implement it in a few tools at work in the near future.

I’ve so far used it in
1. An admin interface for Athena’s email accounts,
2. An inventory listing for a work project
3. The newest one, a “suggestion” field for Wordnet searches against the RDF store I just imported this morning.

Danny alerted me to the existence of a new Wordnet dataset. I grabbed the full set, dropped it into Redland, and set up a sparql search against it. The top box there is the nifty one though: type in a string (say, apple) and watch the right side as a list of suggestions is populated.

I still need to get it actually doing a Google Suggest-like dropdown box, but haven’t had the time to hack WICK to do what I want as far as that goes.

I’m still learning, and as such, the code is sucky. I wouldn’t recommend reading it for an example: it’s a quick hack, but it works. Still many bugs to work out – for example, if you type apple, it still searches for app, appl, apple in the process. But I’ll get there. (Okay, so I just did a few bug fixes that make it much better, and switched the search mechanism to use MySQL rather than an 11 Meg PHP array. Much better now.)

Anyway, I think it’s cool. RDF people can mark it down in the “another SPARQL datastore”, Javascript people can mark it down as “Another idiot trying to use XmlHttpRequest and doing it wrong.”

Lemme know if you’ve got suggestions!