Technical Ramblings

Attended my old Linux Users Group in NH last night: there was no scheduled speaker, so it was mostly just a “hang out and talk” type of meeting at Martha’s, in Nashua, NH.

Here’s a summary of how the meeting went from my point of view:

Had fun with all the MerriLUG folks last night. I also got to experience driving out of the Boston area during rush hour last night for the first time, which was significantly less fun.

Some things which were discussed:

* Results of the recent quarterly meeting, and location of the next one
* General questions on Linux:
** Why won’t my screen turn off when the computer goes into standby?
** How can I do load testing of MySQL and network traffic?
* Raffling off of books from Ken
* How Vendor/Client relationships are like teenage sex: They’re hormone driven and have no basis whatsoever in reality.

I also led some discussion on Ning. The reason I brought it up was in part because it is following the model of the open source world so much more closely than many other
services out there:

* Ning provides a “hosted” PHP framework
* All code, by default, is “open source” — can be viewed by anyone with an account.
* Users can “clone” applications: take the current application and customize it to their own liking from the same code
* Data is stored in a universal content store, and data is (by default) accessible to all applications across the server. So, my application “gnhlugbookshelf” can also read from the “restaurantreviewswithmaps application
* Income comes in via advertisements sold on the sidebar of the applications, as well as premium services (more space, removal of view source links, and the like)

Most of the time when this is described, it’s described as an “experiment” – can a company make money solely off ads to run their servers? Can premium services pay for all this? My experience with LiveJournal says yes: LiveJournal makes all its money off premium services (no ads), and they gross several million a year. However, their employees are paid much less than Ning’s are, so who knows.

I also mentioned the creation of GNHLUG Bookshelf, an application which “aims to store the suggestions and recommendations of the New Hampshire Linux Users Group on technical books that are the most useful of the bunch.”

Lots of interesting discussion on the business model behind Ning, where it could go from here, and how service is really where the money is these days. Give away the code: sell the service. RedHat learned from this model, and others are doing so too – Ning is simply a widescale demonstration of “give away the code”.

There are aspects of Ning which aren’t given away: the code that runs the playground itself is not open source, but the applications that run on it are. The “secret bits” are still probably important enough to Ning that open sourcing would give away a competitive advantage, but the PHP bits aren’t, since they’re easily reproducible.

Another thing that came up is how much control you have over code you write for Ning and taking it elsewhere. This is a question that has come up in developer discussion before, about how to take your application code elsewhere. What it comes down to is that Ning provides a lot of functionality: shared content store, tagging, user auth, etc., that doesn’t exist anywhere else. There’s no clone of the functionality which you can drop in and replace with something else. There’s nothing to stop people from mirroring the API and creating a way to drop Ning apps into your own webserver, it just hasn’t been done yet. So although you can take the code with you – you own it – it doesn’t do you much good without a lot of work to reproduce the functionality that Ning already provides for you.

Ever heard of Rent-a-Coder? Post a project and a bounty, and grab the next person who comes along. Like so many great apps, it’s now got a Ning equivilant: Find-a-Developer.

I’ve been playing with this app, and I can honestly say that it is the best architected web application I’ve seen in a long time. Links where I need them, when I need them, without me needing to think about them. I don’t need to look through navigation or search through mystery-meat menus: everything just flows so nicely.

The only problem I have with the app so far is that there are so few Ning developers. I know there are more people than just me out there who are interested in working on Ning, and feel they’d be up to it. The challenge isn’t that hard: get some code together, knock out an App (or link to some older projects) and get going!

Also, there’s the use case for someone who’s not a developer: the kind of people who want to get something done on Ning, but have more cash than time to throw at it. As you can see from Gina’s example, it’s easy to set out what project you want people to work on. Think you can tackle her request? Just drop her a message via the interface. But before you do, you might want to throw a profile together, just in case.

As with all apps on Ning, Find-a-Dev is clonable. Don’t want to work on Ning, but have a desire to set up a Python marketplace? Clone it, change some words, and you’re off. Want to create your own marketplace for knitting services and projects instead? Go for it.

Want to set up a marketplace for bodyguard services — but don’t think you have the PHP code for it? Well, a great place to start would be to post your idea to find-a-dev, and get some people interested in it. After all, there’s probably lots of people out there looking to help write some code for you — you just need to meet and interact. What more perfect playground to find them in than Ning?

(Meta-info: I had a bug in my RSS feed, so the past two entries were delayed. You’ll probably see three updates at once: sorry about that.)

In my spare time (hah!) I’ve been hacking on an app on Ning, called PeopleFinder. The goal is to be a geolocation app, supporting a variety of things social. One of my many goals for a long time has been to create and use some kind of tool which allows me to generate MeNow data with no effort.

I’ve finally got it.

With the help of Geocoder, I can now type in an address, and have it automatically update my location on the website, including showing my position on a GMaps interface. The PeopleFinder API is still a bit weak, but using it, I can write a Client for Series 60 Python, which allows you to update via a GPS lat and long, or via an address.

I’ve created a token system for authentication via the API — when logged in, you simply hit the token link, grab the token, and put it into the client (in the “token” variable).

Eventually, once I write the functionality, it is my hope that I will be able to provide more features via this API — the ability to look for people nearby you, get their information, send them messages, and so on. However, right now this is just a 2-3 hour hack, and I’ve got piles of work to do – but this is so cool I had to share it.

As far as I know, this is the first example of a Ning App exporting a semi-usable API to remote clients. In part, this may be due to previous limitations on Ning’s end which have recently been resolved: looking at the phone client code, you’ll see the ?xn_auth=no parameter, which allows you to skip cookie authentication. It’s a pretty nice solution to the problem in my opinion — it solves what I need while not interfering with the rest of Ning.

So grab your phone, grab a client, and update your address on the go — then, the next time I see you in the neighborhood, I’ll drop by and give you a wave.

Today I did some work improving one of the Ning example apps, making it work faster, better, stronger. The app is Bookshelf, and it’s a ton of fun to watch. Basically, users add books (via the Amazon API), rate them, and comment on them.

Some aspects of the app were running way too slow, since it was designed and tested when the system was under no load at all, a situation which makes load testing very difficult. I won’t go into the technical details, but the long and short of it is that I took the book adding process and quartered the time it takes.

Apparently my changes were important, since Gina blogged about it. Although I’ll admit I had no idea who Marion Jones was until I Googled it, it sure made me feel special.

Gina also passed along the good news to me that helping out other projects in the free time I’ve got isn’t a problem from her end – so long as Ning projects come first. So, if you have interest in working to get an app running on Ning, but don’t have the spare time to learn all the ins and outs, feel free to drop me a line.

This is such a freedom after my last job, where doing any work outside the company was considered treason. It simply reassures me, yet again, that taking the job working for Ning is the best thing that has happened to me since I have left school — I’m working on fun projects, in a forward-looking environment, with great people, and a great management team above it all. While wedu is busy building the next version of We Don’t Get The Web, I’m helping to build the future, and I’m having the time of my life doing it.

Now to get some of my other apps up to snuff: a recent change in the core means that I can get access to pages without the use of cookies, which means it’s time to write that location updater client for my cell phone.

Updates as they come!

Most of my work on Ning thus far has been in editing other apps, a large portion of it in testing and QA work. Since the launch, I’ve been taking a couple days “off” — still playing with ning, but at my own pace rather than at a “Almost time to release” pace.

Today, I built my first complete and clonable app from scratch.

Planet Ning is a planet style aggregation tool, supporting RSS 2.0 fields and the ability to rate any RSS item. This app took a total of about 1.5 hours to put together, a large part of that simply playing with design to get it to look semi decent. It uses the RSS alpha component for RSS parsing, XNC_HTML for the page bottom navigation, XNC_Rating for ratings, and the “bot” option to load new posts.

Pretty cool little app. One spot of trouble I ran into: XNC_Rating is designed to have only one form field per page. I overrode just one function to fix it though, testament to the flexibility of the code behind some of the XNC components. This is also a testament to Object Oriented code, and PHP 5 in general: in PHP4, I’m sure this would have been much harder, but PHP5’s exceptional class support, along with exception handling, make it so much more fun to work with than PHP4 was for me.

You can feel free to clone the app and play with it: it’s not ‘complete’ by any means, but it definitely works.

I noticed someone had said Marc Canter wrote about Ning. I think that he may have missed an important part of what Ning is about, though.

Why?

Well, let’s start from the top:

I’m pleading with Gina Bianchini to have Ning PLEASE support the PeopleAggregator APIs once it’s out – and I don’t see any reason why she won’t.

No longer does the “Someone else needs to do it” mentality need apply: Applications on Ning are open source. Code can be mixed, cloned, and run any way you want to — including a way to load files and modules from other applications! So, if you have an API you want to support, support it: just develop it and let people know to use XN_Application::includeFile(), document it for ning users, and you can develop your API for whatever you want and have other users use it.

So, once the API for the website you’re talking about is complete — write some code, and put it on Ning, then get people to use it. That’s what Ning is about: Sharing, putting things together, and bringing “View Source” back to the people. This is your chance to make good on something web browsers learned that Macromedia never has: the ability to look at the way something works inside is a huge boon to development, as I think we’ll see as time goes on.

Of course, there’s also the question of whether Canter still believes that Andreesen: sure as hell hasn’t done shit since – what 1995? 😉

I mentioned in an earlier post about Ning today that I felt that the content model used by functionally no different from RDF. This needs a bit of explanation, I’m sure, and thus far there isn’t a lot of talk out there about how to actually use the content store. (This is probably related to the significant lack of beta developer accounts so far, something that will hopefully change in the near future.)

A quick tutorial on creating Content Objects on Ning first (drawn from XN_Content documentation):

$object = XN_Content::create(“TypeOfObject”, “Title of Object”, “Description of Object”);
$object->save();

This creates an object with a Type, Title, Description. These are the three most commonly used “System” attributes — they are present on almost every object created in some form or another, so they can be used in displaying that data.

In addition to these system attributes, there is the ability to add developer attributes:

$object->my->name = “Christopher Schmidt”;
$object->my->age = 21;
$object->save();

If we look at the content for this object, via the $object->debugHTML() method, we see:

XN_Content:
  id [198390]
  createdDate [2005-10-05T06:50:23-07:00]
  updatedDate [2005-10-05T06:50:23-07:00]
  type [TypeOfObject]
  title [Title of Object]
  description [Description Of Object]
  tagCount [0]
  contributorName [crschmidt]
  ownerName [Example App]
  ownerUrl [exapp]
  isPrivate [false]
  my attributes [
    [-1] name : Christopher Schmidt : string
    [-1] age : 21 : number
  ]

Here we see some interesting bits that we hadn’t before: the id, Date, and owner fields are all useful for where the data is coming from, and the contributorName for who the data is coming from. For the time being, however, we’ll concentrate on the my attributes and the id and type system attributes.

Mapping this to an RDF representation is relatively simple:

ning:198390 a TypeOfObject;
dc:title "Title of Object";
dc:description "Description of Object";
ning:name "Christopher Schmidt";
ning:age "21" .

You’ll see here the main issue with representing Ning content objects as RDF: the fact that types and predicates (developer attributes) are stored only as strings, not the URIs that RDF typically asks for. This results in a less descriptive format than you would typically find in most RDF descriptions: there’s no URIs for predicates, so you can’t do some of the “magic” that RDF is famous for.

However, I have been working with RDF for more than 1.5 years now, and I have never had any use for that magic.

The ability to uniquely identify predicates may be useful in a general sense, and it provides the ability to accurately and adequately describe these predicates for use in other systems… but at the base level, they are designed to be able to attribute semantics to the terms. In many cases, these semantics are simply unneccesary: if I have a Book with an “isbn” property, I probably know what it’s going to be.

If you do need this level of semantics, all hope is not lost! You can still achieve your goals! Typically, apps use an attribute in only one way, and with each content object is the application which owns it. (In this case, exapp — which doesn’t exist, for the record. The minimum length is 6 characters.) So, you can take the URL for the app (http://exapp.ning.com/) and create a URL based on that: whether it’s in the app’s namespace (squatting on a URL that it isn’t using, for example) or in your own. I could coin http://crschmidt.net/ns/ning-exapp# for a prefix to predicate URLs in this situation.

The semantics attached to a predicate are extremely weak in the Ning content store — but that doesn’t mean that they are useless. It is my opinion that a simple RDF representation of Ning content objects can be useful in many cases, and that the model that is in use succeeds in proving that the RDF model as an application data store is not useless, but instead can lead to rich applications sharing data, as Ning is specifically designed to accomplish.

One thing I didn’t mention above is that it is possible to create links, not only to literals and numbers, but also to other content objects:

$object->my->otherContent = XN_Content::create(“OtherObject”);
$object->save();

This creates a link to another content object, by that object’s ID. This fits perfectly into the RDF model, where URLs identify an object: the Content Object’s ID is its unique identifier within the Ning universe, and the backing store lets you link any number of these objects together. You want to grab a book that someone added, and tie it to your own? That’s fine! Simply create a Content Object and add both of those as attributes. You want a whole list of books from someone else’s bookshelf? That’s fine too: just grab the IDs, and attach em. This casual interlinking of data is exactly what is going to make Ning a success — and it’s exactly the kind of interrelationship that proves the RDF model is not wrong.

I know that some people will strongly disagree: they will say that the Ning content model is more demonstration that people “don’t get it”. What I see it as is exactly the opposite: it’s proof that regardless of the underlying way content is stored, it can be presented to applications in a way that is easy to use, and useful. By giving objects a Unique Identifier (whether it’s a URI or an ID), a way to connect them together, and a decent API to put it all in place in code, you can create magic.

For the past 4 weeks or so, I’ve been working on a project known previoiusly as 24 Hour Laundry.

Now, it’s no longer 24HL: Welcome Ning.

A development playground with all kinds of neat and nifty toys, Ning is attempting to do to application and code sharing what other apps have done to photos, bookmarks or other arenas. Allowing people to clone, mix, and create new apps.

There’s a lot of cool things here, and I’ve got a pretty bad headache, so I’m not going to be able to cover all the things that I would like to here, but here’s some of the cooler things about the site:

* System wide content store. Public content which is created can be accessed by any application. This content store is well abstracted, and has a content creation and query system. You don’t have to worry about scaling up: You can leave that to the professionals in the backend. At the same time, you can collect data from all the other apps in the playground. You want to create a book reviews site? First, grab everything that’s known as a Book from the site, and then use the built in classes for ratings and comments to build a discussion board. The possibilities for content mix and match are really spectacular. However, if you don’t want others touching your data, you can mark it as “private” and use it only in your app – but why would you want to?
* Built in classes for lots of things. Build a calendar. Interact with Flickr. Make a GMap. Talk to Amazon. The code’s all done for you, you just use it. Bookshelf makes extensive use of the Amazon classes, Restaurant Reviews With Maps uses Google Maps to show where you’re going — Bay Area Hiking Trails shows you how to get there.
* RSS feeds of content. The Ning Pivot is a really cool way of looking at the content flowing by, but not only can you watch it, you can watch it flow by.

There’s about a half doezn other really nifty things here that I can’t even think of at the moment because it’s 5am and I’ve been walking like a Zombie for two weeks to get this stuff complete.

But the coolest thing is:
* All data added is placed under CC By-SA license. (If you don’t like this, ning isn’t for you.)
* All app code is completely open, and you can make it your own in 2 seconds.

Screw Ruby on Rails: who needs a 2 minute app, when you can write a 2 second app? All depends on how fast you can click.

If you run into problems with ning, feel free to drop them here: You can never fix all the bugs before release, but I think that the team working on Ning has done an absolutely incredible job with all the work they’ve put together here. I’ll pass them on as best as possible.

There’s a lot of other stuff I want to write — one that others here might find interest in is how similar Ning’s content store is to RDF, and why I think that there’s no functional difference. Of course, Marc and I got into a nice “discussion” on that one on IRC the other night, so maybe I’ll wait til I’m a bit less exhausted and can adequately express my points on the topic. 🙂

Some of you may have noticed if you’re regular subscribers that I’m currently working on the Feedback In-Feed that I added a few days back: specifically, trying to make it less obtrusive. However, one of the things that I didn’t realize in all my efforts is that I almost always already *have* the user’s homepage information: if they’ve left a comment on the site, it’s stored in a cookie in their browser (which is probably what they’re going to be submitting the form through).

“But”, I hear the audience crying, “You can’t see their cookies! They’re posting the form from their aggregator!”

Ah, my feeble minded friends, this is true, but this is unimportant. What matters is the user agent which hits the final form – which is stored (now) in the same directory as this blog, meaning it has access to all the cookies set by this blog. Including, as it happens, the name and URL of the person, assuming they’ve left a comment here before.

Of course, this has some limitations: persons who have not commented here before will not have a homepage set, or whose cookies have expired. (They expire after one year in WordPress, it seems.) Still, at the cost of saving space, I have gone the route of removing the homepage from the form, as well as pulling out some of the HTML I didn’t really like in the first place, and cleaning it up in general. I’ve been working on this over the past few days, in large part with help from jc, who was helping me figure out ways to make controls smaller. He said that my feedback form was “Somehow even more annoying than Adsense in feeds” – something that I was loath to admit, but in the end had to agree with.

Since I know there are a large number of people who read this who never comment, there is also a built in “more info” link, via which users can set their Name/URI. So, if you have any interest, feel free to use that link to set your name/URI, which will then be stored with the data.

Another thing that I’m doing, which I hadn’t yet written or talked about in public yet (despite Danny’s conviction otherwise) is the fact that I am capturing the referer information for the feedback, and exposing it via the RDF interface, attached to the review. This is a more generally useful property: a review for something can come via Amazon, a blog, or any other of a number of things. I’m simply using it to store the referer information: So I know whether someone came in via Planet Swhack, Planet RDF, Planet Mobile, LiveJournal, Bloglines, or any other of a number of web resources. Sometimes, of course, there is no referer information, in which case it was probably an aggregator, but I haven’t gotten far enough to analyze that yet. Unfortunately, User-Agent in most of these cases probably doesn’t make a bit of difference: The form is going to be posted via the browser, not the aggregator.

Danny advocates an extremely minimalist feedback mechanism, but I think that that’s less likely to get people to submit feedback, especially once they realize he asks for more information afterwards. LiveJournal’s polls always get more feedback than comments, because they’re low impact in comparison. The same idea applies here: but something which is just a tickybox or a simple form press is not enough to provide the user with a sense of offering helpful feedback. I hope that I’m achieving (for the most part) low impact as well, by redirecting to the referer once the form is posted – but I want to have low impact, informative, useful feedback. I think the recent design changes will help that.

I do think that this is the right idea, and that implementation is the problem, rather than anything more basic in the idea. So, I’ll be working with the implementation to make it better as time goes on.

Matt Croyden mentioned the other day that there is a new Python for Series 60 Alpha release. Reading through Erik Smartt’s post on the topic, I realized that this offers a number of the features that I had wanted built in in the original release: Camera access, Address Book and Contact APIs, and other similar things.

I had put off working on Symbian Python work for a while, but with the new release, I think I’m going to put some more effort into it: use of the new APIs will make things easier (like automatically uploading pictures taken to flickr, one of my original goals) and makes me want to get hacking again.

Congratulations to the Python-on-Symbian team, and I’m looking forward to starting work with the new alpha release.