Technical Ramblings

About a month ago, I participated in a couple of meetings at the Berkman Blog Group regarding Wikimania, the Wikimedia annual conference, which will be held this year in Boston. Talking with the local organizer, I was interesting in helping in a number of ways where I’m more useful than many other people — having a place to stay and possibly offer to other people, as well as technical skills which not everyone has that might be of assistance.

During the second meeting, I had the oppourtunity to engage with other Wikimedians who would be participating, and to get some of their feelings about the conference and the Berkman Blog Group’s attempt at trying to make things easier for them here in Boston during their convention.

I got a negative vibe from one of the people I spoke to, and later attempted to determine whether my understanding – that Wikimania was dedicated to Wikimedians, and local help in a significant way by people who were not active Wikimedia participants would be appreciated only so long as it didn’t affect the conference – was correct. Although I don’t have an exact quote of my question and the answer, the feeling was, again, negative.

The local organizer here asked me to clarify why I felt pushed away by the Wikimedia participants I had talked to, and this is what I wrote in response:

When I asked if it was the case that the purpose of Wikimania was to further the goals of Wikimedians, and that local people were not being invited to participate in the same way, I was informed that yes, this was the case: assistance from non-Wikimedians was unneccesary. (Sadly, I don’t have the log of this due to a client crash, because it’s the one thing that was said more than anything else that convinced me not to participate.)

Wikipedia has always had a very distinct aura of excluding outsiders. Although the wiki-nature of Wikipedia would seem to act as a counter to that point, many friends who have in the past been heavily involved in Wikipedia have left due to issues relating to negative personal interactions between themselves and other contributors. I have always held Wikipedia at arm’s length due to this, but given the local oppourtunity presenting itself, I thought that maybe I could be convinced otherwise.

What I stumbled into was exactly what I would have been led to expect – an elitist attitude taken by core contributors towards anyone who is not one of the “good ole boys”. Although you were extremely positive towards external help from the people on the ground, every other person I spoke to who in any way represented Wikimedia only left me with a bitter taste in my mouth.

I was hoping that Wikimedia was really a good group of people, done wrong by the masses who consider themselves to be “better than you” because they participate more. Instead, I found out that that attiude is perpetrated all the way to the top, at least from my external point of view.

As a result of the general vibe and the specific statements requesting that non-Wikimedians not take part in the planning and activities surrounding Wikimania, I’ve decided simply not to bother.

Another friend of mine put it best: “The only way to participate in Wikipedia is to just edit, and ignore all those people behind the ‘Discussion’ link.”

Something I was previously unaware of:

The reason for the two different variations of “and” and “or” operators is that they operate at different precedences.

PHP’s && vs. and have different precedence? Who thought this was a smart idea? What end or purpose could it possibly serve?

Yet another one of the things that just makes me smack my head. It does explain a few things — why I couldn’t do variable setting as I might in Perl, a la:

$var = $test || $default;

In this case, the test occurs first, setting $var = 1; Had I used:

$var = $test or $default;

All would have been fine.

How can someone think this is sane?

Today I did some work improving one of the Ning example apps, making it work faster, better, stronger. The app is Bookshelf, and it’s a ton of fun to watch. Basically, users add books (via the Amazon API), rate them, and comment on them.

Some aspects of the app were running way too slow, since it was designed and tested when the system was under no load at all, a situation which makes load testing very difficult. I won’t go into the technical details, but the long and short of it is that I took the book adding process and quartered the time it takes.

Apparently my changes were important, since Gina blogged about it. Although I’ll admit I had no idea who Marion Jones was until I Googled it, it sure made me feel special.

Gina also passed along the good news to me that helping out other projects in the free time I’ve got isn’t a problem from her end – so long as Ning projects come first. So, if you have interest in working to get an app running on Ning, but don’t have the spare time to learn all the ins and outs, feel free to drop me a line.

This is such a freedom after my last job, where doing any work outside the company was considered treason. It simply reassures me, yet again, that taking the job working for Ning is the best thing that has happened to me since I have left school — I’m working on fun projects, in a forward-looking environment, with great people, and a great management team above it all. While wedu is busy building the next version of We Don’t Get The Web, I’m helping to build the future, and I’m having the time of my life doing it.

Now to get some of my other apps up to snuff: a recent change in the core means that I can get access to pages without the use of cookies, which means it’s time to write that location updater client for my cell phone.

Updates as they come!

I’ve got a number of books on my “To read” list, most which were given to me as part of my birthday or as random gifts. One of them which was gifted to me by Tom Croucher from my Amazon wishlist (as a thank you for help in his dissertation, I believe). The book is Where Wizards Stay Up Late: The Origins of the Internet, and it’s one of my favorite non-fiction computer-related books.

The book is a relatively detailed study of how the internet came to be: the development of the theories behind it, the actual hardware, the proposals from the Defense Department for the creation. The origin of RFCs, the way TCP/IP was invented, things like AlohaNet and ethernet as well.

It’s because of this book that I still carry around Vinton Cerf’s business card, which I obtained at a 10th anniversary discussion of Mosaic. Ever since I acquired the card, I was fascinated with the design that was printed. I then planned on getting some for my business from www.dxprintingperth.com.au with exactly the same layout. Cerf was there discussing the idea of an interplanetary internet and how it would work. I was there drooling at the fact that I was in the same room as Vinton Cerf, and actually got up on stage afterwards and shook his hand. I keep that thing well – it used to be in my wallet, now it stays seperate – because it means that much to me, and it was inspired by this book.

So, a big thank you to Tom for the book, and a suggestion that you all go out and read it.

I’ve been doing some work with FOAFNaut, SVG, and other related technologies lately. For the most part, the changes in and of themselves are too small to track in a weblog format, but I did build myself a little tool to store recent updates to crschmidt.net last night, so I could share them. crschmidt.net site updates has an HTML view, as well as an RSS 1.0 and RSS 2.0 view, and is used to display information on the front page on what has changed recently.

Today, I spent a big chunk of my afternoon playing with julie alongside DanC. He asked if I planned on implementing SPARQL in the bot any time soon (which I do, as soon as a Redland release supporting the turtle format for SPARQL queries comes out). We also talked about GRDDL support, and some other related things. He offered some interesting files which I added to the database, teaching julie more about W3C proceedings and allowing for some more interesting queries in that respect. I need to start keeping track of my todolist for julie so that I can get organized in the freetime I have to do something about the state she’s in. I’m really starting to think another refactoring may be in order: although I received a pretty gigantic patch at one point, I still really haven’t “thrown one away” yet.

I also decided to install trac earlier today for some reason, something that was reinforced when I was asked to start a wiki foaf FOAFNaut internals as I was playing with it. You can check out the listing of projects I have here, which will grow as time continues, because I’m going to be moving more and more of my stuff into Subversion and more and more of what’s in Subversion to trac. It’s really nifty software, and I’m looking forward to playing with it. Who knows, it might shove a few more people into getting involved in my current projects. It’s got everything I need but have been too lazy to install in one place: wiki, bug tracking, source viewing, revision… quite nice, really.

Other than that, not much going on: Keep an eye on the site updates as I continue to do more little changes in and around crschmidt.net to my various projects.

So, a long time ago, I posted about how I planned to update the FOAF that LiveJournal spits out to become more friendly, since I really didn’t know anything about RDF at all when I was first creating the format. So, I wanted to do some updates, to match things that I had discussed with members of the FOAF community over time.

However, I got a real job, lost my free time, blah blah blah, these things happen. The patch has sat around in various incarnations for at least 6 months, probably longer. Recently, someone asked me what had happened to it, and I responded that I had gotten tired of pushing for changes to FOAF, and given up. He said he had time to spare on pushing people, so I cleaned up my patch, and posted about it – a detailed rundown of the changes the patch makes, for evaluation by the community.

Since the community involved is larger than LiveJournal, I’m here to encourage absolutely anyone who might have an interest or knowledge in the arena to read through the explanation or the patch, and let me know what they think. I really would love to see this happen, but like I said, I’m not going to fight for it anymore: it took a lot of hard work to get the previous FOAF patch accepted, and I just don’t have the energy or free time to do that again. So, this one is going to depend on community feedback and demonstration of interest. If you think that LiveJournal’s FOAF data might be suboptimal, say something about it. That entry is a great place to comment and discuss. Open a dialouge with developers about what you’d like to see, because if enough people want to see it, someone may actually push it in.

So, recently I’ve been watching a little bit of a FeedBurner problem arise between some friends on IRC, over at Mobtiopia. Seems that MobileTech uses Feedburner for his feed, via http://feeds.feedburner.com/Mobiletech.

However, Erik was having some problems, where every time the feed loaded, all items would look new. He dug into the problem a bit, and found what the root of the problem most likely is: Every 30 minutes or so, the “Last-Modified” header for Feedburner updates. It caches it per IP for about 30 minutes to an hour, it seems, but once you get past that, it just snaps back to the current time.

Now, I’m not one to criticize over small mistakes like this, but I feel bad for Feedburner: Their conditional get mechanism is completely broken by this difference. Rather than returning a 304 not modified, Feedburner is returning the full feed every hour someone asks for it. That’s got to be quite a hit on their bandwidth.

In this case, there were a couple other issues relating to the problem: Tarek’s feed was previously powered by RDF, which FeedBurner seemed to chew up and spit out in a quite ugly way in his case. However, I haven’t seen it happen the same way in other cases. In Tarek’s case, the feed was actually using what looked like uniquely generated local IDs for an rdf:about – which, although fine in RDF-parlance, is not allowed according to the RSS 1.0 specification, and doesn’t pass the feed validator. Most likely, Erik’s newsreader, newzcrawler, saw these funky looking IDs and didn’t treat them as permalinks, contributing to the problem.

Regardless of other issues though, I’ve checked a few other feedburner feeds, and every single one of them has a Last-Modified header in the past hour. This is simply not a good plan for your bandwidth, or for RSS in general: you’re dealing with a lot more traffic than you need to. 304 Not Modified is your friend, either via Etags or If-Modified-Since. RSS readers are doing good at cleaning up their act and using these headers – if the servers don’t support them, that’s just going to discourage such use in the future, and contribute to the load problem that RSS has become for so many people.

I’m going to let Feedburner know about this in more detail, and this is really not a slight against them. Headers for HTTP/RSS are hard to get right, not something that just “works” out of the box typically. So, I understand the difficulties attached to them. Getting them wrong, however, has some major consequences on all parties, so I hope they can figure out what’s up and get it fixed, both for their sake and for the sake of people that use them.

One of the biggest things that “irks” me when I’m commenting on some other weblog that I seldom read is that I will most likely never see any response to what I write, even if one is directed at me. For this reason, I often write my own posts, expecting that with Trackback, the original author and subsequent commenters will be able to see my thoughts as well as reply to them.

However, occasionally I don’t want to write a full post, yet I still think that the discussion is worth having. For those times, I really wish that more weblog software packages would support notification of comments, preferably via email. LiveJournal has implemented this since long before I had an account, and the discussion I’ve seen, even on the most minor topics, are gigantic in comparison with the discussions you can find between commenters on most weblogs.

I understand that the cultures are different, and I understand that the goals of each are different, but this is one practice that I definitely miss from my old tools. For that reason, I have installed a wordpress plugin which allows you to ask for subsequent comments to entries to be mailed to you.

I hope that this will allow people, if they desire, to continue threads of conversation longer, and breed more communication between commenters to the site. I certainly know that to me, seeing more talking back and forth among people in comments has always been a great way for issues to be raised that might not seem “Deserving” of a fully thought out response in the form of a new post.

My major concern at this point is with spam: will people who comment find themselves innundated with spam? I hope that I can work with wordpress to keep spam to a minimum, thereby protecting those who receive those emails as well.

The next thing that I’m interested in is threaded comments. However, I’m pretty sure there’s not a “simple” plugin for those, and I do know that threading comments introduces a variety of issues, from display issues to conversion of a tree format to a SQL database. We’ll see. At the moment, I’d like to hear your thoughts on whether allowing people to receive notification of new comments is a good thing or bad thing. Also, since it is new, feel free to let me know how it works in your opinion. You don’t get copies of your own comments at the moment: something I still haven’t decided whether I’m going to change or not.

A number of people have expressed interest in my posting some of my more technical ramblings as a seperate resource to the oftentimes personal weblog I’ve used in the past, my LiveJournal. Because I’ve always been one to aim to please, I have decided that I will seperate my more personal postings from my technical ones, although not because I mind people reading my personal life, but more because I want to ensure that people can read what they want and not feel discouraged, as well as ensuring that I can post in specific places and have people read it only if they want to.

Of course, this entire problem could be alleviated if LiveJournal ever got a decent categories system in place, under which people could subscribe or filter to certain categories, however, I do not think that this is a very likely occurance.

LiveJournal has long been my “home” on the web. It offers a number of features that I don’t understand the lack of in other tools: for example, the emails for comments act much like any other mailing list which takes place through email, something that I’ve never seen another blogging system do. However, as the author of this weblog, this is less likely to be an issue: there will probably be few “threads” back and forth, and even if there are, I will receive information on all the comments being left.

Perhaps one thing to do with this new blog is to make communication be a bit more like a mailing list; email subscriptions to posts, so that when they’re updated or a comment is left, people can tell.

I’ve chosen a relatively odd permalink structure for this weblog. In part, this is to prove that it can be done: Many people do not realize that WordPress offers the ability to completely change what your permalink structure is like. In part, it’s simply to shorten post URIs: with only %postid% and %posttitle%, I’m keeping it simple, but still unique from both the %postid% perspective, and informative to the human who may be dereferencing the URL manually.

Expect me to add categories, links, plugins, and possibly change the style over the next few days. If anything I change is something that you feel makes this weblog less usable, please let me know.

The weblog that I was using previously will continue to be available, although I will no longer update it. From now on, posts go to LiveJournal or to here, depending on their nature.