Technical Ramblings

This weekend, I built a moderately functional web app. I’ve done this before, of course — I used to work in web development back in The Before Times — but this is only the second app I’ve built in 5-6 years, and it included some new things I haven’t done before! So, in no particular order, here are some things I learned. (Many of these things may have been true for a decade or more; that doesn’t mean that I didn’t just learn them!)

• If you intend to do anything with media queries, it seems that the common knowledge is that you *must* include a specific viewport meta tag: <meta name="viewport" content="width=device-width, initial-scale=1.0"/>. Without this, things work fine on desktop browsers, but fail on mobile browsers. This does not make sense to me, but there it is. (Thankfully, they at least also fail in Chrome’s mobile device inspector mode, so it’s actually possible to debug.)
• mod_wsgi has a python-home directive that allows you to specify the virtualenv that you would like to use to run your particular application. This is useful if you are using virtualenv to package up your dependencies; it means that you don’t have to mess about with weird path includes in order to get your virtualenv working in your Apache webserver.
• localStorage seems to work by default across major browsers at this point. This means that I was able to store information locally without any magical … anything. For minimizing server-side storage entirely, this seems cool and good to know. localStorage is available in the default Javascript scope, and has two functions: getItem('key') and setItem('key', 'value').
• If you have a div with contenteditable set, by default it appears to have a height in Safari and Chrome, but not in Firefox; so make sure you set a min-height on the element in Firefox.
• CSS flexbox support has been available in every major browser for years. This means that you can actually horizontally and vertically position elements without using tables. This is amazing to me. (In fact, out of desparation, I briefly attempted to use a <table>, only to find it screwed up my layout more — and that CSS flexbox was *easier* than using a table for what I wanted.) This is pretty amazing to me.
• If you are using a contenteditable field, then the following will ensure that when you paste into the field, it is pasted using plain text. (This is necessary in Safari; not in Chrome, not sure about other browsers.) I haven’t thoroughly tested how cross-platform this snippet is.
  

  document.getElementById('input').addEventListener("paste", function(e) {
// cancel paste
e.preventDefault();

  // get text representation of clipboard
  var text = (event.originalEvent || event).clipboardData.getData('text/plain');

  // insert text manually
document.execCommand("insertHTML", false, text);
});

• pusher.com is a quick way to get support for a moderate size of websocket connections (100 active connections; 200,000 daily messages) supported by a wide variety of push + receive client libraries, with almost no work. It honestly feels a little bit like magic; as someone who had been putting things off specifically because of the pain that is *either* polling or running a websocket service, this was really quite nice to see. Also supports posting out received messages via webhooks for clients that might want that instead of websockets. I expect that if I ever reach that level of scale, I’ll probably just set up my own websocket server rather than pay the $50/month for the next tier, but for a getting started, this was really useful. I started with this guide on building a chat app in Javascript and found that it was easy to get started with and to modify, since it only has around 30-50 lines of code.
• I probably need to bite the bullet and start moving towards Python3, even though I don’t really care about it. This is annoying as my gap of 3-5 years since I built new apps has left my server environments atrophied and weak.
• People really prefer a “dark” theme to a light theme in the social spaces that I run in. (I’m surprised that Facebook doesn’t have a dark mode yet.)
• Modern browsers continue to change constraints — on things like autoplaying videos, third-party cookie restrictions — in ways that are likely good for privacy and user experience, but break older apps without much way to fix it and with user control over those changes minimized. Old apps that worked fine are now non-functional because of these restrictions.
• If you try to copy a virtualenv from a different machine, you get *weird* behaviors. Also, if you’re creating a virtualenv, the first thing you want to do is probably upgrade pip. (Turns out that when the pip you’re using doesn’t support modern dependency descriptors, you also get really weird failure modes!)
• I really miss the handiness of Google’s auto-formatting facilities when writing code. Everything I’ve written this weekend is indented poorly, and it’s frustrating.

Anyway, starting around midnight on Sunday until early Monday morning, I built myself an app; it includes synchronized viewing of YouTube playlists, plus a completely built-from-scratch chat app that is moderately functional and improves in a lot of ways over YouTube chat for the use case I have. I built media queries that let the UI actually work on a mobile device, and it actually looks halfway decent. I didn’t have to use much in the way of browser-specific hacks (modulo comment on min-height in Firefox).

That’s a pretty cool experience.

So, I have something I’d like to write a series of posts about. They would be longish-form text with a few images tossed in.

In the old days, I would have just set up a LiveJournal (now: Dreamwidth) account, and posted to that. I might have sent links around to my existing friends, though in this case, my imagined target audience is different than my existing friend group, so I might not have cross-posted links.

But the thing is: Part of the reason I want to write is because I want people to find the content and respond. And I’m no longer sure how to build up those connections — and so I imagine myself leaning on the crutch of recommendations algorithms. “Oh, I should make a Facebook page!”, I think — because if it’s on Facebook, maybe people who I don’t tell about it directly will know.

But of course, that’s mostly bullshit: Absent bringing a starting audience with you, most social media platforms don’t provide you a magic discovery mechanism where you’ll be found by others; there really is no difference between any of these platforms on that front.

I run through this all the time: Do I go with tumblr? With Facebook? With Medium? With Dreamwidth? Where am I going to get the most shares? Is this content better for one platform than another? Etc. etc.

I’m not sure what changed. Is this something that changed in me: Do I feel more intimidated and scared to reach out to new people? Am I unwilling to do the work and engage meaningfully with relevant communities?

Is it something that changed in the world? Is it harder to get people to click through to a link that isn’t on Facebook these days? To read the article, rather than a 27 tweet long thread?

I don’t know how I got to where I am, but I do know this: There are a number of things over the years where I have tried to write them and never felt like I found a good home for them. I wanted some feedback, some sense of connection from them… and never came to the conclusion on where I might get it from.

It feels like a step back from where I was a decade ago, and I wish I knew how to move on from it.

If you ever see a video, and you’re like “Why is this watch next recommendation from YouTube so crappy?!”, then you should fill out this form for bad watch next suggestions.

If you ever see a video, and you’re like “Holy crap, it’s like I’m watching a dithered display from 2002 in here; these blocky artifacts are terrible!”, then you should fill out this form for bad video quality.

Feel free to pass these on.

It seems like my 15 minutes of fame are mostly coming to a close. I’m still getting some 6000 hits/hour, but it’s nowhere near what it was — it peaked at over 130000 hits/hour right after the Daily Mail published their article. All of the other metrics have died down as well — Twitter posts, news articles, textual abuse at the hands of Joe Rogan and Anthony Cumia fans…

As some of you have seen, it’s my intent to donate advertising revenues from the video on YouTube to the Massachusetts Audubon Society. (Just seemed like a good cause to me; I don’t generally believe in advertising for padding my own pockets, but I’m happy to use advertising for good — and frankly, I’ve always wondered how advertising on YouTube worked, so this seemed like as good of an opportunity as any.) As you may have seen in my tweets, a couple of bogus ContentID claims have resulted in some of that revenue/earnings being missed out on: when a video is being disputed under ContentID, *nobody* can run ads, which means that the revenue is lost.

All the stats below are based on rough estimates of views from the YouTube views analytics dashboard combined with a timeline of activity; none are based on any AdSense data.

A brief summary of my monetization history:

• First million views: Only display ads in-page, starting around 100k views. These are the ads on the upper right corner of the page. Since much of my content was in embeds, rather than visits directly to YouTube, these were not particularly lucrative, but due to the overall popularity, I did have something like 800k monetized visits on the watch page (this excludes around another 250k of non-monetized visits due to ContentID.) These ads are the only ads that have been on for the majority of the time.
• At 1 million views, I enabled overlay ads. In total, I believe that I have had something like 1.5M monetized views with overlay ads enabled. (Another 800k have not been monetized due to ContentID claims.) Towards the end of this (at around 1.8M) there was a content ID claim, which was later cleared up.
• At 2.2 million views (after my first ContentID dispute was resolved), I enabled Trueview ads. There have been approximately 600k monetized playbacks with TrueView enabled.

I don’t actually have any real earnings statistics yet, because those things can take up to 7 days to process. (I also don’t know if I could share them if I did have them; a quick read of the ToS suggests probably not.) A quick Google search points to this article on YouTube CPM, but explains that it varies significantly depending on what your ratios are of videos getting pre-rolls. (With the vast number of embeds, and short content, I think it’s probably fair to think I’ll have a lower TrueView/skippable ad ratio than most videos would, but again — no numbers available yet.)

This is by far the most common question I get about my experience: “How much money are you making?” With the numbers I see reported on the internet, I think the answer is that you can expect to see about $2 in revenue for every 1000 views. (55% goes to you, 45% goes to YouTube.) So if I had been running ads on my channel for the entire duration, and had had no ContentID claims, it would be approximately $3500 in ad revenue thus far. With the actual reality of the situation, I’m probably significantly less than that, due to not advertising aggressively at the start, the type of video I actually published, and other issues.

Overall, this is just one component of what has been an eye-opening, thrilling, and exhausting experience. I’ll be doing more write-ups about how things have gone over the coming days and weeks — I can certainly say I’ve learned a lot about what having a viral video looks like, and as might be clear from the opening paragraph, it hasn’t all been sunshine and roses.

This morning, while out flying, a hawk attacked my quadcopter, while flying along the Charles River, in Cambridge.

You can see the video — one of my most popular ever — on YouTube:

Of course, because it’s popular, it also means you can see it in a lot of other places — LiveLeak being a ‘primary’ source of the copyright infringement. (I expect this is LiveLeak’s fault as much as it is YouTube’s fault when random people upload copyrighted crap.) I did find that on LiveLeak, the video claimed to have over 9000 views, compared to the 2000 or so on YouTube.

I’m getting a quick lesson in what happens when you create really popular content: People steal it left and right. 🙂 I’ve already filed two copyright claims against people who reuploaded to YouTube and put ads on it — I guess I’ll also be learning about YouTube’s copyright infringement resolution process.

In the mean time, if you want to check it out, I encourage you to share, like or watch on YouTube, rather than on one of the many clones out there 🙂 There’s only one real thing!

Like everyone on the internet, I’ve seen a lot about Ello in the last week or so. While I’m not convinced Ello is the next big thing, more recently, there have been articles about how Ello must be planning to sell you out, because their proposed business model can never work, and all Venture Capitalists require an exit strategy. Regardless of how true the latter may be, I am not convinced the former is true at all.

My initial forays into the online world were based on GeoCities and Tripod, like many other people of my generation. In my transition to college, LiveJournal became my home on the internet. It was my first work with an open source project. It was where I made friends, and it was even the website where I met my wife. It was also a website which was run, for years, based on a funding model which was entirely ad-free, at a time when banner ads were the way of the internet.

When the website started, in the early 2000s, “No ads, ever” was the mantra of the site (like Ello). The site was originally invite-based, so that growth was somewhat limited (like Ello). The site didn’t collect and sell your information to advertisers (like Ello). The site was funded by users who paid for additional features (like Ello); for LiveJournal, features included things like more user pictures, the ability to make posts by making a phone call, domain forwarding, advanced customization options for look and feel.

LiveJournal functioned as a business this way for a number of years; from at least 2002 – 2005, when it was bought by Six Apart, LiveJournal seemed like a functional business from the outside. It was a small business run by a small number of employees and supported by a dedicated volunteer base who worked to run areas like user support. There was enough of a business here to result in a sale to Six Apart in 2005; while no details of the deal were ever published, it seems reasonable to assume that it was considered to be a viable business at the time of the sale.

Now, LiveJournal was never started to be a business. It was started as a way for the creator to keep in touch with his friends. It was run as a semi-business, but as with many things started by people in their idealistic years during and shortly after college, sometimes they lose the ability to maintain the dedicated interest necessary to keep them going. (See also: Most of my early software.) After the sale to Six Apart, the “No ads on LiveJournal” policy slipped somewhat, and a number of social shifts caused a bit of a fall from grace in the somewhat utopian ideals that LiveJournal had. (Not the least among them that ads probably became significantly more profitable and effective…)

But LiveJournal isn’t the only social network that had this policy. After LiveJournal’s sale to SUP, some of the volunteers from LiveJournal decided that the things LiveJournal stood for were good, and that the system it had was workable, but it needed a bit more realistic business approach, and started Dreamwidth; like LiveJournal, the site is funded through people who purchase additional features for their accounts, rather than advertising. (One of the site’s Guiding Principles is “We won’t accept or display third-party advertising on our service, whether text-based or banner ads. We are personally and ideologically against displaying advertising on a community-based service.”)

Dreamwidth was founded in 2008, and opened to the public in 2009; it started with invite codes and later was able to move away from them. The site has more than 2M registered accounts, and although it’s not going to be the next Facebook, it’s probably reasonable to assume that the site isn’t losing money hand over fist. (It has been around for 5 years, and shows no signs of unhealthiness that I can see from the outside, though I have no inside knowledge.) It is funded by people who purchase additional features for their accounts.

The idea of free accounts being paid for by people who want additional features is not new. The claim from some that “…no one has ever tried it as a central business model, at least not in social” is clearly false. Some people have tried it. It has even, to some extent, been successful. And although it may be that Ello is not planning to do what they say they’re going to is possible, it seems entirely more likely that Ello is trying to follow in the footsteps of those who have gone before it and created social networks that millions called home in the earlier days of the internet.

If what Ello wants to make is a “sustainable business”, as they’ve claimed, then there is no reason to think that they can’t do it by following exactly the funding model they have proposed. I hope all goes well for them, and that they’re able to hold onto those principles. If they’re not, and you’re still looking for that ad-free, friendly environment that you miss from the earlier internet… there’s always the comfort of Dreamwidth.

In 2007, I posted a video to YouTube; it was just a 5 minute, silent how-to video showing how to load data that you had in a shapefile, open it in QGIS, style it, export it to a mapfile, and load it into OpenLayers. I’ve given pretty much this exact presentation to groups around the world: from Cape Town, South Africa, to Osaka, Japan, but at the time it was just a quick demo I put together, related to a wiki page: Mapping Your Data, in the OpenLayers wiki.

I hadn’t paid attention to it in forever — I uploaded it to YouTube back in 2007, and I haven’t really thought about it since. So as I’m using YouTube a bit more recently, I actually looked at my analytics… and realized that this video still gets *400 views every month*, with an average of two minutes watched per view.

This means that 20 minutes gets wasted watching this video every day (on average); that is more time than I spend on YouTube in an average week. (Given my new employer, I can imagine that changing somewhat in the near future.)

Amusingly enough, for a long time this wasn’t my most popular video; the OpenLayers video is a bit long, and with no sound, can be a bit of a drag. (The pace of it, even 7 years on, still impresses me though; I spent a whole weekend just going through the motions to get the flow down. It really does work nicely.) My most popular video for a long time was an N95 Accelerometer Demo:

This demo showed the use of a Python script to use the accelerometer and simple 2d graphics to move a ball around the screen. (The Symbian Python APIs for interacting with 2d graphics were terrific, and I wish modern phones had something similarly easy.) In the week after that video launched, it had *1500* views; but it was a flash in the pan, and hasn’t maintained its popularity, getting only 2 watches in the last 30 days. (This video was popular enough that I was invited to join the YouTube monetization program, unlike the OpenLayers video, which was never ‘viral’ enough to get there.)

I’ve never been much of a video guy before — another thing I can see changing — but I’m now putting together some of the videos from my quadcopter flights. Last night, I published my bloopers from the first couple days of flying:

But I guess I can never expect, based on my current views, that anything I do on YouTube will be more popular than a silent video I published about OpenLayers back in 2007.

I guess this really just goes back to: OpenLayers was a unique experience, and is probably the most impressive thing I will actually work on for the benefit of the internet at large… ever.

So, recently I realized that my website essentially hasn’t been updated since 2007. (I noticed last week that I had an announcement on there about the presentation I gave at FOSS4G… in 2006. As a current event. Yeah, my website isn’t particularly up to date.)

So, I’m working on moving bits and pieces of it to git, so I can track history and changes more easily, and also because I think that having it open presents no risk and provides opportunity for me to more easily track what I’m doing there in public. However, since the website has been essentially a dumping ground for my various crap for the past 8 years, I’m being a bit cautious about it, and moving things in one by one. (There is some content in my webdir which should not be publicly available — everything from passwords to old client content which is currently protected by password.)

In the process, I came across a collection of articles that … I might have written.

I say might, because I have no real memory of writing them. However, they have examples which use my name; they are written in a style which is semi-consistent with how i would write, and most importantly, they’re hosted under my formal/writing directory (which no one else has ever had access to), which is also explicitly prevented from being crawled by robots.txt. The modification dates on these files are in November 2005 (all on the same day; which likely means they were copied en masse from something even older).

They’re all of things that were of interest to me at the time — how to integrate RSS into IRC bots, how to parse mood information from LiveJournal RSS feeds, how to parse the LiveJournal live stream, etc.

The thing is… I have no idea why I have them. Or why I hid them.

At the time, I was working on getting contracting work — this was before I had started working for Ning, during a time when I was grasping for contracting work to make ends meet, having ended my commute back and forth to wedü. I guess this means that I was actually (essentially) unemployed for a brief period there… perhaps I thought writing articles like this was a good way to draw attention to myself. But then my question becomes… why did I hide them from my robots.txt?!

My biggest fear is that this content is actually stolen from somewhere else — the topics are general enough that it could be. However, given the specifics of the topics, I think that they can’t all be.

• Setting Up Your Own Planet Aggregator
• Find Current Mood and Music of LiveJournal Users
• Creating a Feed From A Subversion Repository
• Watch LiveJournal Entries Flow By
• Adding RSS Content to Your Hompeage
• WordPress Daily Delicious Post
• Receive RSS Updates Via IRC
• del.icio.us Links

While I would believe that someone else could write some of those, it would surprise me that anyone else wrote all of those, and put them in one place. So I think this *is* writing I created… but I have no idea why. With no historical version control, I can’t refer to that (curse you, younger, more foolish self!), and at this point I’ve switched computers enough times that there isn’t much left in the way of detrious of conversation logs from that time; even my email records only go back to 2006 consistently. I think this was actually content that was probably created before transitioning my website to a hosted webserver; this was from the days when it just ran off the server at the apartment.

I’ve googled titles and snippets of these; I see no other evidence of them on the web. I think this means these probably are my original content. Given that, I think I’m going to go with: share and enjoy! I’ve put these into the github repo under formal/writing, where you’re free to do with them what you will. I may try to tidy them up… heck, I may even try to write a few more, because I actually enjoy the writing style.

I just… find it weird that I have this content that the internet at large doesn’t have access to which I apparently wrote 8 years ago and have absolutely no memory of… what a weird thing to find.

(If you have any memory of me writing these articles, why, or evidence I’ve shared them in the past, I’d love to hear it!)

A friend of mine recently had his account ‘restricted’ by Flickr for “potentially offensive content”. His comments on Flickr’s treatment of the issue are concerning to me: as he describes in his entry:

They still haven’t gotten around to telling me what this content IS or why it is potentially offensive. … “There are millions of people visiting Flickr who may not see the world the same way you do” is certainly true, but a dangerous way to run a community.

I understand Flickr’s needs for the adequate use of controls on users accounts in order to make the site they want it to be, but threats of account termination on a service like Flickr are very concerning to me. I’ve invested a lot of work in my photos on Flickr, and although I’d like to continue to use the service, the feedback of anonymous users being able to lead to account termination with no human feedback in the loop is a scary prospect.

I’d love to hear that this is an oversight somewhere, but if I don’t see Dan get some feedback on his issues, maybe I should consider abandoning Flickr as the primary place to host my photos. Convenience isn’t really worth the possible risk of having an account terminated — presumably taking all its content with it — without even having feedback from a human on why. Clearly, that risk always exists with the Terms of Service that exist on the site — almost every site I’ve seen which has lawyers behind it has some sort of “We can do whatever we want, and in that case, you’re screwed” clause in their terms of use — but to have it become clear that they follow this path in something that seems a routine matter to resolve is a somewhat scary precedent.

In the Flickr authentication API documentation regarding their Authentication Specification, the example in section 9.2 seems to have an invalid value.

They provide the following:

• API Key: 987654321
• Parameters: method=flickr.auth.getFrob
• Secret: foobarbaz

According to section 8, this should result in taking the md5sum of:

foobarbazapi_key987654321methodflickr.auth.getFrob

Which is 91216d07a9e945a8e2bf2b2660e7ba86 . Hoewever, the md5sum used in the api_sig parameter in the example is 5f3870be274f6c49b3e31a0c6728957f. (I’ve tried several different combinations, and been unable to figure out what this value might be based on.)

This yws-flickr post also confirms that the documentation appears to be incorrect.

I couldn’t find the appropriate place to put this given a short search, so I’m publishing it here in hopes that it will remind me to inform someone who might be able to fix it. (There are 5 authors listed on the spec: Emailing all five of them feels like overkill, especially since I’m pretty sure that not all of them still actively work on flickr.) If you know anyone who might be involved, feel free to pass it on…