Archive for the 'Web Publishing' Category

Hawk attacks Drone: The Hype Dies Down

Posted in Web Publishing, YouTube on October 14th, 2014 at 23:04:14

It seems like my 15 minutes of fame are mostly coming to a close. I’m still getting some 6000 hits/hour, but it’s nowhere near what it was — it peaked at over 130000 hits/hour right after the Daily Mail published their article. All of the other metrics have died down as well — Twitter posts, news articles, textual abuse at the hands of Joe Rogan and Anthony Cumia fans…

As some of you have seen, it’s my intent to donate advertising revenues from the video on YouTube to the Massachusetts Audubon Society. (Just seemed like a good cause to me; I don’t generally believe in advertising for padding my own pockets, but I’m happy to use advertising for good — and frankly, I’ve always wondered how advertising on YouTube worked, so this seemed like as good of an opportunity as any.) As you may have seen in my tweets, a couple of bogus ContentID claims have resulted in some of that revenue/earnings being missed out on: when a video is being disputed under ContentID, *nobody* can run ads, which means that the revenue is lost.

All the stats below are based on rough estimates of views from the YouTube views analytics dashboard combined with a timeline of activity; none are based on any AdSense data.

A brief summary of my monetization history:

  • First million views: Only display ads in-page, starting around 100k views. These are the ads on the upper right corner of the page. Since much of my content was in embeds, rather than visits directly to YouTube, these were not particularly lucrative, but due to the overall popularity, I did have something like 800k monetized visits on the watch page (this excludes around another 250k of non-monetized visits due to ContentID.) These ads are the only ads that have been on for the majority of the time.
  • At 1 million views, I enabled overlay ads. In total, I believe that I have had something like 1.5M monetized views with overlay ads enabled. (Another 800k have not been monetized due to ContentID claims.) Towards the end of this (at around 1.8M) there was a content ID claim, which was later cleared up.
  • At 2.2 million views (after my first ContentID dispute was resolved), I enabled Trueview ads. There have been approximately 600k monetized playbacks with TrueView enabled.

I don’t actually have any real earnings statistics yet, because those things can take up to 7 days to process. (I also don’t know if I could share them if I did have them; a quick read of the ToS suggests probably not.) A quick Google search points to this article on YouTube CPM, but explains that it varies significantly depending on what your ratios are of videos getting pre-rolls. (With the vast number of embeds, and short content, I think it’s probably fair to think I’ll have a lower TrueView/skippable ad ratio than most videos would, but again — no numbers available yet.)

This is by far the most common question I get about my experience: “How much money are you making?” With the numbers I see reported on the internet, I think the answer is that you can expect to see about $2 in revenue for every 1000 views. (55% goes to you, 45% goes to YouTube.) So if I had been running ads on my channel for the entire duration, and had had no ContentID claims, it would be approximately $3500 in ad revenue thus far. With the actual reality of the situation, I’m probably significantly less than that, due to not advertising aggressively at the start, the type of video I actually published, and other issues.

Overall, this is just one component of what has been an eye-opening, thrilling, and exhausting experience. I’ll be doing more write-ups about how things have gone over the coming days and weeks — I can certainly say I’ve learned a lot about what having a viral video looks like, and as might be clear from the opening paragraph, it hasn’t all been sunshine and roses.

Hawk Attacks Quadcopter!

Posted in Web Publishing, YouTube on October 9th, 2014 at 01:21:53

This morning, while out flying, a hawk attacked my quadcopter, while flying along the Charles River, in Cambridge.

You can see the video — one of my most popular ever — on YouTube:

Of course, because it’s popular, it also means you can see it in a lot of other places — LiveLeak being a ‘primary’ source of the copyright infringement. (I expect this is LiveLeak’s fault as much as it is YouTube’s fault when random people upload copyrighted crap.) I did find that on LiveLeak, the video claimed to have over 9000 views, compared to the 2000 or so on YouTube.

I’m getting a quick lesson in what happens when you create really popular content: People steal it left and right. :) I’ve already filed two copyright claims against people who reuploaded to YouTube and put ads on it — I guess I’ll also be learning about YouTube’s copyright infringement resolution process.

In the mean time, if you want to check it out, I encourage you to share, like or watch on YouTube, rather than on one of the many clones out there :) There’s only one real thing!

Social Networks and Business Plans

Posted in LiveJournal, Web Publishing on September 28th, 2014 at 10:20:27

Like everyone on the internet, I’ve seen a lot about Ello in the last week or so. While I’m not convinced Ello is the next big thing, more recently, there have been articles about how Ello must be planning to sell you out, because their proposed business model can never work, and all Venture Capitalists require an exit strategy. Regardless of how true the latter may be, I am not convinced the former is true at all.

My initial forays into the online world were based on GeoCities and Tripod, like many other people of my generation. In my transition to college, LiveJournal became my home on the internet. It was my first work with an open source project. It was where I made friends, and it was even the website where I met my wife. It was also a website which was run, for years, based on a funding model which was entirely ad-free, at a time when banner ads were the way of the internet.

When the website started, in the early 2000s, “No ads, ever” was the mantra of the site (like Ello). The site was originally invite-based, so that growth was somewhat limited (like Ello). The site didn’t collect and sell your information to advertisers (like Ello). The site was funded by users who paid for additional features (like Ello); for LiveJournal, features included things like more user pictures, the ability to make posts by making a phone call, domain forwarding, advanced customization options for look and feel.

LiveJournal functioned as a business this way for a number of years; from at least 2002 - 2005, when it was bought by Six Apart, LiveJournal seemed like a functional business from the outside. It was a small business run by a small number of employees and supported by a dedicated volunteer base who worked to run areas like user support. There was enough of a business here to result in a sale to Six Apart in 2005; while no details of the deal were ever published, it seems reasonable to assume that it was considered to be a viable business at the time of the sale.

Now, LiveJournal was never started to be a business. It was started as a way for the creator to keep in touch with his friends. It was run as a semi-business, but as with many things started by people in their idealistic years during and shortly after college, sometimes they lose the ability to maintain the dedicated interest necessary to keep them going. (See also: Most of my early software.) After the sale to Six Apart, the “No ads on LiveJournal” policy slipped somewhat, and a number of social shifts caused a bit of a fall from grace in the somewhat utopian ideals that LiveJournal had. (Not the least among them that ads probably became significantly more profitable and effective…)

But LiveJournal isn’t the only social network that had this policy. After LiveJournal’s sale to SUP, some of the volunteers from LiveJournal decided that the things LiveJournal stood for were good, and that the system it had was workable, but it needed a bit more realistic business approach, and started Dreamwidth; like LiveJournal, the site is funded through people who purchase additional features for their accounts, rather than advertising. (One of the site’s Guiding Principles is “We won’t accept or display third-party advertising on our service, whether text-based or banner ads. We are personally and ideologically against displaying advertising on a community-based service.”)

Dreamwidth was founded in 2008, and opened to the public in 2009; it started with invite codes and later was able to move away from them. The site has more than 2M registered accounts, and although it’s not going to be the next Facebook, it’s probably reasonable to assume that the site isn’t losing money hand over fist. (It has been around for 5 years, and shows no signs of unhealthiness that I can see from the outside, though I have no inside knowledge.) It is funded by people who purchase additional features for their accounts.

The idea of free accounts being paid for by people who want additional features is not new. The claim from some that “…no one has ever tried it as a central business model, at least not in social” is clearly false. Some people have tried it. It has even, to some extent, been successful. And although it may be that Ello is not planning to do what they say they’re going to is possible, it seems entirely more likely that Ello is trying to follow in the footsteps of those who have gone before it and created social networks that millions called home in the earlier days of the internet.

If what Ello wants to make is a “sustainable business”, as they’ve claimed, then there is no reason to think that they can’t do it by following exactly the funding model they have proposed. I hope all goes well for them, and that they’re able to hold onto those principles. If they’re not, and you’re still looking for that ad-free, friendly environment that you miss from the earlier internet… there’s always the comfort of Dreamwidth.

OpenLayers: Still popular on YouTube, years later.

Posted in OpenLayers, YouTube on March 22nd, 2014 at 06:32:32

In 2007, I posted a video to YouTube; it was just a 5 minute, silent how-to video showing how to load data that you had in a shapefile, open it in QGIS, style it, export it to a mapfile, and load it into OpenLayers. I’ve given pretty much this exact presentation to groups around the world: from Cape Town, South Africa, to Osaka, Japan, but at the time it was just a quick demo I put together, related to a wiki page: Mapping Your Data, in the OpenLayers wiki.

I hadn’t paid attention to it in forever — I uploaded it to YouTube back in 2007, and I haven’t really thought about it since. So as I’m using YouTube a bit more recently, I actually looked at my analytics… and realized that this video still gets *400 views every month*, with an average of two minutes watched per view.

This means that 20 minutes gets wasted watching this video every day (on average); that is more time than I spend on YouTube in an average week. (Given my new employer, I can imagine that changing somewhat in the near future.)

Amusingly enough, for a long time this wasn’t my most popular video; the OpenLayers video is a bit long, and with no sound, can be a bit of a drag. (The pace of it, even 7 years on, still impresses me though; I spent a whole weekend just going through the motions to get the flow down. It really does work nicely.) My most popular video for a long time was an N95 Accelerometer Demo:

This demo showed the use of a Python script to use the accelerometer and simple 2d graphics to move a ball around the screen. (The Symbian Python APIs for interacting with 2d graphics were terrific, and I wish modern phones had something similarly easy.) In the week after that video launched, it had *1500* views; but it was a flash in the pan, and hasn’t maintained its popularity, getting only 2 watches in the last 30 days. (This video was popular enough that I was invited to join the YouTube monetization program, unlike the OpenLayers video, which was never ‘viral’ enough to get there.)

I’ve never been much of a video guy before — another thing I can see changing — but I’m now putting together some of the videos from my quadcopter flights. Last night, I published my bloopers from the first couple days of flying:

But I guess I can never expect, based on my current views, that anything I do on YouTube will be more popular than a silent video I published about OpenLayers back in 2007.

I guess this really just goes back to: OpenLayers was a unique experience, and is probably the most impressive thing I will actually work on for the benefit of the internet at large… ever.

Hidden Writing Archives

Posted in Web Publishing on November 2nd, 2013 at 23:48:18

So, recently I realized that my website essentially hasn’t been updated since 2007. (I noticed last week that I had an announcement on there about the presentation I gave at FOSS4G… in 2006. As a current event. Yeah, my website isn’t particularly up to date.)

So, I’m working on moving bits and pieces of it to git, so I can track history and changes more easily, and also because I think that having it open presents no risk and provides opportunity for me to more easily track what I’m doing there in public. However, since the website has been essentially a dumping ground for my various crap for the past 8 years, I’m being a bit cautious about it, and moving things in one by one. (There is some content in my webdir which should not be publicly available — everything from passwords to old client content which is currently protected by password.)

In the process, I came across a collection of articles that … I might have written.

I say might, because I have no real memory of writing them. However, they have examples which use my name; they are written in a style which is semi-consistent with how i would write, and most importantly, they’re hosted under my formal/writing directory (which no one else has ever had access to), which is also explicitly prevented from being crawled by robots.txt. The modification dates on these files are in November 2005 (all on the same day; which likely means they were copied en masse from something even older).

They’re all of things that were of interest to me at the time — how to integrate RSS into IRC bots, how to parse mood information from LiveJournal RSS feeds, how to parse the LiveJournal live stream, etc.

The thing is… I have no idea why I have them. Or why I hid them.

At the time, I was working on getting contracting work — this was before I had started working for Ning, during a time when I was grasping for contracting work to make ends meet, having ended my commute back and forth to wedü. I guess this means that I was actually (essentially) unemployed for a brief period there… perhaps I thought writing articles like this was a good way to draw attention to myself. But then my question becomes… why did I hide them from my robots.txt?!

My biggest fear is that this content is actually stolen from somewhere else — the topics are general enough that it could be. However, given the specifics of the topics, I think that they can’t all be.

While I would believe that someone else could write some of those, it would surprise me that anyone else wrote all of those, and put them in one place. So I think this *is* writing I created… but I have no idea why. With no historical version control, I can’t refer to that (curse you, younger, more foolish self!), and at this point I’ve switched computers enough times that there isn’t much left in the way of detrious of conversation logs from that time; even my email records only go back to 2006 consistently. I think this was actually content that was probably created before transitioning my website to a hosted webserver; this was from the days when it just ran off the server at the apartment.

I’ve googled titles and snippets of these; I see no other evidence of them on the web. I think this means these probably are my original content. Given that, I think I’m going to go with: share and enjoy! I’ve put these into the github repo under formal/writing, where you’re free to do with them what you will. I may try to tidy them up… heck, I may even try to write a few more, because I actually enjoy the writing style.

I just… find it weird that I have this content that the internet at large doesn’t have access to which I apparently wrote 8 years ago and have absolutely no memory of… what a weird thing to find.

(If you have any memory of me writing these articles, why, or evidence I’ve shared them in the past, I’d love to hear it!)

How to Scare Your Users, by Flickr

Posted in Flickr on October 24th, 2008 at 13:05:58

A friend of mine recently had his account ‘restricted’ by Flickr for “potentially offensive content”. His comments on Flickr’s treatment of the issue are concerning to me: as he describes in his entry:

They still haven’t gotten around to telling me what this content IS or why it is potentially offensive. … “There are millions of people visiting Flickr who may not see the world the same way you do” is certainly true, but a dangerous way to run a community.

I understand Flickr’s needs for the adequate use of controls on users accounts in order to make the site they want it to be, but threats of account termination on a service like Flickr are very concerning to me. I’ve invested a lot of work in my photos on Flickr, and although I’d like to continue to use the service, the feedback of anonymous users being able to lead to account termination with no human feedback in the loop is a scary prospect.

I’d love to hear that this is an oversight somewhere, but if I don’t see Dan get some feedback on his issues, maybe I should consider abandoning Flickr as the primary place to host my photos. Convenience isn’t really worth the possible risk of having an account terminated — presumably taking all its content with it — without even having feedback from a human on why. Clearly, that risk always exists with the Terms of Service that exist on the site — almost every site I’ve seen which has lawyers behind it has some sort of “We can do whatever we want, and in that case, you’re screwed” clause in their terms of use — but to have it become clear that they follow this path in something that seems a routine matter to resolve is a somewhat scary precedent.

Flickr Authentication API Documentation Bug

Posted in Flickr, Web Publishing on October 18th, 2008 at 08:54:17

In the Flickr authentication API documentation regarding their Authentication Specification, the example in section 9.2 seems to have an invalid value.

They provide the following:

  • API Key: 987654321
  • Parameters: method=flickr.auth.getFrob
  • Secret: foobarbaz

According to section 8, this should result in taking the md5sum of:

foobarbazapi_key987654321methodflickr.auth.getFrob

Which is 91216d07a9e945a8e2bf2b2660e7ba86 . Hoewever, the md5sum used in the api_sig parameter in the example is 5f3870be274f6c49b3e31a0c6728957f. (I’ve tried several different combinations, and been unable to figure out what this value might be based on.)

This yws-flickr post also confirms that the documentation appears to be incorrect.

I couldn’t find the appropriate place to put this given a short search, so I’m publishing it here in hopes that it will remind me to inform someone who might be able to fix it. (There are 5 authors listed on the spec: Emailing all five of them feels like overkill, especially since I’m pretty sure that not all of them still actively work on flickr.) If you know anyone who might be involved, feel free to pass it on…

Flickr’s Interestingness Patent

Posted in Flickr, Photography on October 16th, 2008 at 06:02:07
Easter Sunday at First Parish
My most ‘interesting’ photo

One of the things that has always confused me is how Flickr’s ‘interestingness’ score worked. It’s clearly not based directly on views — my most ‘interesting’ photos have ~500 views, while my most popular ones have ~5000. The same is true of comments, tags, groups, etc: Some of my most interesting photos are, all in all, the least obviously ‘popular’ ones.

However, reading through the patent on Interestingness, I see:

[0027] The statistics engine generates statistics and other metrics based upon aggregated metadata. In one embodiment, the statistics engine determines the popularity of metadata (e.g., tags) within a grouping of media objects over a predetermined time period. For example, the statistics engine may determine the number of different users that have assigned a particular tag to one or more media objects within all groups on the system, within a single group, or within a set of media objects, over the last 24 hours. The aggregation engine may determine (and display) a histogram of the tags, and may determine the most frequently assigned tags (at any point in time or over a predetermined time period) by determining those tags either having a frequency exceeding a minimum threshold frequency or belonging to a predetermined number of the most popular tags.

The patent application doesn’t directly claim that this is used in the interestingness calculation, but this type of time-period based tag aggregation/valuation is clearly a non-obvious metric that can’t be calculated by looking directly at user-visible information on a single photo.

Still, none of this really changes the fact that I wish that flickr had a per-person ‘most interesting this week’ sort: my ‘eastern bunny’ photo above has been at the top of the interesting list for ages, and I want new interesting photos, damnit!

Finished FOSS4G Photo Uploads

Posted in FOSS4G 2008, Flickr on October 10th, 2008 at 05:13:26

I finally finished uploading all the photos I took at/around the FOSS4G conference.

FOSS4G 2008 Set

Includes photos from:

  • Various nights hanging out at the bars/hotels during the conference
  • A trip up Table Mountain with OpenLayers/OpenGeo folks
  • GeoDjango Workshop
  • Sessions, exhibition hall, etc.
  • Pictures from the Gala Dinner at Moyo
  • Closing Session
  • OSGeo AGM
  • OpenLayers Workshop
  • GeoServer Workshop

Photos that are taken at the conference center/of conference proceedings are also tagged with foss4g2008:

foss4g2008 tagged

And most photos of people who I recognized are tagged with first name in the title, and with a username (as used on IRC or other unique identifier) attached as a tag:

Photos of Arnulf
Photos of Steven

Photos of the OSGeo AGM are also tagged as such:

OSGeo AGM Photos

I’ve made my tagging settings as open as possible, but I believe you still need to be a contact on flickr in order to add notes/tags to photos: simply add me as a contact, and I’ll add you back.

Any names I got wrong, please let me know, either via email or by simply commenting on the flickr photo.

Thanks to all for helping to create such a photogenic conference.

Free Maps for Free Guides

Posted in Locality and Space, Mapserver, OpenGuides, OpenLayers, TileCache on February 11th, 2007 at 08:46:05

A bit more than a year ago, when I was just learning how to use the Google Maps API, I put together a patch for the OpenGuides software, adding Google Maps support. It seemed the logical way to go: It wasn’t perfect, since Google Maps are obviously non-free, but it seemed like a better way to get the geographic output from OpenGuides out there than anything else at the time.

Since I did that, I’ve learned a lot. Remember that 18 months ago, I’d never installed MapServer, had no idea what PostGIS was, and didn’t realize that there were free alternatives to some of the things that Google had done. Also, 9 months ago, there was no OpenLayers, or any decent open alternative to the Google Maps API.

In the past 18 months, that’s all changed. I’ve done map cartography, I’ve done setting up of map servers, and I worked full time for several months on the OpenLayers project. Although my direction has changed slightly, I still work heavily with maps on a daily basis, and spend more of my time on things like TileCache, which lets you serve map tiles at hundreds of requests/second.

So, about a month ago, I went back to the Open Guide to Boston, and converted all the Google Maps API calls to OpenLayers API calls. The conversion took about an hour, as I replaced all the templates with the different code. (If I was writing it again, it would have taken less time, but this was my first large scale open source Javascript undertaking, long before I gained the knowledge I now have from working with OpenLayers.) In that hour, I was able to convert all the existing maps to use free data from MassGIS, rather than the copyrighted data from Google, and to have Google as a backup: a Map of Furniture Stores can show you the different. You’ll see that there are several layers — one of which is a roadmap provided by me, one from Google — and one from the USGS, topographic quad charts.

It’s possible that some of this could have been done using Google as the tool. There’s nothing really magical here. But now, the data in the guide is no longer displayed by default on top of closed source data that no one can have access to. Instead, it’s displayed on top of an open dataset provided by my state government.

This is how the world should work. The data that the government collects should be made available to the people for things exactly like this. It shouldn’t require a ‘grassroots remapping’: There are examples out there of how to do it right. I find it so depressing to talk to friends in the UK, who not only don’t have the 1:5000 scale quality road data that Massachusetts provides, but doesn’t even provide TIGER-level data that the geocoder on the Open Guide to Boston uses.

Free Guides, with Free Maps. That’s the way it should be. The fact that it isn’t everywhere is sad, but at least it’s good to know that the technology is there. Switching from Google to OpenLayers is an easy task — it’s what happens next that is a problem. You need the data from somewhere, and it’s unfortunate that that ’somewhere’ needs to be Google for so many people. I’m thankful to MassGIS and to the US Government for providing the data I can use, and to all the people who helped me learn enough to realize that using Google for everything is heading the wrong way when you want to not be beholden to a specific set of restrictions placed on a corporate entity.