Archive for the 'default' Category

Creating Sculptures of the World with Computers and Math

Posted in default on September 18th, 2014 at 22:12:35

The world around us is a complex place. Sometimes you just want to hold a tiny piece of it in your hand — and with some relatively low cost technological investment, you can do so. Using a $500 quadcopter, I have successfully captured images of a building, converted those images to a 3D model, and 3D printed that model — creating a small model of Cambridge City Hall that I can hold in my hand. The process requires no special skills — just some financial investment and time.

IMG_20140914_203339

In March of this year, I purchased a Phantom FC40, a $500 everything-you-need quadcopter. This device is easy to fly, comes with a built-in GPS, on-board camera (with a mount for a GoPro), and a remote — everything you need to start doing some amateur aerial photography. (You can see some of my videos in the FC40 Videos and One Minute Onboard to see some of the aerial photography I’ve done.)

P3214334

Capturing Photos

With quadcopter in hand, this weekend, I ventured to Cambridge City Hall. While there, despite the gusty winds, I captured approximately 20 minutes of video, attempting to film the building from as many angles as possible.[1] I was using the GoPro Hero 3+ Black I recently got, but for the purposes of this excercise, the FC40 camera would probably have been sufficient. I shot most footage in Narrow or Medium mode, to reduce the fisheye effect of the very wide angle GoPro lens; for the one section of video I shot in wide-angle, I removed the wide angle aspect using GoPro Studio before using the video.

Once I had the videos, I reviewed them, doing manual frame-grabs from the video to get coverage. On average, I took one shot for about every two seconds of usable video. (Usable video excludes video where the quadcopter is taking off, where it is facing the wrong direction, where it is flying to get to a different part of the building, where it is occluded by trees, etc.) Another option would be to simply use a program like ffmpeg to extract one frame every second:

ffmpeg -i ~/Documents/input-movie.mp4 -r 1 -f image2 ~/output/project%03d.jpg

The reasons not to do this are:

  • When flying the quadcopter, some portions (even in a sub-second window) are better than others. Motion blur is a non-trivial problem, even with 60fps capture rates; targeting manual screengrabs at slower motion, or during a more steady period makes a small but noticable difference.
  • Many of the shots were in the exact same coverage — largely due to the available landing space being all in front of the building. This means that extracting regular shots would have extracted many very very similar images, which would have increased processing time without noticably increasing quality of results.

Instead, I simply opened each video in VLC, and snapshotted the images that seemed to improve coverage of the building. (Option-Command-S on Mac; in the Video menu.)

Photo from City Hall Shoot Photo from City Hall Shoot Photo from City Hall Shoot Photo from City Hall Shoot Photo from City Hall Shoot

Building the Model

Once done with this, I loaded the images into a program called PhotoScan, the workhorse of this operation.

PhotoScan is an amazing tool. I say this, having tried a number of other tools — including commercial products like Autodesk’s 123d Catch and open source tools like VisualSFM. Nothing combined the ease of use and functional output of PhotoScan by a long shot. I’m currently using PhotoScan in 30 day trial mode, but despite the relatively steep price tag ($179 for single-user ’standard’ license) for what is only a hobby, I’m pretty well convinced I’m going to have to buy it, because the results are simply amazing.

With my 328 photos in hand, I added them to a chunk of a PhotoScan workspace, and set up a Batch Process (Workflow -> Batch Process).

Workflow

  1. Job Type: Align Photos. Change Point Limit to 5000, due to relatively small image size (1920 x 1080); further experiments show that this number ends up creating a better model than either 10000 or 20000 points, in a significantly shorter time window.)
  2. Job Type: Build Dense Cloud.
  3. Job Type: Build Mesh. Ensure that the Source Data is “Dense Cloud”.
  4. Job Type: Build Texture

Kicking off the build for these 328 photos uses all of the CPU on my laptop for approximately 1 hour. The majority of this time is spent matching photos via the “Align Photos” step. (An attempt with 20000 points took about 4 hours instead of just one.)

Setting up workflow

This produces a textured model, fully visible in 3D. In this particular case, anything other than City Hall is pretty … ‘melty’, as I like to call it, since it was only captured incidental to the primary flight objective (city hall itself). From here, you can save the model as a .obj file to use in your favorite 3d program. You can also share it via the web: once exported as a .obj, you can zip the resulting files (including the texture) up, and share for free on Sketchfab: Cambridge City Hall on Sketchfab.

Photoscan assembledPhotoscan assembled Photoscan assembledPhotoscan Assembled

My final goal is a physical version of the centerpiece of this model: City Hall. To achieve this, my next step is Meshlab. Meshlab can open the “Wavefront Object (.obj)” file I saved from Photoscan without a problem. Using the “Select Vertices” tool and the “Delete Vertices” tools, I am able to remove the extraneous parts of the model, leaving behind only City Hall itself. Using the “Export Mesh As” functionality, I can export this as a .stl file — the file format that my 3D printer uses.[2]

Trimming City Hall Trimmed City Hall

Printing the Model - aka ‘hacking it to work’

The next step is to load up the STL file. Since I don’t actually know how to rotate my model, I’ll load it into Repetier-Host, so I can do rotation in my plating process. Playing around with the angles, I take my STL file, and find that a rotation of 204 degrees in the X direction, -5 degrees in the Y direction, and -15 in the Z direction appears to give me a reasonably sane looking model. However, it’s still floating a bit above the bottom, thanks to a small portion of the model that is particularly warped due to low photo coverage. I choose to slice the model anyway, using Slic3r to generate gcode.

3d Printing: Plating

As expected, the model has generated some pretty bogus first couple layers. However, judicious use of copy paste can help me: Using the Repetier jump-to-layer buttons, I remove the first 3 layers of the model, then duplicate the g-code for the 5th layer (The first ‘real’ layer with more than a few spots of actual content), replacing the Z index with the correct height for the first, second, third, and fourth layers.

3d Printing Layers

With these relatively minor modifications made, my model is ready to print; I copy it to my SD card, and send it off to the printer. An hour or so later, I have a 3D sculpture that matches my model pretty well.

IMG_20140914_203339

IMG_20140914_203015 IMG_20140914_202925 IMG_20140914_202855

[1] This can be a challenge in an area where your building is occluded by many trees; shooting shots from the ground can help with this, but I didn’t do any of this for this particular project.
[2] The model that I produce from Meshlab is frankly pretty crappy. A lot of people with experience in this space could probably trivially improve on what I’ve got; I just don’t know much about 3D Model work. Whenvever I open blender, I start with a cube, and end up with something that looks more like a many-tentacled one of Lovecraft’s imagining than reality. As such, the 3d printing process can be a bit … fraught.

Response to new FAA Policy Document

Posted in default on June 25th, 2014 at 14:57:44

Recently, the FAA issued a new policy document on their intended enforcement of drone activities, re-iterating their stance that commercial activities are prohibited, and that they have full regulatory authority over remote controlled model aircraft under their authority to regulate all aircraft.

They asked for comments on regulations.gov, and I submitted my comments. Since comments are only made public after review by the agency involved, I am posting it here as well. (My submission identifier is 1jy-8cv7-nzgk.)

This entire policy is based on a mistaken notion that the “the FAA has considered model aircraft to be aircraft that fall within the statutory and regulatory definitions of an aircraft”; the definition that the FAA uses (”contrivances or devices that are “invented, used, or designed to navigate, or fly in, the air.””) is so broad as to describe everything from a commercial jetliner to a paper airplane, and was clearly never the intent of the creators of the FAA. Considering model aircraft included in the FAA’s regulatory definitions is an absurd notion, as supported by the one case decided so far by the NTSB court: In Pirker v. Huerta: “It is concluded that, as Complainant has not issued an enforceable FAR regulatory rule governing model aircraft operation; has historically exempted model aircraft from the statutory FAR definitions of aircraft by relegating model aircraft operations to voluntary compliance with the guidance expressed in AC 91-57, Respondent’s model aircraft operation was not subject to FAR regulation and enforcement.” (http://www.scribd.com/doc/211088332/Pirker-Decision)

Given the current lack of authority that the FAA has to regulate model aircraft under current rules, this policy of enforcement can not be based on a set of rules: instead, the FAA is attempting to govern based solely on policy documents, entirely evading the process of rulemaking that is required for passing regulations.

This document/rule does not appear to be changing anything in this regard: it is another case of the FAA making an effort to define a policy for enforcement without changing the basic rules and regulations that the agency is working under. The continued lack of any rules or regulations that support this policy mean that this policy is (again) attempting to invoke fear, uncertainty, and doubt into a fledgling industry, hurting the operators of model aircraft who are currently doing so with commercial intent.

While I understand that the FAA has a mandate to ensure safe integration of civilian sUAS into the airspace, this policy does not move further towards that goal; it sets up limitations which are not based on rules with no eye towards safety, and does nothing to move towards the requirements of the FAA Reauthorization Act of 2012.

This policy document continues the FAA’s enforcement of non-existent rules, treating advisory guidelines as law. This position is untenable, and allowing this rule/enforcement policy to stand as is is unhelpful and harmful to the overall community; it will only increase uncertainty, as it continues to be based on FAA policy rather than an official rulemaking process.

Video: It’s Hard

Posted in Photography, default on May 7th, 2014 at 07:23:53

So, I’ve been an amateur photographer for years. I started helping to take pictures of family vacations with my dad’s AE-1 back when I was in grade school. I took photo classes in college, and I got my first digital camera in 2003. (I still have it, by the way.)

close up of a flash for a canon camera

I’ve had a Digital Rebel for the past 9 years. (I got my first in December of 2005.) I’ve had my most recent camera for another 3 or so. I’ve brought these cameras to a dozen conferences and events; I’ve used them in bars, on skating rinks, at the beach. I have uploaded over 6000 public pictures to Flickr — which means that in that time, I’ve probably *thrown away* another 12000 or so. (My rate of success has drifted over time, from as low as 10% to as high as 75%.)

I won’t claim I’m an expert photographer — I’m good at taking candid portraits, but a lot of other things continue to evade my skills — but I’ve taken a lot of pictures, and I like a lot of what I take these days.

And now that I’ve started looking at video, that all changes.

I have no meaningful experience with video. I helped run the camcorder at a couple of events for my dad (who has been doing video for years), but that’s it. Now I’m faced with an entirely different medium, with a different set of requirements, and a different set of tools, and I find myself feeling completely frustrated by my efforts.

The number one thing that I learned with digital photography is you have to prepared to throw away 90% of what you take, especially when you get started. (It took me a while to get good at this.) Most of the pictures you take will be crap; even if you got the frame and content you tried to capture, you might have caught a person blinking, or have a photo that’s more out of focus, or with the wrong color balance to the point you can’t adjust it any more.

With video, the same is true (at least, I feel that way): most shots don’t come out the way you want them to. Even worse: a minor edit to a shot can change it from being perfectly reasonable to being silly (or from being perfectly silly to looking just reasonable). So in addition to finding the perfect shot, you’ve got to edit it right.

Attention spans are short. I once read an article from someone complaining about ‘quick cuts’ in modern TV. Since then I’ve paid careful attention, and learned that it is rare for most TV to have single visual cuts that are longer than about 3-5 seconds. (If you watch fast-paced shows like The Amazing Race, this drops by half; you have tons of super-fast clips.) Even for a relatively short, 3.5 minute video, my Ohanami video, on average people only watch one minute of it before they stop. (And I consider that video one of my better / more entertaining efforts so far!)

picture of cherry blossoms

When I started editing video from my quadcopter, Jess jokingly sent me a link to the Friend Who Sent Link To 8-Minute YouTube Video Must Be Fucking Delusional Onion article, in response to my first Phantom FC40 edited video. I defended my position, indicating that I had put a lot of work into trimming out the boring parts… but since then, I’ve become much more ruthless. (In the case of the Ohanami video, I edited approximately 1.5 hours of total video footage into a 3 minute clip reel, and it was relatively well-received.) As usual, my lovely wife was right: people don’t care about 90% of the footage I’m filming, a lesson I should have learned from photography.

Even beyond editing, I’ve got new equipment problems. While the DSLR is great for shooting photos, shooting video with it means holding it out far from me, which makes steady shooting hard. You get jumpy, jerky footage (as you can see in the Us at the Zoo video I put together from this weekend); zooming with a lens is great for photos, but crappy for videos.

In part, this is my choice of tool. The DSLR is a fine camera now that I’ve learned to use it, but it’s not clear that it’s the best tool for shooting video, but it’s what I’ve got for the moment. But even ignoring that, I’ve got the bigger problem: Shooting candid, entertaining video is an entirely different ballgame.

I’m enjoying learning new tricks, and producing slightly more compelling footage and editing. I’m enjoying this new medium; I’ve been watching lots of other people’s videos, to learn how to do it right — or at least better. (My favorite candid family gathering video is this Easter video that someone posted in one of the quadcopter forums due to its use of a quad for a couple of the shots. This has actually been a primary motivator for me to get out there and see what I can do.) So far what I’m learning more than anything else is “This is hard work!”

Flying Machines within Yosemite

Posted in default on May 3rd, 2014 at 08:40:39

The NPS released an announcement today related to drone use within Yosemite. It claims that a statement that ‘“delivering or retrieving a person or object by parachute, helicopter, or other airborne means, except in emergencies involving public safety or serious property loss, or pursuant to the terms and conditions of a permit” is illegal.’ applies to drone use.

Drone flying over a waterfall in a wooded area

Now, I personally find that sentiment a bit odd — my quadcopter isn’t ‘delivering or retrieving [an] object’, as far as I know; I would be hard-pressed to consider that phrasing to apply to me if I were reading it. But even more importantly, reading the CFR that is referenced, there is a much more obvious section that the FAA would probably claim applies: 36 CFR 2.17 (a)(1) says: “Operating or using aircraft on lands or waters other than at locations designated pursuant to special regulations.”

Given that the NPS has instead chosen to go with section 3, this seems like a position of the NPS/DOI that the FAA’s position on drones/remote controlled model aircraft being treated as ‘aircraft’ is just wrong. However, weaseling this into another regulation (with the dubious claim that the ‘object being delivered is the drone itself’, as they claimed on their Facebook page), is almost as bad of an approach. It at least has the benefit that it doesn’t affect flying outside the parks, so I find it less personally concerning for long-term survival of the hobby.

They already have a prohibition (36 CFR 2.12 (a)(1)) which prevents ‘operating motorized equipment … [t]hat exceeds a noise level of 60 decibels measured on the A-weighted scale at 50 feet; or … makes noise which is unreasonable, considering … [the] purpose for which the area was established [and] impact on park users…’ I think this is an entirely reasonable regulation, but the noise level from the quad almost certainly doesn’t meet the “60 decibels at 50 feet”, so only the “bothering other people” aspect would apply — and therefore make it reasonable to operate in an area where there are no other people around, or where people won’t be bothered by the noise (such as where I was on the Presidio, where no one could even hear the quad over the wind).

I think this has three big take-aways for me:

  1. The people who wrote this document did not feel that drones are aircraft. This runs counter to the FAA position, but is good, because there is no legal defense for calling these things aircraft. This means that there continues to be support for the legal notion that there are no current regulations which apply.
  2. The NPS has a desire to block the use of drones, and will do whatever they can to do so. This means that regardless of what the law says, you should not expect to fly in a National Park and not get told you can’t.
  3. Given the other regulations prohibiting the use of ‘motorized toys’, I expect the NPS will change their regs to explicitly ban drones in the future, at which point you will not have a legal leg to stand on — so if you want to pick a fight, you might as well do it now.

I continue to be supportive of reasonable regulation regarding drone use in the national airspace, and continue to be supportive of the NPS/DOI passing reasonable regulations that apply to model aircraft. I dislike the use of wordsmithing to try and fit these things into existing regulations, and I think that the NPS should work hard to make their regulations legally match their claimed authority.

Flying — or not — within National Parks

Posted in default on April 24th, 2014 at 21:26:38

The United States National Park system is a terrific system designed to protect and preserve some of the most beautiful parts of this country. Ever since I was a kid, I’ve loved the NPS; from rangers putting together brilliant education programs to the beautiful sights protected in the untamed wilds that National Parks serve to protect, I have a strong place in my heart for this part of the federal government. (I’ve often joked that I deal with my taxes primarily by deciding that my money is clearly only going to fund the parts of government I like — and the National Parks are the most common example I give of where that money would go if I could choose.)

The Presidio of San Francisco, a large area in the northwest end of San Francisco, is one of these National Parks. The park is a former military base, and is now a home to cultural and natural pleasures, and includes some great open spaces with great views of the Golden Gate Bridge. Unfortunately, these spaces are the types of things that you are prohibited from seeing from the air, because the National Parks Service has strict rules against the flying of UASes within a park.

The NPS hosts information on Unmanned Aerial Systems in the aviation section of their website talking about the benefits of these aircraft. The website references “Operational Procedures Memorandum (OPM) 09-11″ (with a link that no longer works) as the rules that govern UAS flight within National Parks. Some searching suggests that the replacement for this is OPM 11-11: available from the DOI library (though that seems to have expired Dec 2013 with no replacement I can find).

The summary is: All UASes are aircraft, and the FAA controls aircraft, and the rules are you can’t fly them (without following strict rules that most RC multi-rotor pilots aren’t following). No mention is made in the document of exceptions for model aircraft, unlike UAS Guidance 08-01, the most recent FAA guidelines, which state that hobbyists should seek guidance from “Advisory Circular (AC) 91-57, Model Aircraft Operating Standards”.

Now, normally I’d just chalk this up to oversight: This document must be written targeted at public agencies wanting to fly UAS within NPS airspace, not something that people are actively enforcing. Unfortunately, evidence to the contrary seems strong. After flying around in the Presidio, I was walking back to my car carrying the quadcopter when a ranger stopped me, and let me know that if he saw me flying it around, he’d have to kick me out of the park. This type of comment from NPS rangers is not something unique to me: I have talked to coworkers who fly, and have also been asked to leave National Park spaces when flying RC aircraft as a hobbyist.

It’s frustrating to see the National Parks Service take an approach which is so narrow towards hobbyists for reasons that aren’t clearly stated anywhere I can find. (Perhaps the belief is that RC planes would spoil things for everyone else. If so, I understand that, but I wish that it was stated publicly somewhere. I would still be frustrated, but I would at least understand.) As it is, a policy which seems designed and targeted towards heavy duty governmental use of UASes is being applied to hobbyists, and I find that frustrating and sad.

Prior to knowing this was an issue, I flew a bit in Criss Field in the Presidio, and what felt like a great view at the time. It makes me sad to think that images like the ones I captured are the types of things that the NPS wants to prevent.
Golden Gate Bridge as seen from Criss Field

Bitcoin: Just as Broken as Cash

Posted in default on February 27th, 2014 at 08:44:45

According to a Business insider article, Senator Joe Manchin III is suggesting that lawmakers pursue a ban on Bitcoin. He wrote a very cogent letter on the topic, included in the linked article. In the spirit of his letter, I would like to share my own letter on a related, but slightly different topic.

Dear Senator Manchin,

I write today to express my concerns about the US Dollar. This currency has allowed users to participate in illicit activity, while also being highly unstable and disruptive to the world economy. For the reasons outlined below, I urge regulators to take appropriate action to limit the abilities of this highly unstable currency.

By way of background, the US Dollar is a currency that has gained notoriety due to its varying exchange value and relation to illegal transactions. Each US Dollar is backed by the US Government, thus the US Dollar is not only a token of value but also a method for transferring that value. It also means that this currency allows for anonymous and irreversible transactions.

The very features that make the US Dollar attractive to some also attract criminals who are able to disguise their actions from law enforcement. Due to cash’s anonymity, the market has been extremely susceptible to hackers and scam artists stealing millions from US Dollar users. Anonymity combined with cash’s ability to finalize transactions quickly, makes it very difficult, if not impossible, to reverse fraudulent transactions.

Cash has also become a haven for individuals to buy black market items. Individuals are able to anonymously purchase items such as drugs and weapons illegally.

That is why more than a handful of countries, and their banking systems, have cautioned against the use of the US Dollar. Indeed, there have been moves to ban the US Dollar in several different countries— most recently Russia, where lawmakers are predicting a complete collapse of the currency by 2017. Several other countries, including many in Africa, have issued legal threats over the use of the US Dollar. While it is disappointing that the world leader and epicenter of the banking industry will only follow suit instead of making policy, it is high time that the United States heed our allies’ warnings. I am most concerned that as the US Dollar is inevitably banned in other countries, Americans will be left holding the bag on a valueless currency.

Our foreign counterparts have already understood the wide range of problems even with the US Dollar’s legitimate uses - from its significant price fluctuations to its inflationary nature. Just two years ago, the US Dollar’s prices plunged after the currency’s major holders in the form of banks, experienced major financial issues. This was not a unique event; news of plummeting or skyrocketing US Dollar prices is almost an annual occurrence. In addition, its inflationary trends ensure that only speculators, will benefit most from possessing the currency. There is no doubt average American consumers stand to lose by transacting in US Dollars.

The clear ends of US Dollars for either transacting in illegal goods and services or speculative gambling make me weary of its use. Before the U.S. gets too far behind the curve on this important topic, I urge the regulators to work together, act quickly, and prohibit this dangerous currency from harming hard-working Americans.

Sincerely,

Christopher Schmidt
United States Citizen

Watching someone grow up on TV

Posted in default on November 26th, 2013 at 07:33:07

I watch reality TV.

One of the shows I watch is Gold Rush; I’ve talked about it before.

On Gold Rush one of the people is Parker Schnabel. Parker’s been on the show in part since Season 1; he’s an enthusiastic kid who started out helping his grandfather mine (and taught the rookie Hoffman crew a thing or two in the first season), and has since taken over his grandfather’s mine, and then moved onto the Yukon to mine his own gold claim.

I love watching Parker — not for the usual reason, which is that I enjoy feeling superior (yes, I will admit that watching people be idiots on TV gives me a small sense of superiority, even if I know most of it is scripted drama that isn’t real) — but because I’ve been able to watch him grow up in a lot of ways.

Parker has basically lived his life around the TV show for the past 3 years, and in that time, he’s gone from being a helper to running a mine under his grandfather to running his own — and he’s grown up. He’s grown from being a bit of an overeager snot — to actually being someone who is taking on responsibility and able to be the boss. He’s grown from someone who is putting all of his responsibility on others, to someone who feels very clearly the responsibility is on him, and wants to make it work.

Maybe I see some of myself in Parker; he’s the kid who moves 3 times faster and works 5 times harder than those around him, and still feels like he’s only breaking even; he is smart, cocky, and doesn’t tolerate fools well. He’s all in on a risky venture without proof that it will work, and without any idea what he’s getting into most of the time.

And when shit goes to pot, or he needs help, he’s still got parents who care about him and help him out; even if his dad is strict, he’s still got Parker’s best interests at heart, and they clearly want the best for him.

In the end, I guess I like watching Parker is a little bit like how I think of myself. Even if it’s stupid, mostly scripted, overdramatized reality TV, I like Parker, and I like to watch him, because he’s how I think of myself.

Coursera: Discovering I’m Still Bad at School

Posted in default on November 23rd, 2013 at 21:40:32

I’ve taken a handful of Coursera courses.

Or really, I should say, I’ve tried to take a handful of Coursera courses.

As with all other formal (or in this case, semi-formal) educational opportunities, I always start off super-committed. “I will do the homework early. I will watch all the lectures. I will take the quizzes without reskimming the lectures and use my copious notes.”

As always, it fails. I fall behind; I miss a week; I don’t start my homework until two hours before it’s due.

In a recent course, I was supposed to write a 2000 word data analysis paper. I pulled it up about two hours before the due date — without having watched most of the videos, or done anything else for that week.

After about 1.5 hours of work, I ran `wc` on my analysis … and discovered that I had what I thought were 2700 words, despite feeling like I was way under my limit. I spent the next 20 minutes trimming things out, and getting it down to just 2000 words… and then pasted into the online text editor, which reported that I had actually written 300 words. I looked back at my `wc` output and found that I had actually been looking at the number of *characters* — so I had written a 2000 *letter* report instead of a 2000 word report.

In the last 9 minutes before the deadline, I drastically tried to bring back some of the text I just deleted, and add some more things I had been meaning to add. In the end, I think I did very poorly on the assignment, but I have no one to blame but myself.

It’s the same as it was in college and high school; I’m actually semi-decent at some of the tasks, but sitting down, getting organized, and actually *doing the work* is always the problem.

Ah well. At least I’m learning some new things.

Joel Test for my current project

Posted in default on November 20th, 2013 at 07:50:14
  • Do you use source control? - yes
  • Can you make a build in one step? - yes
  • Do you make daily builds? - yes
  • Do you have a bug database? - yes
  • Do you fix bugs before writing new code? - yes
  • Do you have an up-to-date schedule? - no
  • Do you have a spec? - no
  • Do programmers have quiet working conditions? - no
  • Do you use the best tools money can buy? - yes-ish
  • Do you have testers? - Not convinced ‘tester’ applies in the sense that is meant by the Joel Test, but we have a QA team; most test code is written by programmers though.
  • Do new candidates write code during their interview? - yes
  • Do you do hallway usability testing? - no, but we don’t create a UI.

Going over this, I do have a feeling that the Joel test is directed primarily at things that users see and touch; with that not being the case, the projects I work on lack some of what is deemed as important, specifically because “hallway usability testing” doesn’t seem to apply the same way and ‘testers’ can’t really help as much as I think might otherwise apply.

Improving the World

Posted in default on November 17th, 2013 at 22:14:19

I like to improve the world.

If I described my day job to most people, they would probably say I’m not improving the world very much. I work on improving the local search product of a commercial entity; the work I do doesn’t directly contribute to solving hunger, or saving lives, or what have you. I still feel like I am improving the world though, even if it’s in a smaller way.

Nokia phones are used by millions of people around the world every day. We get millions of users using our local search product every day. I can use the numbers we have to put a percentage on how many search queries are successful every day, and track that over the past couple years since our team started working on the problem.

What is a successful search query worth? Well, on a mobile phone, a bit more than you might think. In rapid-fire testing — attempting to run search queries as quickly as possible, with good knowledge of what I’m looking for and a solid understanding of the search experience — a single search query might take as little as 20 seconds. Reviewing some logs in the past and looking at typeahead, we have seen that for users on some devices, simply typing a query may take some users upwards of a minute.

If I can make search .1% better, with 1 million daily users, I’ve just saved 1000 users 30 seconds or so — or about 8 hours of productivity has been created that might not otherwise exist, assuming that the amount of productivity lost is equal only to the time to do a new search.

In the search team at Nokia in the past year, we have made much more significant gains than this; in fact, given our usage and our improvements in the past two years, the amount of productivity we save each day via improved search alone is more than 3 times the total working hours per day *of our entire team*. That’s right, with the improvements we have made in search, all the time we have spent getting to this point will be paid off in increased productivity in the world in 1/3rd the time we have invested into it; and we continue to make improvements at an approximately linear rate (without, so far, drastically growing the size of our team at a supra-linear rate).

While increased productivity isn’t the same as solving world hunger, or even more mundane acts of saving the world, it is a little bit nice to work on a product where the work we do is a net positive on human productivity. It’s certainly not going to save in the world, but it does help improve it.