Joel Test for my current project

Posted in default on November 20th, 2013 at 07:50:14
  • Do you use source control? - yes
  • Can you make a build in one step? - yes
  • Do you make daily builds? - yes
  • Do you have a bug database? - yes
  • Do you fix bugs before writing new code? - yes
  • Do you have an up-to-date schedule? - no
  • Do you have a spec? - no
  • Do programmers have quiet working conditions? - no
  • Do you use the best tools money can buy? - yes-ish
  • Do you have testers? - Not convinced ‘tester’ applies in the sense that is meant by the Joel Test, but we have a QA team; most test code is written by programmers though.
  • Do new candidates write code during their interview? - yes
  • Do you do hallway usability testing? - no, but we don’t create a UI.

Going over this, I do have a feeling that the Joel test is directed primarily at things that users see and touch; with that not being the case, the projects I work on lack some of what is deemed as important, specifically because “hallway usability testing” doesn’t seem to apply the same way and ‘testers’ can’t really help as much as I think might otherwise apply.

Improving the World

Posted in default on November 17th, 2013 at 22:14:19

I like to improve the world.

If I described my day job to most people, they would probably say I’m not improving the world very much. I work on improving the local search product of a commercial entity; the work I do doesn’t directly contribute to solving hunger, or saving lives, or what have you. I still feel like I am improving the world though, even if it’s in a smaller way.

Nokia phones are used by millions of people around the world every day. We get millions of users using our local search product every day. I can use the numbers we have to put a percentage on how many search queries are successful every day, and track that over the past couple years since our team started working on the problem.

What is a successful search query worth? Well, on a mobile phone, a bit more than you might think. In rapid-fire testing — attempting to run search queries as quickly as possible, with good knowledge of what I’m looking for and a solid understanding of the search experience — a single search query might take as little as 20 seconds. Reviewing some logs in the past and looking at typeahead, we have seen that for users on some devices, simply typing a query may take some users upwards of a minute.

If I can make search .1% better, with 1 million daily users, I’ve just saved 1000 users 30 seconds or so — or about 8 hours of productivity has been created that might not otherwise exist, assuming that the amount of productivity lost is equal only to the time to do a new search.

In the search team at Nokia in the past year, we have made much more significant gains than this; in fact, given our usage and our improvements in the past two years, the amount of productivity we save each day via improved search alone is more than 3 times the total working hours per day *of our entire team*. That’s right, with the improvements we have made in search, all the time we have spent getting to this point will be paid off in increased productivity in the world in 1/3rd the time we have invested into it; and we continue to make improvements at an approximately linear rate (without, so far, drastically growing the size of our team at a supra-linear rate).

While increased productivity isn’t the same as solving world hunger, or even more mundane acts of saving the world, it is a little bit nice to work on a product where the work we do is a net positive on human productivity. It’s certainly not going to save in the world, but it does help improve it.

Hidden Writing Archives

Posted in Web Publishing on November 2nd, 2013 at 23:48:18

So, recently I realized that my website essentially hasn’t been updated since 2007. (I noticed last week that I had an announcement on there about the presentation I gave at FOSS4G… in 2006. As a current event. Yeah, my website isn’t particularly up to date.)

So, I’m working on moving bits and pieces of it to git, so I can track history and changes more easily, and also because I think that having it open presents no risk and provides opportunity for me to more easily track what I’m doing there in public. However, since the website has been essentially a dumping ground for my various crap for the past 8 years, I’m being a bit cautious about it, and moving things in one by one. (There is some content in my webdir which should not be publicly available — everything from passwords to old client content which is currently protected by password.)

In the process, I came across a collection of articles that … I might have written.

I say might, because I have no real memory of writing them. However, they have examples which use my name; they are written in a style which is semi-consistent with how i would write, and most importantly, they’re hosted under my formal/writing directory (which no one else has ever had access to), which is also explicitly prevented from being crawled by robots.txt. The modification dates on these files are in November 2005 (all on the same day; which likely means they were copied en masse from something even older).

They’re all of things that were of interest to me at the time — how to integrate RSS into IRC bots, how to parse mood information from LiveJournal RSS feeds, how to parse the LiveJournal live stream, etc.

The thing is… I have no idea why I have them. Or why I hid them.

At the time, I was working on getting contracting work — this was before I had started working for Ning, during a time when I was grasping for contracting work to make ends meet, having ended my commute back and forth to wedü. I guess this means that I was actually (essentially) unemployed for a brief period there… perhaps I thought writing articles like this was a good way to draw attention to myself. But then my question becomes… why did I hide them from my robots.txt?!

My biggest fear is that this content is actually stolen from somewhere else — the topics are general enough that it could be. However, given the specifics of the topics, I think that they can’t all be.

While I would believe that someone else could write some of those, it would surprise me that anyone else wrote all of those, and put them in one place. So I think this *is* writing I created… but I have no idea why. With no historical version control, I can’t refer to that (curse you, younger, more foolish self!), and at this point I’ve switched computers enough times that there isn’t much left in the way of detrious of conversation logs from that time; even my email records only go back to 2006 consistently. I think this was actually content that was probably created before transitioning my website to a hosted webserver; this was from the days when it just ran off the server at the apartment.

I’ve googled titles and snippets of these; I see no other evidence of them on the web. I think this means these probably are my original content. Given that, I think I’m going to go with: share and enjoy! I’ve put these into the github repo under formal/writing, where you’re free to do with them what you will. I may try to tidy them up… heck, I may even try to write a few more, because I actually enjoy the writing style.

I just… find it weird that I have this content that the internet at large doesn’t have access to which I apparently wrote 8 years ago and have absolutely no memory of… what a weird thing to find.

(If you have any memory of me writing these articles, why, or evidence I’ve shared them in the past, I’d love to hear it!)

Gold Rush: A metaphor for programming

Posted in default on October 28th, 2013 at 06:28:00

For the past several years, I have been a semi-religious watcher of Gold Rush (originally Gold Rush: Alaska).

Gold Rush started in 2009, during some of the worst part of the economic downturn. A bunch of down-on-their-luck guys from Oregon decide to lease a gold mine in Alaska, because they can’t afford to make their house payments, pay bills, etc. and figure that a get rich quick scheme is the best way to fix it is to do something entirely new that they’ve got absolutely no experience with, because they hear that’s the way to make good money.

Getting together 6 guys who work in various roles at home — from Real Estate agent to sheet metal fabricator — they head north in a convoy, bringing their entire families with them. They plan it perfectly, so that if everything goes right, they’ll have a whole hour to spare to get all the equipment they need to bring north onto the barge that heads to Alaska (otherwise, they’ll need to wait another week). Plenty of time! Until something goes wrong. Like the trailer with your bulldozer on it popping a tire.

This bunch of guys get to Alaska, and for the first time, they put together their gold-catching plant. Or try. One guy decides he’s going to go off and build a house for himself. One guy is too busy digging random holes all over the property. One guy has so little experience with power tools that he cuts right through the power cable to the tool he’s currently using. And they’re all living surrounded by, amazingly enough, bears.

Once their morphine-using mechanic finally gets their machine put together, it’s not catching any gold. They decide that the right solution to that is to put more dirt in. (Ignoring the fact that in order to obtain gold, you need *dirt with gold*.)

Of course, you’ve got other experienced miners around, who are very clear: The thing you do to make money is drill, find where there is gold, dig there, and so on. But these guys? They believe that so long as you work hard enough, you’ll get rich. All you have to do is *really care*, and the money will fall into your hands.

Over the last 3 seasons, the team has grown — they’ve actually become somewhat competent as a team, and in their latest season were pretty successful. But the first season (and to some extent the second season) were so much a monumental example of the pitfalls of daily life working with some programming teams that it’s impossible to ignore the likeness. Some team deciding that they’re going to do the thing that will work quickly. Ignoring past experience and advice, and believing passion will fill in the gaps. Failing at execution again, and again, and again.

I’m happy to work with a team that is good at avoiding these traps. A team that knows that you don’t start big — you start small, whenever you can. You start with proven technology, not whatever’s the new shiny thing (the $250k ’super trommel’ in Season 3 is a brilliant example of the latter); you start by consulting with experts, not pulling them in when your project has already failed; but most importantly, you start by thinking about the problem, and trying to predict what will go wrong, and doing your best to prevent it. These are important things for any team anywhere to recognize, and I think is a key aspect of the difference between success and failure for many teams.

Stay close to what you know. Feel free to take risks, but do them knowing they are risks, and don’t treat them as a sure thing. And above all else: Think.

User research talk at BarCampBoston

Posted in default on October 26th, 2013 at 13:50:14

Notes from user research talk at BarCampBoston.

4 types of testing
 - (Cheapest one first)
   Card sorting
   How do users like to categorize? Which terms do users understand, not
   understand, which terms they relate to other terms?

   simplecardsort.com -- 

 - Questionaires

 - Focus Groups
   - Good for finding out what people like
   - Bad at what people don't like
   - nobody wants to be the bad guy
   - You need bad news! They weon't tell you that in a group setting
     because it's 'rude'
   - One or two dominant speakers who talk over everyone else, everyone
     else is marginalized, narrow spectrum, wasting a lot of people's
     time.

 - one-on-one interviews
   - what does the person know
   - what does the person do
     - narrow
     - get them to estimate how much of what they do takes what amount of
       time -- daily, weekly, monthly
     - Figure out the things people are spending the most time on, and
       concentrate on *that*, not on the things that they spend less time
       on.
   - what is the user's feedback
     - What do you like?
     - What do you *not* like?
     - General feedback
     - People

LinkedIn Recruiters

Posted in default on October 26th, 2013 at 07:23:30

I just did my bi-monthly (as in once every two months, not twice a month; stupid English) cleanout of LinkedIn spam. I am surprised to learn that many recruiters no longer send InMails; instead, they just give you their pitch in a “Connect” request. Is InMail too expensive now? Or are they just trying to scam me?

meetings of the day

Posted in default on September 26th, 2013 at 06:23:22

Every morning, I check my meeting schedule. As a member of an R&D team with staff in a big company, I have many (too many) meetings. (In an average week, I spend about 30% of my time in meetings — some weeks, as high as 50%.)

Today, my thoughts are:

  • Ugh. an All Hands call. Wait. Is it really “All” hands when it only affects your team, which composes less than 1% of the organization?
  • Meeting Location: “A door-less, broken room in Berlin” — yeah, that sounds about right.

Things I might do if I did fun things

Posted in default on August 4th, 2013 at 22:10:36

Make a map of my trip to North Carolina, including route information and foursquare checkins; using the trip log I recorded, make a graph of average speed over various spans. Highlight all the points along the trip where something went terribly wrong.

Create a game out of taking photos of POIs — sort of like foursquare, but for photos. Come up with an algorithm to suggest what places should get photos based on number of times they show up in things like search results, or are clicked, and which ones don’t have photos. (I think this is probably what that Google thing is, but I don’t really know.)

Try to build a tool which can decide what place a particular photo is attached to via text recognition or by associated metadata on a photo sharing site like flickr. Start with geocoded photos; see if it’s possible to extend somehow to non-geocoded photos (though I can’t think of anything obvious).

Build a dotmap-style map of every point address that NAVTEQ has in its database, possibly adding in POIs or something. Also, see if it’s possible to rewrite the dot map as something that looks as good, but can be dynamically generated for lower zoom levels.

Using the streetview-style imagery that Nokia has, build a tool which would let users pick a ‘good’ view for POI, using a suggestion starting from a geocoded coordinate. This would probably involve building a slippymap based viewer for Nokia’s streetview imagery, because as far as I know, there’s no dynamic scriptable viewer for that data.

These are just a few of the things that I’ve thought of over the past month or so, but the motivation to do any of them is always pretty low. Maybe I should get a new hobby. Or weed the garden instead.

Why I no longer do fun things with Maps

Posted in default on August 4th, 2013 at 11:18:51

Ed note: Since there has been some concern, I think I should preface this post with this: I think that my employer has a very solid business position and a solid line and business plan around what we *actually do*; I think there’s serious misunderstanding internally about what we do though, which leads to a lot of this. I also don’t see this as an indictment of the team I work for or what I actually do; my day job is productive and adding usefully to our platform. It just isn’t ‘fun’ in the same way as hacking up a demo in a weekend is.

A long time ago, I used to do fun things with maps.

I would make a map of Mario levels. Or build a draggable routing system. Or work on an editable map display on top of Google Maps.

I would plot out my route, or put my photos on a map, or other fun things.

But now, I don’t do those things anymore. I don’t make cool toys, or experiment with maps, or do anything particularly interesting with maps at all. Observant observers would probably (rightly) observe that this started approximately when I started working for my current employer — but probably not for the reasons that most would think (or at least, not the reason I’d have expected when I started working there).

Frankly, working with the rest of the stuff in the world is just depressing when you have to go back to working inside of a giant corporation when the weekend is over. I see experiments like I used to do as having two options: following the company line, or really pushing the envelope and doing something cool in the world.

Neither is as exciting, because frankly, doing something cool — but not using our corporate tools — means that I can’t share it with the people I spend the rest of my week with. “Why are you using Google Maps?” is the canonical question for almost all things — even when its clear that the answer is “Because we doesn’t provide a competitive offering.”

Using our offerings to do cool things is like pulling teeth. In the space of API availability, design, and support, we still living years in the past, and there is no major shift towards the future; as an enterprise-targeted company, our interests really don’t match that of the consumer market, but there’s a lack of acceptance of that, which leads to people thinking that we *should* be competitive.

I feel differently: I feel that we’re a successful enterprise company, and we should capitalize on that. We have an offering which no one can really compete with in that space — Google’s current enterprise story is poor, and their worldwide story is less good than ours — but it’s hard to convince people we shouldn’t try to compete with Google at everything we do.

In either case, I find working with maps a million times more frustrating now than I used to. The idea of making a cool demo is completely tamped down by the knowledge that using OpenStreetMap as a base map — even when it obviously presents a significantly improved experience — is considered disloyal.

In the end, it’s easier just to not bother. While I really enjoy what I do as my day job — to the extent that I spend a lot of off-hours work on it — it’s not a *cool* thing; my job is to make a small portion of the maps experience a tiny percent better. (To me, this is actually pretty exciting — every time we make a problem 1% better, we probably make thousands of people’s lives slightly easier every day, something I couldn’t possibly have said before.) But There’s very little of external interest in what I do on a day to day basis. And the things that I used to do which *were* interesting to others, I no longer do, because I feel stymied by the environment I work in.

It’s also why I seldom am aware of the latest news in the geospatial world. It’s just frustrating; to work for a company which has significant resources, but constantly feel like I’m watching us miss opportunities so that we aren’t competitive; to constantly see questions go by about “why can’t we do X?” when the answer is “Because we have massively misunderstood that market, and don’t want to try to compete.”

Working for a big company is also frustrating in and of itself. As we work on problems, then hand them over to new people whenever one group fails, there is no *learning* process. I watch as we make the same mistakes over and over — and I’m sure that others would say the same about the work that I do, though I feel like we’re at least making progress — and don’t learn from them. I watch as people seriously misunderstand how far ahead in some areas the rest of the geographic data space is, and learn the wrong lessons — and still think that we’re competing in the consumer space, when in reality, we should capitalize on our *successes* instead.

The lesson of OpenStreetMap is not “You should attempt to build your own collaborative mapping platform” — OSM has taken 9 years to build what they have, and there is no practical way that anybody else will be seriously competitive in that space.

The lesson of MapBox is not “People like prettier maps, so we’re going to make our maps be the ultimate in pretty.” The lesson is that people want *different* maps — and they want the process of creating them to be easy, all the way down to adding new data and details to it.

The lesson of the success of the Google Maps API — and later charging for it — is not “Mapping APIs are a valuable distinguishing factor in map data use, and we should charge an arm and a leg for access to the data.” People want data in their own applications, and if you provide a sufficiently compelling service, you may be able to get people to pay for it — so long as they can use it the way they need to.

Watching the company I make for make so many mistakes, to some extent, I just bury my head in the sand. I no longer do fun things with maps because every fun thing will be questioned, and I never liked the questioning part; I just wanted to build something cool.

I loved working at MetaCarta because I had a cool platform by which to create and have some people see the work I’d made. I had a group of coworkers who were interested in similar things, and would comment on them, and I worked in an environment where “anything goes” was the watchword. I always had the support of the company in spending time building a cool demo that used MetaCarta infrastructure to produce something was new and exciting.

I miss that about working for MetaCarta. I miss working in an environment which really was “Anything goes”; I miss working in a place where creativity was rewarded with praise instead of complaint; I miss working in a place where it was accepted that the best tool for the job isn’t always the tool that we have built.

That lack of insight from the rest of the company I work for is disappointing, and leaves me wishing that I had a bit of that magic back. Since I don’t: I don’t do fun things with maps anymore, because it just isn’t as much fun.

Indoor Mapping

Posted in default on April 21st, 2013 at 15:42:44

I’ll admit it: I’m obsessed with map data. Not the maps themselves — not how they look, or anything else about them — but the data, the bits that make up the way we find our way in the world.

This wasn’t always the case. In the past, I didn’t care much about the data — I just wanted pretty pictures so I could show things. But as I’ve changed from being a map maker to working on the search side of ovi Nokia HERE Maps, I’ve moved away from caring how the map looks, to caring what is underneath.

Every time I walk into a place, my brain immediately tries to think about how I’d map it. Is the Central Cafe in the middle of Union Station a single entity, or is the upstairs eating area a second floor — even though the structure is standalone? What is the right way to represent the curving staircases up to the second level of shops in the main station?

Every time I walk into a complex place like this, I just want to spend a week mapping out every detail. Where are the stores? What is in them? Can we attach a frontage photo of each? How would you represent the Godiva chocolate — do you mark the path through it as a public hallway, since it’s used that way, or as part of the store?

I want every complex building in the world to have a fully annotated set of data about it, not so I can look at it, but so that I can be routed along it.

It’s not that the technology and approach to do this don’t exist: the data behind things like Bing’s Venue Maps (http://binged.it/ZGAGfT), sold through Nokia as Destination Maps, are typically completely covered for routing. Public spaces are demarcated. Entrances are noted. Every piece of info that you could want is there. But these maps exist for so few places, and they’re so poorly integrated with the rest of the mapping experience.

I’m tired of wandering around for 20 minutes looking for the luggage lockers. Of not knowing where the restrooms are. Of being trapped in the maze that is the International Spy Museum, looking for the way out. (By the way: International Spy Museum — awesome place, definitely worth the price of admission. Great mix of gadgets and pop culture, artifacts and pop culture, plus a huge exhibit on James Bond.) I love these places, but I hate not knowing where I am!

“More than 4230 venues”, says Bing. Well, great, but it appears that you have less than 10 places in Washington DC. No Union Station, no Air and Space Museum, no American History Museum. No National Archives, no White House.

These are the easy places. Every one of these places has a map — most of them produced by the Smithsonian, and if they’re not public domain, they probably would be perfectly happy to help publish the data more. There are 18 Smithsonian Museums in DC alone, and pretty much every one of them has an interior map they hand you when you walk in the door.

I know that this isn’t something that will happen top down, not realistically. This is a case for something like OpenStreetMap, for a crowd-sourced approach. Because I’m tired of “4230 venue maps!” I want them for every strip mall, for every department store, for every place where I’ve ever had to follow a sign to restrooms, asked for directions, and gotten lost anyway.

Until we have that — and until I’m carrying it in my pocket — I’m never going to be able to stop thinking “Damn, I wish that I could sit in here for a week and make a damn map.”