Technical Ramblings

So, a couple people have asked me why Google thinks that my site may harm your computer.

On Wednesday, I received an email from “Google Search Quality”:

We recently discovered that some of your pages can cause users to be infected with malicious software. We have begun showing a warning page to users who visit these pages by clicking a search result on Google.com.

I looked through, and sure enough, the links they offered were indeed ‘infected’: I’ve always used Wordpress despite my knowledge of the fact that it has security exploits more often than I get around to fixing them (though I do try to keep up). They had has a small iframe included, which claimed to be ’stats tracking’: Instead, there was Javascript included which, presumably, was malicious.

To the best of my knowledge, I solved this problem on Wednesday night, by removing the mal-links that were pointed out, and patching the security holes I could find fixes to in Wordpress. (I just upgraded to Wordpress 2.5; In the past, upgrading has been painful, but it wasn’t so bad this time, and there are not yet any known security holes for 2.5 that I’m aware of.)

All in all, not a bad thing: Google emailed me, I fixed the problem, everyone wins. Except…

Following Google’s FAQs, I went to Webmaster Tools, signed up, verified my site, went to their tools…

And in the site management tools, found no such link as they described. Great.

At the time, I assumed only Google was using stopbadware: I’ve since discovered that other things are using it, so I’ve requested reconsideration there.

Still, Google now tells users that my site may be dangerous, despite the fact that it no longer is, and there appears to be no tool in the website management ‘tools’ panel to have them check it out again. Stretches the definition of ‘Do No Evil’ a bit… (Edit: Okay, not really, but it always works when you really want to get a response out of Google to just tell them they’re being evil: People get defensive and help you out ;))

In any case, my web site should be safe. Sorry that people have been confused by the problem.

Edit: JohnMu in comments pointed out why I was having a problem: Since crschmidt.net/blog/ was the only thing listed as ‘infected’, I had to sign up and verify for crschmidt.net/blog/ *seperately* from crschmidt.net. Certainly not exactly intuitive, but doing so allowed me to request a review of my site, so hopefully soon people will be able to view my site again in FF3, and won’t be caught out by Google’s warning (assuming I got all my malware off).

It’s my Birthday!

Mapping related gifts I’ve picked up so far:

• Mapping Boston, which is gigantic and heavy and beautiful.
• A Wonder Globe, which is small and spinny and will go on my desk at work
• A National Geo map of Germany, for hanging on the walls at work
• Transit Maps of the World, which a friend who is staying with us picked up from off my recently revamped Amazon Wishlist

Not a bad haul, since I’ve only been 24 for 18 minutes

I’m in DC on Wednesday morning for a meeting: Flying in tonight, arriving around 6, staying at the Holiday Inn in Reston. If there’s anyone who I should be stopping over and having a beer with tonight, let me know: at the moment, my plan is just to wander around DC monuments for a couple hours tomorrow night, since I Haven’t been able to find anyone I should be meeting. (I’m booked until my plane leaves on Wednesday; the trip is not about seeing DC, just a quick hop down for work.)

OSGeo Users Group meeting in Boston tomorrow (well, it’s already tomorrow, but you know what I mean) night, 7pm at MIT Museum in Cambridge.

When: Wednesday, Oct 17th, 7PM
Where: MIT Museum, Downstairs, “MIT360″ space,
265 Massachusetts Ave., Cambridge, MA
http://web.mit.edu/museum/
Boston Freemap
What: First Meeting of Boston OSGeo users group: intro, and maybe review of FOSS4G?
Who: Anyone interested in open source geo software
Why: Because it’s there!

I figure we’ll spend an hour or two getting to know people, talking about FOSS4G (for those of us who went… that might just be me), etc.

If you plan to be there, please respond to the list so people know to expect you! (A ‘maybe’ is fine.)

My first attempt to do something useful with IronPython:

>>> import urllib
>>> urllib.urlopen("http://crschmidt.net/")
Traceback (most recent call last):
  File httplib, line unknown, in getreply
  File httplib, line unknown, in getresponse
  File httplib, line unknown, in __init__
  File System, line unknown, in set_ReceiveBufferSize
  File System, line unknown, in SetSocketOption
WindowsError: Invalid arguments

Note that I’m working on OS X, and my exception is a WindowsError. Fancy.

(I was inspired by Bill Thorp’s efforts to get TileCache working on IronPython: Round 1, Round 2. However, I’m not all that inspired now.)

Still, it is kind of cool that IronPython just ran — I didn’t expect that to work. Maybe there’s something to this mono business after all.

At the Google Developer Day. Sitting in the Google Gears session: it’s pretty frickin cool. I just created an OpenLayers Map, wrote 15 lines of code, and that page will now load, even when I’m online.

Other things it does:

• WorkerPool — run code in the background in your browser. Demoing finding prime quadruplets: user interface continues to be responsive — can run multiples at the same time, and user experience is not bad. (Oh man, how I wish this were  built in by default.)
• Storage — local storage in a sqlite database, including full text search: millions of documents, working on fts3, which will be 10s of millions.
• Local server — caches data for offline use. 

There are tons of limitations — very early release — but it’s available for Firefox and IE, and can be built for Safari.

Made a ’screencast’ (at least, I think that’s the right buzzword) of OpenLayers + QGIS today:

Mapping Your Data: Using QGIS, OpenLayers, and MapServer to serve open data

Tried uploading it to YouTube: the resolution was shrunk to the point of being unsable. Tried uploading to Google Video: despite the upload completing 11 hours ago, the upload is still ‘processing’. So I went through with ffmpeg and converted it to Flash myself, and set up the environment for playing it on openlayers.org.

To create the video itself, I used:

• Snapz Pro X (Shareware: $60)
• ffmpegx (Shareware: $15)
• Flash Video Player (Free for Non-commercial use: $15 commercial license)

In the video, I used TextMate (Shareware), QGIS (Open Source + Free), Firefox, and OpenLayers.

(There is no audio in the presentation, because I felt like my voice would be a distraction. I suppose next time I should put some fancy soundtrack in place.)

At BarCampBoston today, I got to talking to the OLPC folks. I was demoing our new vector editing support in OpenLayers, and talking to them about it. They wanted to see if it would run on their laptop, and we got an error that SVG wasn’t supported in their browser. After a bit of effort after getting home tonight, I was able to find out how to check for SVG 1.0 instead of 1.1 — after modifying the OpenLayers code to do a 1.0 check instead of a 1.1 check, I got a confirmation that vector display works on the One Laptop Per Child laptop! Woot. Filed a ticket with a patch for OpenLayers: 540, just need to get someone to review it and check it in.
So, the next step is to help the OLPC folks get up and running with OpenLayers.

So, last week was an OpenLayers hack week. One of the things that we did was make adding support for new vector formats trivial. Instead of modifying several parts of the code, you only need to create two functions: a ‘read’, which takes a set of data — XML, strings, Javascript object, what have you — and returns a list of OpenLayers.Feature.Vector objects, and a ‘write’ which does the reverse — takes a list of objects and returns a string or object or XML.

To prove this, I set out to write some additional vector format support last night. I decided to add one read, and one write.

• Read: KML. I added support for KML point display in about 20 minutes, including the time to find data and write a demo HTML page loading some example data. Adding LineString support was another 15 minutes.
• Write: GeoRSS. Support for writing georss:simple points, lines, and polygons was simple… once I found data. I asked for a live example, and was unfortunately unable to find any valid line data outside the GeoRSS website, so I just generated something that was as close as I could come to the examples. I’m lazy, so the export is just RSS 2, and I’m sure that someone will come along and criticize it, but that’s one of the benefits of Open Source: Anyone can offer up a patch. Time from when I created the file stub to when I committed the code was 27 minutes, again, including a demo.

Altogether, the Format support in the new OpenLayers is pretty cool. Because of the way it’s built, I can even do something that is pretty damn ridiculous: Import KML, and export GeoRSS (or GML), all from the browser. Certainly, this is an incredibly crazy thing to do, but OpenLayers is a pretty crazy project.

I’m convinced that there’s nothing in the code that would make it difficult for someone who’s comfortable working with Javascript to write support for any simple-to-parse format. Now, to get the code back to trunk and get the patches rolling in.

Ever since I had some problems with my server and Crucial Paradigm, I’ve been having too much spam in my inbox. I thought I had the exact same spamassasin config, but either I missed something, or the settings have just gotten less effective.

This morning, I finally got fed up, and decided it was time to do something about the fact that my inbox was about 90% junk mail, even after filtering out anything with an SA score about 5.

I’ve had a Gmail account since around August 2004, but I never use it. However, I do know that Google has better spam prevention than I do: something about having hundreds or thousands of humans acting as bayesian filters helps them. (Mm, scale.) After having recently implement Akismet for spam checking on the MetaCarta Labs weblog — with the result being that about 98% of the 3000 spams it gets every week are now gettin caught — I decided it was time to do something about my inbox spam.

So, I set up crschmidt@crschmidt.net to act as a forward to Gmail, and then set up my fetchmail to fetch from Gmail instead of from localhost.

Results so far are extremely positive:  Of the 85 emails that made it past my local spamassasin filter and into Gmail, 75 of them were spam. Gmail caught all but two. So, instead of 85 messages in my inbox, 10 real, I ended up with 12, and 10 of them were real.

If this stays true, my personal email address may actually become a reasonable way to contact me again. Sorting through the 200 spam I get every day and finding the wheat was becoming an almost impossible task, and this may have at least given me a leg up temporarily.