Entries in code (10)


Mapping APIs and web services

A month or so ago, a friend of mine approached me about building a web site for him that would show routes and other information on a map and allow edits and so on and so forth.  I've played with the Google Maps API before but I wanted to have a look at using Open Street Maps instead because I really like using open source software when I can.

A little digging and I actually came across the MapQuest Open Data Map APIs and Web Services.  I signed up for an API key and played around with all the web services like geocoding and static map services before moving on to the JavaScripts Maps.  There is a bunch of sample code, and it's super easy to get up and running so I pretty quickly had a full featured map with controls and geolocation running.  That's when I remembered that I had created an XML file with all (actually as many as I could find), the ski resorts in North America.  This would be a perfect exercise to work with.  After browsing the developer guide, I found some sample code that loads a remote kml file using the MQA.RemoteCollection function.

I got this code up and running pretty quickly, but I wanted to round it out by using my own custom POI marker.  I found two Creative Commons (and GPL) licensed icons and mashed them togther to create the following:

I'm no graphic artist, but it'll do... :)

Here is where I got the two icons from: 

GPS marker

Inkscape icon

This is when I ran into problems.  It seems there is a bug when loading KML that prevents custom icons from loading correctly.  Instead I was only getting the icon shadows rendered:

I debugged the issue for quite some time, but I started seeing that a bunch of functions were not returning data or hadn't been implemented.  As a side note, the built in debugger in Chrome is pretty awesome.  I wouldn't go so far as to say it makes writing javascript painless, but it goes a long way to right direction.

After taking to the forums, I started to find that the Open Data API is pretty rough around the edges.  It works wonderfully for what it is, but there seems to be a lot of bugs still.  Either way, I'm going to keep using it, as this is really the only bug I've personally run into so far.

To get around this issue, I guess you'd need to parse out the KML yourself and manually add the POIs with a custom icon.  I did this for a few of the placemarks in the KML file and it worked great.

Anyway, have a look at the map with the default markers in place.  You can click on the markers to get info that was provided in the KML file.  I have a huge amount of data in the file, but I'm still missing a lot of info about resorts outside of the Western US as that's where I started gathering info:

 Full Screen 

Also, because I wanted to see the map with my custom icon and I already have a gmaps api key, I went and ahead and put this together.  You can click on the markers for info:

Full Screen

Lastly, I went ahead and added the ski resorts kml file I created from my xml data to the github repo where I keep this stuff.  Feel free to use the KML file or the above maps for your site or whatever you want really.  Some day I'd like to get around to adding all the stats for the rest of the ski resorts.

-- Rob


Data redaction

As an SE, I work closely with customers during evals and provide assistance building out a proof of concept during the eval.  I always advise the customer to build a POC with real data rather than trying to make up representative data.  Although there's many reasons why you'd want to use real data during the POC phase, the choice is not often that of the customer's as there may be other over arching rules/regs that need to be adhered to.  For my customers, it's often that there is sensitive data on forms or documents that can't be shared outside of the organisation.

Of course the simple answer is to redact any sensitive information on the forms, when/if you need to share the forms with me or my team.  I say this often to customers, but then they ask me "ok, well how do I redact the information?".  Well, I don't have a good answer to that, and I just ask them to use some sort of 3rd party tool.  This need to have full control over the eval process caused me to build an in house redaction tool.  I call it "Redacto 5000".... :)


Redacto 5k is a WPF/C# app that uses the Parascript FormXtra SDK to load a form definition and then redact all or selected data from a batch of images.  I've also added the ability to grow or shrink the size of the redaction fields.  There are a few other things in the works like being able to draw on custom redaction zones and the ability to move all the zones at once in the X and/or Y direction.  

Although I have experience programming with WPF/XAML, I did have to spend a little time getting up to speed with all the neat and sometimes confusing data binding principles.  Once you get the hang of how bindings works and the syntax in XAML, it's actually not bad, but I did find that it seemed a bit non-obvious at first.  A good place to start if you're new to the concept is the MSDN:






Android and OpenCV

I've been working on an Android app for work, and some ideas where thrown around regarding placing camera guides real time over an object.  The idea being that the guides would automatically adjust to the size of the image we were working with.  I wasn't totally sold on the idea, but I went ahead and investigated it any way.  

Some time ago, I used OpenCV in conjunction with QT to build a webcam app that would do face/person/dog detection, so I had an idea that I could probably use OpenCV in some form or another for the edge detection (I figure detect the edges, and you could place the guides at the corners where the edges intersect).  I was pretty happy to see that a quick google search pointed me to the Android port of OpenCV:


Not only is there a port, but it comes with a pretty cool sample app that has a ton of examples already built.  Literally, all I had to do was modify the sample app and I'd have a pretty good proof of concept.  I'm always amazed at the amount of free information and tools available to people these days.

After reading the OpenCV docs for a bit, I found some information on feature detection, and specifically a function called HoughLines, which finds lines in an image:


A little more digging, and I found a tutorial on how to use the function:


So my work would only consist of writing the Java code from the C++ example code.  I'm probably smart enough to do that... :)

Anyway, I ended up with the following Java code (this code relies on the OpenCV sample app):

          Imgproc.Canny(mRgba, mIntermediateMat, 80, 100);

            Imgproc.cvtColor(mIntermediateMat, mRgbaInnerWindow, Imgproc.COLOR_GRAY2BGRA, 4);
            Mat lines = new Mat();
            Imgproc.HoughLines(mIntermediateMat, lines, 1, Math.PI/180, 150);
            double[] data;
            double rho, theta;
            Point pt1 = new Point();
            Point pt2 = new Point();
            double a, b;
            double x0, y0;
            for (int i = 0; i < lines.cols(); i++)
                data = lines.get(0, i);
                rho = data[0];
                theta = data[1];
                a = Math.cos(theta);
                b = Math.sin(theta);
                x0 = a*rho;
                y0 = b*rho;
                pt1.x = Math.round(x0 + 1000*(-b));
                pt1.y = Math.round(y0 + 1000*a);
                pt2.x = Math.round(x0 - 1000*(-b));
                pt2.y = Math.round(y0 - 1000 *a);
                Core.line(mRgba, pt1, pt2, mColorsRGB[1], 3);


Here's a screen capture from my GS3 while doing real time edge detection:


 Edge detection on Android using OpenCV


In the end, the code didn't perform well enough for my likings, and I think there's still a lot of (non trivial ) work to be done to make this work in non ideal conditions (like where the background is not in stark contrast to the paper for example).  Non the less, I learned some stuff from the exercise, and who knows, we may do a version that works on the static, captured image for cropping or something.





If you're like me, you have a Nuvi 500/550.  If you're like me further still, you've tried to take a track file that someone created on one of their rides, load it on to your Nuvi and then follow the track only to realize that the Nuvi doesn't handle track files.  "No sweat, you probably told yourself, I'll just convert this to a route and then load it on...."....WTF, the Nuvi can only handle routes with up to 200ish points in it? Damn this track file has like 5000 points...

What I've been doing with tracks that I create on Google maps is breaking them into separate files by hand and then using gpsbabel to convert them to routes and then simplify them.  This works great, and I just load 3 or 4 route files for a given track and follow them.  It works surprisingly well, but it takes some time to get the route files created.  Because I got tired of doing it by hand, today I decided to write a perl script to do it.  I call it nuvinator.pl.

All you do is feed the script your full blown track file, and optionally give it a number of routes to be created, and it will make route files each with 200 route points (that's as many as I could get my Nuvi to load).  I just wrote it today, and have only tried it on a few track files, but her is the output from a 7k point track file converted to two routes:





Here are the requirements to run the file:

1. Perl

2. gpsbabel

3. The perl script relies on XML::Simple and Getopt::Long, so you may need to install them.  On Linux it's just sudo cpan XML::Simple Getop::Long.  If you're using Activestate on Windows, I think it has these installed, but if not, check out the command line utility ppm.


You can download the script here: nuvinator.pl

I've only run the script on Windows, but it should run just fine on Linux/Mac.  If you don't have gpsbabel in your path, make sure to edit the script and set the $gpsbabel to the path of your gpsbabel executable.


I hope this helps a lot of people, because it makes it dead simple to convert tracks to Nuvi ready routes.




Windows Binaries for QT Canon

A while back I compiled some Windows binaries for my QT Canon application.  I did a couple of videos with it, but I sold my DSLR before I got to play too much with it.  Until I get a new camera SLR, the QT Canon project is on hold (mainly because I don't have a camera to test with).  At any rate, I figured I'd put the binaries so that others could play around with it.  There are bugs in the app, and this is beta at best, but feel free to download and make some cool time lapse videos.  All you have to do is download the zip file and then extract it some where.  Then plug your camera in, turn it on, and then fire up the application.  On Linux, the app will find your camera even if you start the app before you plug the camera in.  This is not the case on Windows though.  You must plug in your camera, turn it on, and then start the app.  From there you just set up a few things and start shooting.  Have fun!

Download for Windows (XP/Vista/7): QT Canon

I recommend turning off the auto focus when shooting time lapse.  Otherwise the camera has a tendency to "hunt for focus" between shots, and this can produce a jerky video when stitched back together.  Kind of like this one I made with my Canon Rebel: