Getting around the five icon limit in the Google Static Maps API

Our application uses the Google Maps API, including the static maps API. We use custom icons for our markers to help make the maps better match our brand. In the live maps this isn't an issue and we can have as many custom markers as we like. But Google limits you to 5 custom icons in static maps, even with a business license. Most of our icons look roughly the same, they just have different numbers in them to function as our labels, so we quickly exceed that 5 icon limitation.

My methods for getting around this quickly evolved.

I started by not even knowing the limitation, by test case only had 5 icons, so it all seemed fine. In reality as soon as my users would have more than 5 my first 5 were custom and everything after that was the standard Google Map inverted teardrop marker with a dot in it. Not cool as it totally broke our branded look, and made the map look inconsistent.

Once the problem was realized I put a switch in. If 5 or less, use our branded icons, 6 or more, use all teardrops, with the label set to be number I need. This was better, as it was consistent at least, however; the map did not look great when framed with the style of our app when there were more than 5 icons. Furthermore the label property of the teardrop markers is limited to 1 character, so as soon as we had 10 icons it was back to having teardrops with dots in them, which are pretty useless for us.

So now I came up with the solution that probably attracted you to this post. I'm going to try to give it to you at a high level so that you can make it work with your language of preference, but any example are going to be in ColdFusion since that's what we use.

1. You probably already know how, and depending on your source it's going to be different anyway, but collect up all your map data. Required bits are going to be: center point, zoom, map type, and output image size. I am going to assume sensor (if the application has access to GPS) is false. Also you are going to need all of your marker information which will include the icon you are going to use, and the geo coordinates of them.
2. I POSTed this all to the CF page that is going to make all the magic happen.
3. Map your first 5 points as normal. Get the results as a .png, ex1 below
4. Map your next 5 points but add "style=feature:all|visibility:off" to the query string, get result as a .png. This will give you a png with a transparent background but will have all of your marker icons on it. It will be the same size as your initial map, and the markers will be placed correctly withing that rectangle. ex2 below.
5. Watermark that image on top of your initial map. ex3 below, NOTE: this step is probably going to vary the most depending on your language of choice and what image manipulation features it offers.
6. Repeat 4 and 5 until you have all of your markers.
7. Write out you image with all of the markers now on it. ex4 below.

ex 1:

<cfhttp url="http://maps.googleapis.com/maps/api/staticmap" method="get" path="#PathWhereIWillStoreTempMapImages#" file="#TempMapImageFileName1#.png" getasbinary="yes">
 <cfhttpparam type="url" name="size" value="800x800">
 <cfhttpparam type="url" name="format" value="png">
 <cfhttpparam type="url" name="key" value="#MyGoogleAPIKey#">
 <cfhttpparam type="url" name="center" value="38.897701,-77.036527">
 <cfhttpparam type="url" name="zoom" value="15">
 <cfhttpparam type="url" name="maptype" value="roadmap">
 <cfhttpparam type="url" name="sensor" value="false">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon1.png|38.886839,-77.004743">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon2.png|38.903001,-77.018238">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon3.png|38.705634,-77.035854">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon4.png|38.886839,-77.004743">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon5.png|38.884658,-77.023684">
</cfhttp>

ex2:

<cfhttp url="http://maps.googleapis.com/maps/api/staticmap" method="get" path="#PathWhereIWillStoreTempMapImages#" file="#TempMapImageFileName2#.png" getasbinary="yes">
 <cfhttpparam type="url" name="size" value="800x800">
 <cfhttpparam type="url" name="format" value="png">
 <cfhttpparam type="url" name="key" value="#MyGoogleAPIKey#">
 <cfhttpparam type="url" name="center" value="38.897701,-77.036527">
 <cfhttpparam type="url" name="zoom" value="15">
 <cfhttpparam type="url" name="maptype" value="roadmap">
 <cfhttpparam type="url" name="sensor" value="false">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon6.png|39.023456,-77.054743">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon7.png|38.963001,-77.048238">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon8.png|38.755634,-77.055854">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon9.png|38.826839,-77.044743">
 <cfhttpparam type="url" name="markers" value="icon:http://www.mywebsite.com/icon10.png|38.814658,-77.033684">
 <cfhttpparam type="url" name="maptype" value="roadmap">
</cfhttp>

ex3:

<cfimage source="#PathWhereIWillStoreTempMapImages#/#TempMapImageFileName1#.png" name="layer_1">
<cfimage source="#PathWhereIWillStoreTempMapImages#/#TempMapImageFileName2#.png" name="layer_2">
<cfset ImagePaste(layer_0,layer_this,0,0)>

ex4:

<cfimage action="write" overwrite="yes" source="#layer_1#" destination="#PathWhereIWillStoreTempMapImages#/#TempMapImageFileName1#.png">

I reviewed the license and TOS for the static maps API, and I can't find anything about how this might be a violation. If it somehow is, please let me know and I will cease using this method and remove this post.

Windows Subversion Server Hook Scripts in Batch

We run all windows servers over here and that includes the server I get to run Subversion on. This is a less than ideal situation as even the best Windows Subversion servers are still locked down to http instead of the native svn protocol. It's not as bad as it sounds though, svn1.7 is actually really fast, even over http and even with a stupidly large repository. The problem I've really come up against though is hook scripts. There is literally one example hook script for Windows, and all it really does is force comments. Valuable, yes, but this is not all we can do. If you've found this page I'm sure you've already seen it, it goes like this:

@echo off          
setlocal
set REPOS=%1  
set TXN=%2

svnlook log %REPOS% -t %TXN% | findstr ...... > nul  
if %errorlevel% gtr 0 (goto err) else exit 0  

:err  
echo ---------------------------------------------- 1>&2   
echo Please add a log message to your commit.       1>&2  
echo ---------------------------------------------- 1>&2 
exit 1

In fact this actually checks for a commit message of at least six characters using the bastardized version of regex in the findstr utility. Batch is archaic, and I have to admit I am a complete novice with it, but it doesn't work like any language I've ever learned. Where batch really breaks for me is I want my pre-commit hook to not only force comments, but block anyone but me from committing to the trunk. So I use svnlook changed to see what's being changed. it looks something like this:

A   branch/vendors/deli/
A   branch/vendors/deli/chips.txt
D   branch/trunk/test.txt
A   trunk/vendors/deli/sandwich.txt
A   trunk/vendors/deli/pickle.txt
U   trunk/vendors/baker/bagel.txt
AU  trunk/vendors/baker/croissant.txt
UU  trunk/vendors/baker/pretzel.txt
D   trunk/vendors/baker/baguette.txt

The way I figure it, if I see any character in the first two positions, then two spaces, then the word 'trunk' it should be blocked but findstr can either do it's horrible idea of regex, or match a string literal, and there is no escape sequence to match spaces in findstr. What I need is some real regex power. I tried GNU's grep but found that that either I am inept (which is likely) or that regex doesn't work like one would expect. I got to ^..\s\s[at][nr][tu] to match up to the 'tru' in trunk, but could not figure out how to match 'n' and honestly why [at] to match 't'? I give up on that, then I discovered a more powerful tool GNU's sed With this I was able to write a simple, easy to follow pattern that worked, and worked as I expected: ^..\s\strunk. I could now figure out when someone was attempting to commit to the trunk, and with the gotos in batch able to skip this check if it was me. The hook I ultimately came up with looks like this, with added comments to help out the other beginners out there:

@echo off  
::turns off output other than what you echo, ie, doesn't show any
::actual commands you have running
setlocal
::in my reading this is by default, but just in case, prevents 
::variables in this script and any scripts that call this script
::encapsulated from each other  
set REPOS=%1  
::the first argument svn sends to the hoot is the repo path, you
::reference arguments in the order they are received with the %#
::syntax
set TXN=%2
::the second is the transaction, if we were in post-commit this 
::would be the revision
svnlook author -t %TXN% %REPOS% | findstr michael > nul
::svnlook has a lot of subcommands author should just give me 
::the username of whos doing the commit. findstr works okay for 
::this basic task and the pipe (|) makes the svnlook output the 
::input for the findstr command
if %errorlevel% equ 0 (goto commentCheck)
::this is hacky, but the only way I could find to do thing. I 
::attempted to put any line matching 'michael' into the nul 
::handle, if there were no lines found an error code of 1 is 
::generated if it succeeds then no error code is thrown (0), and 
::I use the goto command to skip to check if there is a commit 
::message
svnlook changed -t %TXN% %REPOS% | sed -n /^..\s\strunk/p > temptest.txt
::I use my actual regex here with the sed command and since we
::can't have variables in batch like we are used to, cache the
::output in a file
for /f %%i in ("temptest.txt") do set size=%%~zi
::read in the size of that file (0 means no matches)
if %size% gtr 0 ( goto authErr ) else ( goto commentCheck )
::if trunk is found the file has a size, goto takes me to give a
::not authorized message, otherwise check the comment
exit 1
::sanity check that will block the commit if something wasn't 
::accounted for.
:commentCheck
:: this is a lable and what goto commands are looking for
svnlook log %REPOS% -t %TXN% | findstr ...... > nul  
if %errorlevel% gtr 0 (goto err) else exit 0 
::again findstr is good enough, just checking for 6 characters of
::message, if there are at least 6 then exit with no error (0) 
::which allows the commit
:err  
echo ---------------------------------------------- 1>&2   
echo Please add a log message to your commit.       1>&2  
echo ---------------------------------------------- 1>&2  
exit 1
::the basic message and exit code that will block the commit 
::displaying whats in the &2 handle
:authErr
echo ---------------------------------------------- 1>&2
echo not authorized to make commits to trunk.       1>&2
echo ---------------------------------------------- 1>&2  
exit 1
::same thing,m already discovered you didn't have access and 
::attempted to commit to the trunk, you get this message and your
::commit is blocked.

A quick note on the grep and sed install: even if you use their installer they will not put their bin path in the PATH environment variable, so the this script will fail unless you add it yourself. Just look up some instructions on how to modify PATH in your version of Windows, and add the path to the bin folder wherever you put it when you ran the grep and/or sed installer.

CFDocument ignoring some classes

My co-worker came across an interesting bug the other day. A class he was adding was being ignored by CFDocument. If he just output the content as HTML, there was no problem. It was only when inside CFDocument tags that the class would get ignored. We played around with it, a bit and discovered changing the order of the classes in the class attribute seemed to fix the problem. So with a workaround available we went on our merry way, but it was really bothering me and I couldn't let it go. I decided to make as simple of an example as possible and post a question on Stack Overflow but I took me awhile to even figure out how to duplicate the problem. In the process of duplicating the problem I stumbled upon exactly what the bug was.

The problem occurs when you have two class names applied to the same element, and the second class name matches the right end of the first class name case insensitively. Obviously some bad parsing happening inside the CFDocument engine.

Examples:

    <[element] class="firstClass class">Content</[element]>
    Or
    <[element] class="firstClass Class">Content</[element]>
    Or even
    <[element] class="firstClass stclass">Content</[element]>

Would produce the error, but:

    <[element] class="firstClass class2">Content</[element]>
    Or
    <[element] class="firstClass secondClass">Content</[element]>
    Or
    <[element] class="firstClass Classs">Content</[element]>

Would not. Since there is a workaround Adobe won't bother fixing this, nor would they bother publicizing this in any sort of documentation, just leave it up to their users to somehow know all the quirks of their application. I hope publishing this made your day easier.

A few things that are driving me away from ColdFusion are:

• Adobe's disinterest in fixing long standing bugs, and if there is a work-around, even if it involves their users needing to stand on their head and type with their toes, then it might as well be labeled "Won't Fix Ever"
• No documentation about what work-arounds exist for known issues, except on their bug report pages, which are a bitch to navigate in any useful way.
• The constant movement of their documentation, including live doc pages and bug reports. It's so frustrating to find a forum with a link that says, "Here's the work around" with a link to a page at Adobe that no longer exists. Couldn't you at least leave us a re-direct Adobe?
• This list could go on and on, but would mostly be re-hashes of the first point. Adobe, please fix broken things before adding more new features that less than 5% of your users care about. 

CFPDF musings

Since I've been heavily modifying PDFs lately I've discovered that CFPDF has got to be the most under-documented tag in Coldfusion. Shame really, you would think a company that does so much with PDFs would have the most top notch documentation for the manipulation functions they include in their own server application.

The good news is that it seems everything we would want to do calling in java objects and using the iText library can be accomplished with this tag, the bad news is that there is very little help as to how.

So here are some things I've learned lately that may help y'all out.

1. If you have a process that may generate multiple PDFs and you want to join them together with the merge action, there seems to be a bug that requires you to specify the pages argument if there is only one pdf in your source list. The live docs say that pages is optional, but the server throws an error if it's not included in this instance.
2. I guess this is standard when dealing with print, but it makes no sense to me. When you use the watermark action your origin is actually the bottom left instead of the top left. So when it comes to positioning things, you have to measure the height of the watermark, and calculate how much to adjust the y axis in the negative. By negative I mean that you give it a positive number to move the water mark upwards, so the opposite of how you might set a div with position:absolute
3. When you use the getinfo action it will give you a height and width, but in points instead of pixels, so be ready to do some .75 and 1.25 conversions to properly deal with pixels to points and back. And since I just mentioned watermark positioning, you position in pixels, even though the getinfo is in points, so keep that in mind when you are attempting to line things up. Sub point to this, if you have a landscape page it will still say it's 8.5x11 inches instead of 11x8.5, but also that it has a rotation of 90, so remember to take that into account when determining what your height and width actually are.
4. While cfdocument does great converting HTML to PDF, there isn't really a simple way to window one PDF inside another generated with your HTML code, but using the watermark action does a pretty good job of layering one PDF into another, so you can generate your outer PDF then use watermark to layer your inner PDF in that border. You can do this with multi-page PDFs by generating your border, then looping through the PDF you want surrounded by that border, then use the watermark feature iterating through the pages. At the end you merge them all together and you have a nifty new PDF windowed in your custom border. I also used the transform action to scale down the inner PDF to fit within my border so my output was still an 8.5x11 inch page.

That's all I have come up with so far, I'll add more as I come across them.

SVN Error Pristine text not found

Doing some work today I started having a little trouble with my working copy. For reference we use visualSVNServer over an HTTP connection on our local network, and our client tool is SmartSVN. I tried to commit some code and got an error that SVN was busy. Not really sure what that meant, I waited a second and tried again, now it tells me:

Commit failed: Pristine text not found

Again, not sure what that means, lets try to update, same error. OK, obviously something borked in my working copy, clean-up usually fixes that. Again: "Pristine text not found" What does google have to say about it? Not a whole lot, just that other people have seen that error, and no real clues on how to fix it.

Since we upgraded to subversion 1.7 I have noticed that periodically SmartSVN tells me I should validate my admin area. When it's done this in the past I've noticed status messages saying something about pristine somethings or other, so I run the validate admin area command, and low and behold we seem to be getting somewhere. I get a warning that my pristine copy isn't correct and that SVN is going to try and fix it, but if it doesn't I should do a fresh checkout. I let it do it's thing and it makes a few notes that it's removing some items from the pristine table, and warns me that it installed some new pristine files and even downloads a copy of one of my files from the repo to restore my pristine copy.

Interestingly the file it gets is a revision number higher than my current working copy. What's this? It appears that somehow my commit succeeded to the repo, but my client didn't seem to realize it. So in the end, simple client error resulted in a working copy that was out of sync, and validating the admin area was able to clear it up. I'm now able to update my working copy and commit code with no problems again.

Dreamweaver not closing cfquery

We recently upgraded to Coldfusion 9 here at the office, and most of us are using Dreamweaver CS3 as our text editor of choice for CFML, which very has a nice little upgrade that will make code complete work for the new tags and the changes to the existing ones. If you didn't know about it, you can download the patch here.

After I applied the patch I started noticing that the code complete would not auto close a cfquery tag, and would in fact, auto close the parent tag. This was highly annoying behavior that easily goes unnoticed during rapid fire coding and results in instantaneous bugs. I have Dreamweaver set to auto close a tag as soon as I type </ which made this even more annoying as a quick attempt to properly close the cfquery tag, would just result in another wrong close tag being inserted. So I would have to carefully highlight everything after </cf and type in the rest. Fatal bug? No, but after three or four times, annoying enough to need immediate change.

Of course it's not obvious what the bug is, so I have to sort through a lot of results thrown back from the search engine before finally finding a forum post from someone going by the name 'Krakajap' who figured it out. He got me pointed in the right direction, but his solution was a little excessive and I figured out a simpler way to do it.

The problem is a typo in the config files for Dreamweaver. You can find all the xml files that tell Dreamweaver what to do with tags at approximately:
C:\Users\[you]\AppData\Roaming\Adobe\Dreamweaver 9\Configuration\TagLibraries\cfml
or wherever in your OS Adobe sticks these folders. You will see a htm and vtm file for all your cfml tags. You want to edit the vtm for cfquery. You'll see the problem right away on line two. Just change:

<tag endtag="no" name="CFQUERY">

to:

<tag endtag="yes" name="CFQUERY">

Restart Dreamweaver and away you go.

Coldfusion 9 Fails to send mail when using cfMailParam to set the content type to text/html

We recently upgraded to ColdFusion 9 and have had our share of migration pains. Today I discovered a new one. We have a newletter that goes out on a fairly irregular basis and today was the first one since the migration. Hours after it was scheduled to be released I got notified that no-one has received it yet. So I check the undelivered mail and yup, there are almost 4k messages that ColdFusion refused to send to the mail server.

As Sherlock would say, the game is afoot. First, let's rule out the most likely suspects. Looking at the mail log, I see errors for them all, but the error message is blank. I check the mail server connection, it's fine. I try dropping a few back into the spool, but they just get kicked back to undeliverable. I check the SVN logs and see that no code changes have been made to the file involved. Great, not going to be an quick and easy bug hunt.

Interestingly, while looking at the undeliverables, I was still able to see messages hitting the spool and processing normally. Thinking to myself, "what makes them different" I manage to open one before it processes out and start to compare it to the ones that won't. Fortunately the difference was on line one: the ones that won't sent say "type: text/html" the ones that will say "type: text/html; charset=UTF-8". So I change that line on one of the failures, send it back over to the spool, and it processes normally. Great! Fix it in all of them, send the batch over to the spool and we are in business.

But why did this happen?

I pull up the code for the mail that went through fine, and for the one that didn't and look at the cfMail tag. and here is the big difference:

Good:

  <cfMail [typical settings] type="html">
    Mail Message
  </cfmail>

Bad:

  <cfMail [typical settings] [no type attribute]>
    <cfmailparam name="content-type" value="text/html">
    Mail Message
  </cfmail>

The bad code here worked fine in CF7, but just results in undeliverable mail in CF9. I can't tell you why, but hopefully my investigations help save someone out there some time.