Useful ArcIMS tools

In my travels there have been a few nifty tools i have picked up along the way to make my life in ArcIMS a little bit easier.

ArcXML DTD. Is extremely useful for validating those painful AXL files. I dont know how often I have left a field open, or missed a quotation mark. This will validate the structure and report any errors found. Very useful for those doing a lot of editing of AXL’s by hand. I recommend XMLStar to do the DTD and general well-formedness checking, but there are lots of others out there.

Shapefile UPPERCASE.exe. Arcims has an uncanny habit of dropping or simply not adding services with shapefile layers having a combination of lower and upper case field names. This program will open the DBF file and automatically convert them to all uppercase. Sure beats doing it manually.

PHP Logwatch. This is a multipurpose PHP script that monitors any number of log files running on your system and outputs that last X lines (eg. 50). It is particularly useful for monitoring the 400 arcims log file locations and any other system log of any importance (eg. IIS/Apache logs). Oh, and it also has error syntax highlighting. The script detects the latest log file in a given directory, making it even more relevant for ArcIMS where it creates a new log at every restart (not neccessarily each day). The script can be monitored from a remote location with only the PHP script needing direct access to the log files.

Webtail. Webtail is similar to PHP Logwatch in that it tails log files, however the main difference is that Webtail is able to tail any log file visible on the web so it does not need to reside on the same machine. Extremely useful if your sysadmins have given you read access only to a web readable directory, but you dont actually have access to the machine.
Example: Read access only to a website with the logs (eg. http://log.com/arcimslogs/). All we need to do is call webtail via “java -jar WebTail.jar http://log.com/arcimslogs/AppServer_1123623189000_0.log 10″ and at 10 second intervals, it will look for any updates to the file and output the difference. It only needs to retrieve the entire file once, and so lessening the impact of downloading a large log file every 10 seconds. HIGHLY RECOMMENDED

MXD to AXL converter. If you search at ESRI you’ll hear that there is no way to convert between the two, but there is, and there has been since 2003. Jeroen Ticheler’s converter on Arcscripts has been helping me for well over the year doing the arduous task of converting MXD’s to AXL’s. It isnt going to solve all our problems (as you will still need to manually do some of it), but it sure as hell speeds up the process.

Arcexplorer installed on the same ArcIMS machine. Nothing annoys me more than the conflicting installs of ArcIMS / ArcMap and ArcExplorer. “Sorry, we detected X on your system, install cannot proceed.” There is thank god, an easy way to get round this and this is to just use ArcExplorer version 4 (unfortunately i dont have a link handy). Version 9 will throw the error, but v4 runs happily on the same machine. This workaround has saved me 20 times over when we have needed to debug our production ArcIMS server and this was the easiest way to grab a service. Very useful when there is no other access to the machine.

Thats all that comes to mind at the moment, but im sure theres a heap more.

Please post a comment if you have something else to recommend. Anything to make administering the nightmare that is ArcIMS, a little less painful :)

Its google … it must be right.

The following article has been superceded and the issue has been resolved. There is no issue (as far as i am aware) with the state of Google Earth’s data.
After comparing some common overlays from both Mapserver and Arcims (using the PHP code i posted earlier), it was obvious something was up,

Projection descrepencies

  • either my data being projected was wrong
  • or googles base data was out

Upon closer inspection of the cadastre and road overlays, the general accuracy was roughly 35m to the south east.

Projection descrepencies

Makes you wonder doesnt it (yoohoo, metadata would be lovely guys). I have double checked the projection using both Arcims and Mapserver output and it appears at the same level. A collegue in Darwin has had no problems overlaying his own data … has anyone else noticed this? Or is the Perth Digitalglobe data just poorly processed?

Of course, it could just be me :)

Nevertheless, the sooner people stop asking me ‘HAVE YOU SEEN GOOGLE EARTH@!#!$?’, my life would be just *that* much better.

Im sure many other GIS’ers agree that it has limited use or flexibility in the marketplace and goes against pretty much all the open standards that have been put in place over the years.

Yes, you can fly through the grand canyon
Yes, you can “view” anything on the globe
Yes, i now deem myself as GIS professional extraordinare because i can the read the API and draw my name using JS. HEHE
No, i dont know how old, or what accuracy this image is .. who cares! I can see Pam Andersons house!
What do you mean projection? I just use what google gives me, google is always right

If you sense angst, probably because their is. It has its place guys (ie. visualisation), but the sooner everyone realises this the happier i will be :)

Clarification on adding WMS to Google Earth

Ive noticed a few other blogs mention the need for WMS server operators to run my PHP code i posted earlier. This is untrue, so to clarify … read on.

1. If you havent already done so, grab the code here

2. This code will require at least 2 small changes for it to work for you … first off, go to lines 26 to 35. This block contains 2 arrays, $WMS (the full getmap WMS request to the server) and $name (a pretty server/layer descriptor).

3. Each array entry corresponds to a new layer inside google earth. Obviously, depending on how you want to display your map, you may wish to only have 1 layer. I will assume this is what most will like to do, so remove all the other array entries so you are only left with a single layer. eg

[0] = "Cadastre";  $WMS[0] = "http://link.to/your/wms?ServiceName=earth&VERSION=1.1.0&REQUEST=GetMap&SRS=EPSG:4326&WIDTH=".$width."&HEIGHT=".$height."&LAYERS=Land Parcels&STYLES=&BGCOLOR=0xCEFFFF&FORMAT=image/gif&TRANSPARENT=FALSE&WRAPDATELINE=TRUE";  ?>

4. Now, i will assume you understand how to submit a WMS getmap request. Many of the public WMS servers already have documentation on how to do this. The syntax should be similar to the above, but REMEMBER to set the EPSG code to 4326 (WGS84) otherwise your data will not fit googles projection. If the WMS server has the option, its always a good idea to set the background to transparent also.

5. Now assuming you have added your first WMS entry into the PHP code, all you have to do is modify the IP range which you are allowing to access your WMS-KML reflector script. Go to line 46

if ($match[1] != '151.130' && $match[1] != '202.72' && $match[1] != '115.205') {  ?>

Edit the partial IP addresses to reflect that of your own organisation / home computer. I have only done the pregmatch on the first 2 numbers, but feel free to modify it on line 42 if you wish to do a full ip comparison.

6. Ok, time to try adding it to google earth ..

7. Load it up and click Add -> Network Link. Enter an appropriate title for the entry and in the ‘location’ textbox, enter the full URL path to your PHP code eg. ‘http://url/kml.php’. In the dropdown for image request, change it to ‘When camera stops’ and ensure the value is set to 4 seconds.

8. Zoom to an area of interest (one which is covered by your WMS) and stop the camera from rotating. Wait for 4 seconds and a red ‘Loading..’ image should be shown over your extents. This is the standard image google earth places when it is attempting to retrieve the network link (ie. your WMS image). If all goes well, you should have your image overlaid very nicely. Otherwise, if you recieve a nasty error popup check that your WMS server request was entered correctly.

Thats pretty much all there is to it. You dont need to run your own WMS server, you simply need to point to an available public server. Of course, running your own you get added benefits of being able to completely customise the look and feel … but if you’re feeling really lucky, try playing around with some SLD documents :)

As always, any Q’s flick me an email or post a comment. Hope i have cleared any confused.

DMSolutions done it again

Paul Spencer at DMSolutions has posted their latest demonstration of KA-Map which uses AJAX-esque technology to transform mapserver output into a tiled format similar to maps@google but with a lot more features.

Check out the demo site here. It is without a doubt the best demo i have seen showcasing Ka-Map and mapserver’s new WMS support for a time attribute. Excellent stuff … just make sure you press play !

ps. In a somewhat related line, anyone know of any projects working on an opensource routing solution? Let me know