Chris' GISmo's

The tilecache goldrush

Chris Tweedie — Tue, 01 Jun 2010 05:22:13 +0000

Does anyone else not see a problem with the trend over the past few years? “Tile-itis” is reaching critical mass and it is driving me bonkers. We’re taking away styling, reprojection, tile sizes and giving them … tiles. No wait, fast tiles? Really? Oh, so I can put them on Google Maps? Awesome. Can I have them in projection X? No, sorry, we don’t have another terabyte to reseed the cache. Can I have just the streets? No sorry, same problem.

Why do I seem like the only one asking “wtf” when I see something like this at OAM,

This means, as a rule of thumb, that the network must store ((4/3) + 1) * 3 = 7 MB of imagery plus tiles for every 1 MB of source imagery uploaded. If we load up all of the approximately 4 TB of LandSat-7 data at a 30m resolution, and generate a complete tile set, we will need 16-28 TB of storage in the network to hold it all. If stored on EC2, this would cost up to US$3,000 per month — and that’s just for one layer at a low resolution.

Or when a user asks a simple question

We want to serve the US NAIP Aerials in 1m resolution (which are a total of about 4.7 TB of MrSid/Jp2 data) on a interactive web map as an optional map background. [sic] .. we determined early on is that MapServer is too slow to serve compressed imagery such as the native MrSid Jp2 imagery on the fly for our needs. [On using Mapserver to serve uncompressed tifs] … would also “blow up” the total data volume to something about 60 TB … Thus, we are in the process of researching options on how to serve the compressed data as fast as possible “on the fly” and without the need for caching them on disk

All replies, except one from (somewhat ironically ) Christopher Schmidt, ignores the initial constraint and instantly tells the user a cache is required.

The root of the problem is the assumption that for every organisation, every deployment, you absolutely, unequivocally must create a tile-geo-arcgis-spatial-osm-mapproxy-squid-cache. We’ve gotta do what Google does! I truly fear many organisations are being misled and are unnecessarily transitioned to tiling solutions when quite frankly they don’t need to. More importantly though, GIS software representatives are using the community affinity addiction(?) for tiling everything to mask quite frankly, badly poorly performing software to begin with.

So let us all take a deeeep breath next time you’re scoping out an imagery solution. Why do you need a tile cache? That’s great that your cache can max out a 100mbit connection (its not hard), but you’ve not only increased your storage requirements by a factor of 4, 8 or 20 times, you’ve also taken away other functionality for your customers and limited yourself to one convention.

If you do need a cache and by crikey they are needed in many situations, implement LRU or a hybrid cache solution but most importantly, give your customers the original WMS service. For all its warts, at least it gives them some options.

So to answer both quotes above,

Storing 4TB of uncompressed Landsat 7, 30m data for the whole world as a single compressed ECW at 1:20 will be approx. 200 gb, visually lossless and $30 per month to store on Amazon S3. As some examples, i have the following 3 band mosaics
1. Landsat742.ecw, 1,414,317 px x 534,778 px which totals 2,515,088 KB (yes, thats ~2.5gb). Did i mention this was created way back in 2003?
2. Melbourne.ecw, 413,333 px x 346,667 px which totals 30,626,916 KB or ~30 GB from our friends at SKM Ausimage
3. Metro_Central_2007_Mosaic.ecw, 224,100 px x 304,400 px which totals ~11.5 GB from Landgate
ERDAS Apollo can serve all these mosaics, as 256px tiles on demand and still max out the 100mbit network; no problems. To prove, I ran our tiling test tool over a gigabit connection back to Apollo to see the throughput over a short 180 second test plan
1. Landsat.ecw
  1. Random: 31837 tiles, avg 181.79 tiles per second, RT 0.03 seconds, throughput 15.2 MB / sec
  2. Sequential: 60673 tiles, avg 314.41 tiles per second, RT 0.02 seconds, throughput 26.65 MB / sec
2. Melbourne.ecw
  1. Random: 10286 tiles, avg 109.92 tiles per second, RT 0.05 seconds, throughput 13.43 MB / sec
  2. Sequential: 39980 tiles, avg 230.25 tiles per second, RT 0.02 seconds, throughput 34.89 MB / sec
3. Metro_Central_2007_Mosaic.ecw
  1. Random: 35585 tiles, avg 203.18 tiles per second, RT 0.02 seconds, throughput 33.15 MB / sec
  2. Sequential: 47191 tiles, avg 271.19 tiles per second, RT 0.02 seconds, throughput 51.12 MB / sec

So instead of looking at pure throughput of the cache tile server (which has been proven to be a fizzer), if we also take into account the storage requirements and plot the two variables, I know which one I’d choose. That ERDAS Apollo license is looking pretty damn attractive right now, isn’t it … isnt it *starts shaking*?

What I also find interesting is there seems to be a slight resurgence back to on-demand solutions after, invariably, users realise the scalability or flexibility issues with full tile caches. JPEG2000 seems to be making a comeback thats for sure for image serving, but dont forget Kakadu has the same licensing restriction as the ECWJP2 SDK, it aint free-as-in-beer either. OSM Mod_tile is also a good example of a hybrid solution with on demand rendering.

ps. Has anyone tested beyond 100mbit on any other tiling solution?

pps. ERDAS has its own tiling container format known as OTDF. Clearly this is for our most demanding customers where they need performance above and beyond the above

FUD, FUD, FUD some more

Chris Tweedie — Wed, 19 May 2010 07:23:05 +0000

The Simon Hope vs Paul Ramsey posts has some classic asides.

I just had to re-quote the following comment from Atanas Entchev as it made me laugh. I am now personally tasked at seeking out and destroying this mysterious section of psychologists deep within ERDAS headquarters. I will also disassemble all subliminal messages embedded within our marketing and my blog *dons hat*. I have even heard the ESRI psychologist department is some 500 people strong!!

[sic]… the flawed assumption that decision-makers always make decisions based on reason.The “dealers”, on the other hand, know this to be false. So they employ (I speculate) psychologists to design sales tactics (such as FUD) that identify and target decision-makers’ *emotions*. They sell the sizzle, not the steak.

I like my sizzle as well as a good steak. If the steak tastes appalling I send it back. If I didn’t inquire to what I was ordering and expected pork? Well …

And then from Ian Turton,

Does your software use open standards that allow me to switch to another program next year or am I hooked to a conveyor belt of increasing license charges year after year?

Yes, my software does use open standards and yes if you’d like to switch to another program next year be my guest. How many organisations using opensource switch from mapserver to geoserver to mapnik to deegree to mapguide and back again every year? Mapwindow to QGIS to GRASS to UDIG to JUMP? SQLite to Postgres to Mysql? FDO to OGR to Geotools …? Although the FUD from opensource radicals (for lack of a better word) that proprietary solutions have a perpetual ball and chain, this just isn’t true. Sure some workflows are but certainly not to the degree some make out and I’d be damned to think of many without alternatives.

Come on lads, the underlying expectation here is that the vendors are somehow responsible for corporate (or not) entities selecting the wrong tool for the job or paying through the nose when there are viable and cost effective alternatives. Due diligence is king. After all, you are the ones with the $$, the phone to the ear, the door you can close, the conference you didn’t have to attend, the support and maintenance you didn’t have to renew and the software you didn’t have to use. Its my job to prove to you the value of ERDAS offerings, just as its Simon’s job to prove ESRI, Brett’s to prove Mapinfo or FME and Cameron’s to prove Mapserver or Geoserver. Whats the diff, really,between Cameron doing the pushing and the first three?

http://blog.cleverelephant.ca/2010/05/whos-your-dealer.html

ERDAS Apollo results updated

Chris Tweedie — Thu, 22 Apr 2010 01:49:45 +0000

With Apollo 10.1 about to get out the door I have updated the WMS image serving benchmark results. I’m still yet to update the product-by-format graphs as I will be rolling out a more dynamic and easier to maintain page shortly. One main addition was extending the ECW Test plan to from 150 to 300 users. I received a lot of requests from people wondering what the peak throughput was, which turns out to be not much higher at around 120 maps per second (but still, crazy quick at ~2 sec avg response).

ERDAS has also just registered for the Benchmarking event in Barcelona which brings the tally to 11 products which is great to see *queue herding cats picture*. So everyone, please stop asking me

ERDAS Apollo vs ESRI ArcGIS Server

Chris Tweedie — Tue, 13 Apr 2010 23:38:00 +0000

Lets face it, whatever benchmark results a vendor (*gasp*) publishes always draws a certain amount of suspicion. Luckily however, T-Ma py (Czech Republic) have just made available a detailed independant 20 page report on ERDAS Apollo vs ESRI ArcGIS Server.

T-Mapy have a long history with ESRI and now also ERDAS technology so they offer great perspective and expertise on both products. Michal Šeliga has done a wonderful job analysing performance and other metrics for serving a very large (290gb) 10cm aerial photo via WMS. Word of the day goes to “eyemetricaly worse” on page 13

Image serving updates

Chris Tweedie — Mon, 15 Mar 2010 00:06:07 +0000

I had hoped to post a lot more WMS image serving challenge results by now, but to date only Robert Parker at Lizardtech has taken me up on the offer with Express Server 6.1. Apologies to Rob for taking so long to publicize the results as he was very eager to send them through and I’ve been sitting on them for well over a month now. Gold star to Lizardtech.

ESRI? Autodesk? Deegree? Mapserver? Geoserver? Oracle? Manifold? Mapinfo? … Show me your muscles (in my best Arnie voice). Don’t forget that results from real world users, not just developers are just as valuable.

On the benchmark side of things, I have also updated the Mapserver results with the 5.6.1 build. Sheesh, talk about being spammed very vocal ECW support was dropped and unfortunately I was unable to get the Kakadu JP2 driver working. I’ll update the individual graphs when I get some time but here is the formats-by-product result. The solid, bold line represents 5.6.1, the dotted stroke the original 5.4 results. Yes, something crazy happened on the TIFF External test but I reproduced the result over the typical 3 test run … will revisit that one later.

Prohibitively difficult vs Protecting IP

Chris Tweedie — Wed, 17 Feb 2010 10:39:30 +0000

It seems various discussions are appearing postulating that ERDAS is making it “prohibitively difficult” to download the ECW SDK from our website. I’d like to make clear that this is absolutely not the case. From the website,

Request a Download
The ECW SDK 3.3 and the ECW SDK 3.3 Source Code are made available for download on an as-needed basis, after consultation with the product manager, Mr. Paul Beaty. To request a download, please contact by e-mail Mr. Paul Beaty, paul.beaty@erdas.com, with your name, organization, full address to include country, telephone, and email, and a description of your intended use. Use of the SDK requires advance acknowldgement of a EULA.

We have been forced to remove the direct download link due to numerous, frequent disregard for the attached SDK license terms and therefore ERDAS’ Intellectual Property. This post is not aimed at anyone in particular, but I emplore any potential users to email Paul and he would be happy to provide you the SDK. I just got off the train with the guy and he is eager to talk to anyone on our core technology, including a lot of new functionality available in the upcoming SDK v4 series which contains some pretty exciting stuff, v3.3 is over 3 years old guys!

We understand previous license terms have been somewhat ambiguous for some users, therefore emailing Paul will also ensure your intended use (and thus your organisation) complies with the terms and allow ERDAS to better track the usage throughout the community.

I am sure Paul will update his blog with more information very soon *hint hint* …

So I’ve been thinking ..

Chris Tweedie — Wed, 27 Jan 2010 09:30:34 +0000

The raster benchmarks have been a outstanding success with 3,000 + page views over the last 40-odd days. But what I’ve been struggling with is how to expand to more software or more platforms. Clearly I am not the master of web GIS Applications because I still for the life of me can’t get Mapguide configured so I’ve thrown up my hands and will claim DLL Hell. The Deegree guys are keen, but their preferred storage mechanism is tiles. I’ve gotten lots of hits from ESRI, Lizardtech, Cadcorp, Autodesk, Caris, Rolta, Intergraph and Mapinfo (to name a few) so I’m sure they’re keen aren’t you guys? *nods*. Everyone wants stats on different hardware configurations. Everyone keeps emailing me.

So here’s my thought. It might fail miserably; I might get no-one submitting any responses but here goes. If it fails, then there’s always Barcelona i guess and the list

I’d like to propose the following raster challenge to whoever is reading this (eg. you). You do not need to be the software developer on the project, in fact it will be more interesting if there’s both developer and real user feedback!

Download the BlueMarble world-topo-bathy-200406-3×86400×43200 (2.2 gb torrent) worldwide series
Convert, tile, compress, pyramid, overview, palette the original dataset into whatever format or file composition you’d like. Configure your server accordingly to read the dataset and serve out as an OGC WMS
Document the steps used to configure the dataset (hint: reproducible). Include details on the final disk storage and or number/size of files
Document your server hardware configuration. Particular emphasis on OS, CPU, Memory and Disk configuration
Document your software configuration. This time you’re not bound to prior documentation so developers, go for your life … users, do your best
Download the attached JMX plan and reconfigure the server details. Do not modify any other part of the plan apart from Lines 482 to 550
Install JMeter if you havent already and execute the plan
1. jmeter -n -p jmeter.properties -t myserver-bluemarble.xml -l myserver-bluemarble.xml.logs
Run the OSGEO Benchmarking summarizer.py
1. python summarizer.py myserver-bluemarble.xml.logs > myserver-bluemarble.xml.sum
Zip the documentation, myserver-bluemarble.xml.logs as well as myserver-bluemarble.xml.sum
Email the zip to me at chri…@webmapper.com.au and i’ll update the benchmarking page as soon as they come in

The idea behind this is to remove any ambiguity behind a single person configuring all apps, see if the they scale across different deployments, allow applications to use their “preferred” format and most importantly see whether users can reproduce the results!

This is clearly not going to be a comparative exercise. Even if you dont have a crazy 8/16 core server machine, I’d still urge you to submit the results. The point here is to get as many applications documenting how to squeeze the highest peak performance out of each. Results will not be compared as the platform will never be the same by design … Apple. Meet Orange.

Let the games begin~

On the hunt for some more benchmarks ..

Chris Tweedie — Wed, 23 Dec 2009 06:20:03 +0000

Sourcing independant benchmarks or comparisons is always a difficult exercise. To continue on my recent raster benchmark quest, here are some other similar raster performance studies i’ve found round the interweb. If you know of any others, please throw them in the comments!

ESRI Image Server vs LizardTech Express Server

http://geoinfo1.lib.uidaho.edu/loadtest/ (June 2009)

Not much background info on the test setup in particular the input data … but the Pylot results are available and when compared produce a clear winner in LizardTech at all tests. The following are the throughput achieved at 50 user load. The average response times showed Express Server with ~0.5 sec average compared with ~2.5 seconds with Image Server.

Pursuit of the Perfect Digital Ortho File Format

http://www.igic.org/training/pres/conf/2009/perfectortho.pdf (February 2009)

Although again lacking specific reasoning on how they ran the tests., this one is more centered on desktop performance reading a variety of raster formats. There’s a couple of flaws with the support table, as ECW is in fact supported through Microsoft Office and likewise ESRI Image Server can also read the format after purchasing the required license from ERDAS. I would have liked Larry to list the resulting file sizes but his general consensus of running with 2 formats … Geotiff + JPEG compressed with ECW seems like a common conclusion. People seem to forget however that as soon as you introduce enterprise image serving capabilities, the output formats caters for the common request to be able to open in MS Paint or MS Word. I would love to see someone try to open a 700mb JPEG Compressed GeoTIFF in Paint without waiting a very, very long time

Overall though, ECW achieved very good performance in many of the tests with Geotiff winning some large scale tests, presumably when it was requesting 1 reasonably sized geotiff tile

Benchmarks updated

Chris Tweedie — Wed, 16 Dec 2009 07:07:33 +0000

Small update, Added a new ECW 256px tile test Added a new ECW reprojection test Added some more info on my mapguide config problems I’m still keen for someone to verify my numbers. It would be nice to know whether I’m alone in the universe or not even if the setup is different .. Related posts:ERDAS Apollo results updated On the [...] Related posts:

Raster Image Serving Benchmarks

Chris Tweedie — Fri, 11 Dec 2009 02:41:57 +0000

I am pleased to announce my own performance metrics continuuing on from the FOSS4G 2009 WMS Raster tests. So, whats new?

Tests extended from just TIF and ECW to TIF, ECW, JP2, MrSID, TIF Tiled, TIF Internal Pyramid, TIF External Pyramid
Platform changed from RHEL to Windows Server 2008 x64
Increased the threads from 1,10,20,40 to 1,10,20,40,80,150.
Hardware increased from a 4 core to 8 core server
Analysed throughput not only by input format, but by output WMS Format as well. 8bit PNG vs 24 bit PNG vs JPEG vs GIF
Added ERDAS Apollo to the mix along with Mapserver and Geoserver (Deegree and Mapguide was with very limited success … I’ll add these later)

I will endeavour to update the page with new results as there is no question further tuning could be applied. I am not going to comment specifically on the results, as I want to leave the interpretation up to you.

Look out for more performance tests in the coming days..

Enjoy!