So I’ve been thinking ..

The raster benchmarks have been a outstanding success with 3,000 + page views over the last 40-odd days.  But what I’ve been struggling with is how to expand to more software or more platforms. Clearly I am not the master of web GIS Applications because I still for the life of me can’t get Mapguide configured so I’ve thrown up my hands and will claim DLL Hell. The Deegree guys are keen, but their preferred storage mechanism is tiles. I’ve gotten lots of hits from ESRI, Lizardtech, Cadcorp, Autodesk, Caris, Rolta, Intergraph and Mapinfo (to name a few) so I’m sure they’re keen aren’t you guys? *nods*.  Everyone wants stats on different hardware configurations. Everyone keeps emailing me.

So here’s my thought. It might fail miserably; I might get no-one submitting any responses but here goes. If it fails, then there’s always Barcelona i guess and the list

I’d like to propose the following raster challenge to whoever is reading this (eg. you). You do not need to be the software developer on the project, in fact it will be more interesting if there’s both developer and real user feedback!

  1. Download the BlueMarble world-topo-bathy-200406-3×86400×43200 (2.2 gb torrent) worldwide series
  2. Convert, tile, compress, pyramid, overview, palette the original dataset into whatever format or file composition you’d like. Configure your server accordingly to read the dataset and serve out as an OGC WMS
  3. Document the steps used to configure the dataset (hint: reproducible). Include details on the final disk storage and or number/size of files
  4. Document your server hardware configuration. Particular emphasis on OS, CPU, Memory and Disk configuration
  5. Document your software configuration. This time you’re not bound to prior documentation so developers, go for your life … users, do your best
  6. Download the attached JMX plan and reconfigure the server details. Do not modify any other part of the plan apart from Lines 482 to 550
  7. Install JMeter if you havent already and execute the plan
    1. jmeter -n -p -t myserver-bluemarble.xml -l myserver-bluemarble.xml.logs
  8. Run the OSGEO Benchmarking
    1. python myserver-bluemarble.xml.logs > myserver-bluemarble.xml.sum
  9. Zip the documentation, myserver-bluemarble.xml.logs as well as myserver-bluemarble.xml.sum
  10. Email the zip to me at and i’ll update the benchmarking page as soon as they come in

The idea behind this is to remove any ambiguity behind a single person configuring all apps, see if the they scale across different deployments, allow applications to use their “preferred” format and most importantly see whether users can reproduce the results!

This is clearly not going to be a comparative exercise. Even if you dont have a crazy 8/16 core server machine, I’d still urge you to submit the results. The point here is to get as many applications documenting how to squeeze the highest peak performance out of each. Results will not be compared as the platform will never be the same by design … Apple. Meet Orange.


Let the games begin~

One thought on “So I’ve been thinking ..”

  1. We will run this test on Express Server and share our results with you.

    Please note that there is a major flaw in the test design: the final numbers are heavily affected by network bandwidth. No matter how powerful and fast the server may be, the image must travel over a network (which is not controlled in this test and yet heavily affects the results).

Comments are closed.