Raster Image Serving Benchmarks

I am pleased to announce my own performance metrics continuuing on from the FOSS4G 2009 WMS Raster tests. So, whats new?

  1. Tests extended from just TIF and ECW to TIF, ECW, JP2, MrSID, TIF Tiled, TIF Internal Pyramid, TIF External Pyramid
  2. Platform changed from RHEL to Windows Server 2008 x64
  3. Increased the threads from 1,10,20,40 to 1,10,20,40,80,150.
  4. Hardware increased from a 4 core to 8 core server
  5. Analysed throughput not only by input format, but by output WMS Format as well. 8bit PNG vs 24 bit PNG vs JPEG vs GIF
  6. Added ERDAS Apollo to the mix along with Mapserver and Geoserver (Deegree and Mapguide was with very limited success … I’ll add these later)

I will endeavour to update the page with new results as there is no question further tuning could be applied. I am not going to comment specifically on the results, as I want to leave the interpretation up to you.


Look out for more performance tests in the coming days..


9 thoughts on “Raster Image Serving Benchmarks”

  1. Chris,

    Great to see you taking the lead from the vendor standpoint and very interesting results. Lets hope this and the regular FOSS4G benchmarking become the norm as these sorts of exercises are a great resource for anyone looking to select image servers.


  2. Chris,

    The flip side to the fixed amount of FastCGI workers in Mapserver config is that only so much work can be done. :) We chose 8 workers for our 4 core system. Your system has 8 cores, so you’ll want at least 16 workers and it would be interesting to see if going to 24 improves or degrades performance.

    The steep plummet of GeoServer at 80 threads speaks to something else going on, but it’ll take someone more Java-smart than me to figure that one out. I’ll point folks in the direction of your information. Great work!

  3. Thanks for the comments guys. Paul, you’re right .. I will rerun Mapserver with 20 processes to mirror the setup for Apollo.

    In my earlier tests, the extra processes didnt gain any increased throughput except on Mapserver except for ECW (tif presumabily had hit its limits)

  4. Chris,

    Jeff McKenna probably has a 5.6 build available now… there were changes in there and in the latest GDAL/Proj that made some differences, as I recall, particularly in the raster domain. Since you’re re-running things… :)


  5. Dear Chris,

    regarding your deegree configuration problems, I would like to send you some example configuration we use here, if you let me know some email where to send it to.
    Furthermore I’d like to point you to the configuration of single image files described here: http://download.deegree.org/deegree2.2/docs/deegree_wcs_documentation_en.pdf on page 18. ECW files work pretty much the same, just drop in the right bounding boxes, deegree will check the file extension to determine the file type. Configuration is spread over three XML files, but mostly straightforward.
    I’d like to point out, that deegree currently is bound to a rather old version of the ECW library, which is known to have some problems.

    Thanx for your efforts

  6. Bart, afaik the ms4w/osgeo4w builds are missing jp2/ecw plugin support. If they remain excluded from the binary builds i will rerun the tests on the other formats ..

Comments are closed.