ERDAS ECW JPEG2000 v5.0 performance teaser

As the team starts tying up loose ends before the ERDAS ECW JPEG2000 SDK v5 release I thought I’d post a teaser of some performance metrics we’ve collected as part of the QA. To keep it somewhat relevant to the average user and not just developers I have compiled some stats comparing throughput using a variety of ECW and JPEG2000 drivers using the GDAL utilities.

The real aim here was to ensure no regressions for our own ERDAS SDK with recent driver enhancements but it also allows us to draw some other interesting conclusions relevant for anyone using ECW and JPEG2000 formats (via GDAL or not).

Test Plan

  1. Grab a handful of customer sourced, real-world ECW and JPEG2000 images and use them for straight-line decoding via GDAL
  2. Grab a handful of customer sourced, raw uncompressed TIF and IMG images and use them for straight-line encoding via GDAL
  3. Perform the same tests on Windows and *gasp* Linux
  4. Run for two cold test runs. Plot the average result
  5. Hope we outperform our previous SDK releases and hope we outperform other JPEG2000 drivers

Test data

  • world.ecw 86,400 x 43,200 px x 3 band uint8
  • large_ortho.ecw, 550,000 x 390,000 px x 3 band uint8
  • ADS80.ecw, 75,332 x 10,513 x 4 band, uint16 lossy 20:1 (ECW v3)
  • perth.ecwp, 12,445 x 10,594 px x 3 band uint8
  • world.jp2 86,400 x 43,200 px x 3 band uint8
  • CIR_ortho.jp2, 60,202 x 47,934px x 4 band uint8
  • epje-nitf.jpc, 5,654 x 11,677 px x 8 band uint8 (NITF EPJE Profile)
  • pleiades.jp2, 8,496 x 16,384 px x 4 band uint8
  • ADS80.jp2, 75,332 x 10,513 x 4 band, uint16 lossy 20:1
  • landsat_7.img, 12,661 x 12,601 px x 7 bands uint8
  • naturalearth.tif, 21,600 x 10800px x 3 band uint8
  • pan.tif, 35,180 x 28184 px x 1 band uint8
  • ads40.img, 12,236 x 12,196px x 3 band uint8 

Test cases

Decoding

  • ECW #1 : time gdal_translate -of GTiff -co “COMPRESS=JPEG” -co “PHOTOMETRIC=YCBCR” -co “TILED=yes” world.ecw test.tif
  • ECW #2 : time gdallocationinfo world.ecw 2156 30321
  • ECW #3 : time gdal_translate -of GTiff -co “TILED=yes” -projwin 319794 5754321 322641 5752034 large_ortho.ecw test.tif
  • ECW #4 : time gdal_translate -of GTiff -co “TILED=yes” -srcwin 20000 5000 1500 1500 ADS80.ecw test.tif
  • ECWP #1 : time gdal_translate -of GTiff -co “COMPRESS=JPEG” -co “PHOTOMETRIC=YCBCR” -co “TILED=yes” ecwp://demo-apollo.geospatial.intergraph.com/images/australia/perth_05mar04_psh.ecw test.tif
  • JP2 #1 : time gdal_translate -of GTiff -co “COMPRESS=JPEG” -co “PHOTOMETRIC=YCBCR” -co “TILED=yes” world.jp2 test.tif
  • JP2 #2 : time gdal_translate -of GTiff -co “TILED=yes” CIR_ortho.jp2 test.tif
  • JP2 #3 : time gdal_translate -of GTiff -co “TILED=yes” epje-nitf.jpc test.tif
  • JP2 #4 : time gdal_translate -of GTiff -co “TILED=yes” pleiades.jp2 test.tif
  • JP2 #5 : time gdal_translate -of GTiff -co “TILED=yes” -b 2 -b 3 -b 4 -srcwin 2000 5000 800 800 CIR_ortho.jp2 test.tif
  • JP2 #6 : time gdal_translate -of GTiff -co “TILED=yes” -srcwin 20000 5000 1500 1500 ADS80.jp2 test.tif

 Encoding

  • ECW #1 : time gdal_translate -of ECW -co “TARGET=95″ landsat_7.img test.ecw
  • ECW #2 : time gdal_translate -of ECW -co “TARGET=95″ NaturalEarth.tif test.ecw
  • ECW #3 : time gdal_translate -of ECW -co “TARGET=95″ pan.tif test.ecw
  • ECW #4 : time gdal_translate -of ECW -co “TARGET=95″ ads40.img test.ecw
  • JP2 #1 : time gdal_translate -of JP2ECW -co “TARGET=95″ landsat_7.img test.jp2
  • JP2 #2 : time gdal_translate -of JP2ECW -co “TARGET=95″ NaturalEarth.tif test.jp2
  • JP2 #3 : time gdal_translate -of JP2ECW -co “TARGET=95″ pan.tif test.jp2
  • JP2 #4 : time gdal_translate -of JP2ECW -co “TARGET=95″ -co “PROGRESSION=LRCP” NaturalEarth.tif test.jp2
  • JP2 #5 : time gdal_translate -of JP2ECW -co “TARGET=0″ NaturalEarth.tif test.jp2

Environment

  • Windows:
    • Windows 7 x64
    • GDAL 1.9.2 x64 VC100
    • Core i7 1.7ghz, 8gb RAM
    • Data read from Corsair M4 256gb SSD, written to attached 7.2k SATA
  • Linux: (Virtualized)
    • Ubuntu Server 12.04 x64
      • gcc 4.7.2
    • GDAL 1.10 (svn trunk)
    • 4 core 1.7ghz, 4gb RAM
    • Data read from Corsair M4 256gb SSD, written to attached 7.2k SATA

Note 1: The same libecwj2 3.3 library was used on both platforms. The same public patches were applied to mirror what is used in the wild.

Note 2: The environments are not intended to be compared. Please keep this in mind when looking at the results and do not try and draw any Linux vs Windows conclusions

Results

ecwjp2-linux-decoding ecwjp2-linux-encoding ecwjp2-windows-decoding ecwjp2-windows-encoding

Talking points

  1. If you currently use v3 SDK on Linux, v5 will give you significant decoding and encoding improvements no matter how you use our SDK
  2. JPEG2000 stability and robustness in the v5 is much improved. You will have to take my word for it right now but decoding test JP2 #1 shows 0 which indicates a crash in the old SDK. Test JP2 #4 shows a performance drop however the v3 SDK didn’t properly decode all precincts within the file. So although it completed quickly the output was not valid. I am sure many who have used v3 would agree that it had poor compatibility on particular JP2 format profiles so v5 should pleasantly surprise you.
  3. What about (geo) Jasper? What about OpenJPEG? I tested both on Linux however the driver and or SDK’s are just not suitable for these images. I actually recompiled three times all with the suggested patches with no change in behaviour. Both SDK’s were only successful completing JP2 #3, Jasper in 9842 secs (gave up at 50% and extrapolated it out) and OpenJPEG in 387.32 secs. All other decoding tests failed due to out of memory or seg faults indicating fundamental limitations of these toolkits when decoding geospatially-sized imagery. These are not large images so it was surprising to see such poor results for the open source JP2 drivers. So  beware not to draw any negative conclusions about JPEG2000 as a format when using these drivers.
  4. What about Kakadu? I did not have access to the recent kdu release so did not include the library in these results. I’m sure someone else will soon
  5. ECWP decoding gets a significant speedup due to the new ECWP v3 protocol which opens multiple-threads when retrieving the remote image stream. Anyone curious about the GDAL Async Reader should definitely check this out as i’m yet to see any third-party apps using this feature. Although not tested here ECWP progressive reading should outperform JPIP from the JP2KAK driver by a significant margin even when reading from the same input JP2 (ECW over ECWP will be faster still)
  6. Decoding JP2 #1 and ECW #1 tests are equivalent except for the storage format. The test results clearly show that for equivalent compressed files GDAL JP2 decoding is 2x (Windows) to 3x (Linux) slower than the same ECW. For multi-threaded use this gap will widen further in favour of ECW. This will be most noticable in workflows such as tile cache generation or enterprise imagery services. Remember wavelet formats arent all the same
  7. Encoding tests JP2 #2 and ECW #2 are also equivalent. In this case, JP2 compression is 10% (Windows) to 30% (Linux) slower. Although not charted, ECW output file size was 13,265 KB vs JP2 15,060KB for the same GDAL quality target of 95 (or 20:1 ratio). 
ecwjp2-corruptdisplay
Although it looks pretty, corrupt JP2 display like this will be rarely seen with v5

When will it be out? Very soon so stay tuned  ..

 

 

ERDAS Apollo vs ESRI ArcGIS Server

Lets face it, whatever benchmark results a vendor (*gasp*) publishes always draws a certain amount of suspicion. Luckily however, T-Mapy (Czech Republic) have just made available a detailed independant 20 page report on ERDAS Apollo vs ESRI ArcGIS Server.

T-Mapy have a long history with ESRI and now also ERDAS technology so they offer great perspective and expertise on both products. Michal Å eliga has done a wonderful job analysing performance and other metrics for serving a very large (290gb) 10cm aerial photo via WMS. Word of the day goes to “eyemetricaly worse” on page 13 :)

Raster Image Serving Benchmarks

I am pleased to announce my own performance metrics continuuing on from the FOSS4G 2009 WMS Raster tests. So, whats new?

  1. Tests extended from just TIF and ECW to TIF, ECW, JP2, MrSID, TIF Tiled, TIF Internal Pyramid, TIF External Pyramid
  2. Platform changed from RHEL to Windows Server 2008 x64
  3. Increased the threads from 1,10,20,40 to 1,10,20,40,80,150.
  4. Hardware increased from a 4 core to 8 core server
  5. Analysed throughput not only by input format, but by output WMS Format as well. 8bit PNG vs 24 bit PNG vs JPEG vs GIF
  6. Added ERDAS Apollo to the mix along with Mapserver and Geoserver (Deegree and Mapguide was with very limited success … I’ll add these later)

I will endeavour to update the page with new results as there is no question further tuning could be applied. I am not going to comment specifically on the results, as I want to leave the interpretation up to you.

boxing_glove

Look out for more performance tests in the coming days..

Enjoy!

CarbonArc revisited

Jeff Harrison’s mass email caught my eye today with the free trial of CarbonArc. I had the pleasure of taking an early release of CarbonArc for a spin last year and while it was a good start there were a few key things missing from my point of view. Having pushed and prodded Mapinfo 9.5 support for use with our SDI, why not run the new CarbonArc 1.6 through its paces …

From the press release,

CarbonArc PRO 1.6 eliminates barriers to SDI usability through advanced SOA-based discovery, analysis, exploitation, transaction management and security tools for OGC SDI – directly from the ESRI ArcGIS desktop.

Instead of just regurgitating the release, the key question that always gets asked is … doesn’t ESRI ArcGIS already do all this?

Its a tricky question to answer because while on paper i could say, “Yes it does!”, in reality there are just so many annoying quirks and missing features, it quickly becomes a nightmare integrating SDI features into your normal ArcGIS workflow (which is what this is all about, right?).  So with fingers crossed, lets dive right in and see what i can find in 5 minutes flat …

The Good

  1. Well laid out and robust tool set which delivers fully functional WMS/WFS/WFST/WCS capabilities … with no mucking about required!. The thing just works.
  2. Very easy export to Shapefile tool
  3. Full featured filter encoding support … no black magic, user has full control
  4. Full access to the query string for all service types
  5. Caching of features / images when saving to project files

The Bad

  1. GML Integration will always be difficult given its a completely new ArcGIS feature source, but its still not up to the average users expectations I dont think. There is no attribute table, the features are independant of pretty much all other ArcGIS functionality so you are really left with GAIA functionality squished into an ArcGIS window.
  2. Web services request headers are devoid of any accept-encoding headers. I’m still pondering the choice of Expect: 100 requests … but i’m sure there’s a good reason in the depths of HTTP …
  3. The exclusive tool for CubeWerx “Identity Management Service” seems a bit strange as i’m not aware of any services that use this technology? Shoot me a link if i missed something!
  4. Lack of support for any WMS LegendGraphics. Users can choose from multiple Styles, but theres no way to actually view legend information which is a bit disappointing
  5. In the full minute spent trying, i wasnt able to get the WFST support working … i could get as far as the schema but as soon as i’d try to insert a feature the thing would just bail out.

I’m beginning to think the best suggestion would be to somehow merge CarbonArc WFS functionality with an automated export to shape on each filter response. This would allow excellent SDI WFS support, while still giving full ESRI ArcGIS functionality to users without having to reinvent the wheel (for things like the edit system). With the WMS/WCS support pretty solid, i think getting the balance of WFS right could mean the difference between a very promising product and one I would recommend to everyone using our platform.

Thoughts?

WFS-T adventures with Mapinfo 9.5

So i’ve been a bit late taking a look at the new Mapinfo Professional v9.5. With the consistent dissapointment towards the consumption of OGC standards in commercial apps I wasn’t holding my breath … but wait a sec, it did work and it worked damn well. I mean, it worked flawlessly; updates, inserts, deletes, lock support and it also comes complete with a semi-intelligent conflict manager.

Mapinfo

A few more suggestions to improve things further (for anyone listening~) …

  1. Add HTTP compression handling. Huge performance gains with the transfer of features and its really a no brainer to enable in any http library.
  2. I am by no means a “Mapinfo Master” ™, but it would be great to enable an automated WFS Table refresh especially if you are retrieving features based on CURRENT_MAPPER.  I guess the CTRL+F5 shortcut makes it easy-ish … but i certainly found myself wondering whether i had retrieved the features or not and ended up just sending unnecessary requests.
  3. If a transaction is successful, give me some kind of alert. Alerting only when it fails does not instill much confidence whether my long edit session went through or not (even after refreshing)
  4. It would be fantastic to add helpful warning messages when performance drops. I’d imagine most users would skim over the maxfeatures and column / row filters and just add the layer.
  5. If the first request takes 5 minutes and Mapinfo tells me i just retrieved 4000 poly features totalling 10mb and it kindly directed me to the WFS how-to, i’d be more inclined to see what the filtering options were all about :)

So there’s no WFS1.1 support … but i’m still trying to get my head around handling the axis order issue and are more than happy to let sleeping dogs lie … at least for the moment. I only had time to test against our Geoserver installs, but it certainly seems tested against many other apps including Cadcorp, Ionic & Mapinfo. Geoserver specific here, but the advanced security in 1.6.x works very well with the bundled support for basic authentication.

Finally…