Thursday, December 13, 2012

Don't "upgrade" to WMS 1.3.0 unless you really have to, stick to 1.1.1

Chances are that you already heard me say something along the lines of "Don't 'upgrade' to  WMS 1.3.0 unless you really have to because it's a mess, stick to WMS 1.1.1 if you can". This may sound like an old story to those who have been around since the adoption of WMS 1.3.0 in 2004, but believe it or not we still hit those 1.3.0 issues today 8 years later. The last time was a few weeks ago and that got me started on that same old rant once again. I figured that I'd archive a copy of it here once and for all, and next time I can just point people to this blog.

The long story

 

"Don't 'upgrade' to  WMS 1.3.0 unless you really have to because it's a mess, stick to WMS 1.1.1 if you can"... well, I may not have used those exact words, but I mean it, WMS 1.3.0 is a pain to support properly, both on the client and on the server side, and should be avoided unless you have very good reasons to switch to it, like requirement for ISO compliance for instance. If you can stick to WMS 1.1.1 then you will be much happier.

I was involved with the OGC revision working group (RWG) that worked on the adoption of WMS 1.3.0 and I had the "chance" (or not?) to witness what happened. The "problem" is that OGC was trying to get WMS 1.3.0 approved as an ISO spec, and the ISO review process forced them to break compatibility with WMS 1.1.1 and older on a few aspects in order to meet the very strict ISO requirements.

One of those aspects was a requirement to honour the axis order defined by EPSG in its database of coordinate systems. In WMS 1.1.1 and older, early day OGC WMS authors and implementers had made a "mistake" and always assumed an x-followed-by-y axis order for coordinates, so the BBOX for EPSG:4326 (WGS84 geographic coordinates in degrees) was treated as:

  BBOX=lon_min, lat_min, lon_max, lat_max

But the EPSG database that WMS refers to defines the axis order for its 4326 SRS to be latitude followed by longitude (or y-x for the mathematicians around the table), so in WMS 1.3.0, you have to use:

  BBOX=lat_min, lon_min, lat_max, lon_max

However, for most projected coordinate systems, EPSG still defines the axis order as x followed by y, so nothing changes between 1.1.1 and 1.3.0 in those cases. For instance, with EPSG:3857 (Google Mercator projection in meters), the BBOX coordinate order remains the same for both WMS 1.1.1 and 1.3.0:

  BBOX=xmin, ymin, xmax, ymax

This strict compliance requirement imposed by the ISO review process on the OGC RWG means that WMS 1.3.0 server and client code needs to be aware of the axis order for each SRS that it supports and to use the correct coordinate order in its BBOX (and also the BoundingBox elements of the GetCapabilities output IIRC)... that's the mess I was alluding to. (There are a few thousand codes in the EPSG database that software needs to be made aware of.)

The issues that we encounter the most often with WMS 1.3.0 implementation range from incomplete axis order support for some EPSG codes to some implementations incorrectly assuming that WMS 1.3.0 just swaps the axis order to y-x for every SRS. And yes, there are still some implementations in 2012 that don't get this right.

Note that to help work around this issue for those writing simple clients/servers who want to avoid using EPSG codes and be free to continue with x-y axis order everywhere, the WMS 1.3.0 RWG invented a new "CRS:*" namespace for projection codes in which it defined a few well-known coordinate systems. They are listed in Annex B of the WMS 1.3.0 spec. The most common one is CRS:84 which is essentially EPSG:4326 with lon-lat (x-y) axis ordering. Unfortunately this is just a patch and doesn't address the core issue which is a brutal incompatibility between WMS 1.1.1 and 1.3.0.

Examples

 

Since MapServer complies with all those requirements, here are some example showing the change in behaviors between WMS 1.1.1 and 1.3.0 with MapServer WMS, taking the following BBOX:

  bbox=-90,0,0,90

Using WMS 1.3.0 with EPSG:4326 (with lat-lon, a.k.a y-x axis order):

  bbox=-90,0,0,90&crs=EPSG:4326&version=1.3.0

this bbox is interpreted as

  lat_min = 90 South 
  lon_min = 0
  lat_max = 0
  lon_max = 90 East

and you get the following map image:


 However, if you want to keep it simple and stick to x-y axis ordering, and want the BBOX treated as

  lon_min = 90 West
  lat_min = 0 
  lon_max = 0
  lat_max = 90 North

then you can either use WMS 1.3.0 with CRS:84:

  bbox=-90,0,0,90&crs=CRS:84&version=1.3.0

Or stick to WMS 1.1.1 with EPSG:4326:

  bbox=-90,0,0,90&srs=EPSG:4326&version=1.1.1

and in both cases you'll get this:




Hopefully now you'll understand next time you hear me mumble "... stick to WMS 1.1.1...".

P.S. Also note in the examples above that the WMS 1.1.1 "srs=..." parameter was renamed to "crs=..." in WMS 1.3.0, but that's a story for another day.

19 comments:

  1. yeah, the WMS axis order issue is some kind of a neverending story, I think. But I hope within the European INSPIRE process, WMS 1.3.0 will get its breakthrough for client and server use. So WMS 1.3.0 would be the "standard case"

    ReplyDelete
  2. I'm not following the logic here. "WMS 1.1.1 does it wrong. WMS 1.3.0 does it right. Therefore don't change." Huh??? Keep in mind that geographers (you know, our customers) think in terms of (lat, long), not (x, y). We (geospatial software engineers) should cater to their needs, not the other way around. If your software isn't able to handle things properly, perhaps you need to try a little harder.

    ReplyDelete
    Replies
    1. It all depends on one's definition of "right" and "wrong".

      Geographers could not care less what axis order the WMS protocol uses because they do not write GetMap requests by hand (only engineers do that). However they care that the machine to machine convention (the spec) is clear, unambiguous, and supported by their software of choice so that they get predictable results when they connect to a remote WMS service.

      My take is that when you have a stable convention in place and widely used (i.e. WMS 1.1.1 using x-y axis order by default everywhere for years) then it is bad to change something like the axis order handling in the next revision without putting a big red warning on that change, or moving to 2.0 to indicate API breakage.

      MapServer has had this right for years so that's not the question. It's the fact that people implementing WMS 1.3.0 today in their own software still get it wrong that proves my point that we/OGC did a poor job of communicating this incompatibility. In this context, this blog article, in addition to being a personal rant can also be seen as an attempt at documenting the issue for the posterity.

      Delete
  3. I totally disagree, USE WMS 1.3.0 forget about 1.1.1.
    1.1.1 does not respect axis order both on the request AND on the image.
    1.3.0 preserve TRUE axis order in the query.

    You are talking about ONE projection, but do you know how many projections are NOT in lon/lat order ?
    42% of the EPSG database is not in lon/lat, that's over 2000 coordinate reference systems !

    I can understand that people who use only two (4326 and perhaps 3857) it's a pain, but those don't care about the protocol and it's hide in the tools they use. But those who know what a projection is and what coordinates mean do you think it's fun to flip axis in the query on 'some' projections, definitly not.

    The truth is this huge never-ending mess comes from PROJ4 which doesn't care about axis order and place longitude first on all projection. therefor it respect EPSG for only half of all projections (and doesn't respect reprojection recommandation neither). Even postgis database is completely messed up, we have to hack our application to reset axis in the right order to make proper analyze.

    Since EPSG is the reference, everything which doesn't stick to it should be removed, that include WMS 1.1.1.

    ReplyDelete
    Replies
    1. Declaring any system that uses lon,lat as broken simply is not an option in real life unfortunately.

      For the record, PROJ4 and PostGIS did NOT invent the convention of using x-y axis order everywhere. Most GIS use it under the hood as demonstrated by interchange formats from the main vendors of our industry that have followed that convention for years. It's probably in the same spirit that the early days WMS authors and implementers also used that convention:

      - ESRI's Arc/Info E00 format uses x,y or lon,lat for coordinates everywhere

      - MapInfo's MIF/MID format uses x,y or lon,lat for any coordsys

      Those are only two examples off the top of my head, I'm sure there are tons of others.

      Unfortunately we cannot declare WMS 1.1.1 and those interchange formats as defunct and remove them, there are still services and files using those formats that will be around for years to come and that need to continue to be supported.

      Delete
    2. Sure they will exist for years, but if people like you who don't want to change their habits continue to consider them normal they will be here for decades. We better fix our generation mistake before all the next ones pay the price too.

      Something you did't consider, systems are not only 2D, navy/military/meteorology/oceoanography often work with 3,4 and 5 dimensions grids (I work daily with 3D and 4D netcdf / grib datas).

      navigation map where in lat/lon very long ago and are still used in this order, lon/lat is not a de-facto.


      Delete
  4. I know, I m repeating myself, but in fact it is the case that WMS 1.3.0 will definitely play a very big role within European INSPIRE directive as it is defined the base for View services. This means, that for Europe, WMS 1.1.1 "leading role" will turn into obsoleteness. On my opinion the number of 1.1.1 users and active service instances will strongly decrease.

    ReplyDelete
  5. Regardless of who's right or wrong, asking the client to have a copy of the EPSG database to know what's the orientation of each and every CRS is a bit ridiculous, and besides, sometimes you have to setup a custom coordinate system on the server, what's the axis order of it, how are clients going to guess?.
    WMS 1.3 could be made much easier to use if there was a simple way for the client to ask the server if the CRS in use is using north/east or east/north axis order (maybe, add it as an attribute directly in the capabilities document).

    ReplyDelete
    Replies
    1. I think, the axis order is given by CRS definition. Every CRS is defined with an axis order. So if a client supports a specific CRS, it should know how the axis order of it is defined. And of course, the axis order for a CRS should be defined the same for both, client and server side. There is no need for an additional attribute anyway.

      Delete
    2. Yes, the axis order may be part of the CRS definition, but the beauty of the WMS spec (and one reason for its widespread adoption IMHO) is that it is possible to write a simple thin client that knows nothing about the CRS it deals with, it just deals with the BBOX as pairs of coordinates in a cartesian space, and displays the GetMap images to its viewport without any kind of smarts.

      This is a very common real life use case, and asking this kind of thin client to carry a CRS database just to deal with axis order goes against the simplicity of the WMS protocol.

      Andrea's suggestion to add a mechanism to fetch the axis order of a given CRS could be a good way to deal with the issue (better than asking a thin client to carry a DB of thousands of CRS definitions). Maybe a new DescribeCRS request?

      Plus if you define your own CRS (sometimes you have to define your own because EPSG simply does not define every possible CRS in the world) then you need a way to provide the axis order and other CRS params to a remote client that is not aware of this custom CRS. This mechanism would also address this use case.

      Delete
    3. Now you convinced me! On my opinion this should be proposed to the OGC WMS SWG to get it discussed by the standard´s authors. Andrea and Daniel do you have access to the SWG to bring this argument up there? Or is this already done? I could assist here.

      Delete
    4. I have no particular access beyond portal access, which does not seem to provide much of a benefit, OGC favours in person meetings instead. All we can do is issue a change request against the WMS spec, and this is something anybody can do: http://www.opengeospatial.org/standards/cr

      Delete
  6. I can speak for such a change request as I am going to attend the next OGC TC Meeting. I think it would be a nice idea to bring the issue and suggested solution up at the face to face meeting of the WMS SWG in Redlands.

    So you have member access to ogc portal? Then you could subscribe the observer agreement and be an observing member of the WMS SWG for 4 weeks and have access to the SWG mailing list. after those you are a voting member of the SWG.

    ReplyDelete
    Replies
    1. FYI I am not directly involved in OGC TC activities any more. Glad to hear that you are going to champion this.

      Delete
    2. I think, it is of general interest to make usability of WMS better in the future. And as we can see within our discussion, within 3 of the OSGeo Projects implementing the WMS standard (Mapserver, Geoserver and deegree), there are contributors supporting the idea of having the CRS issue clear and the concrete proposals of Andrea and you. So I think we have an enormous argument here for a change request / new mandatory requirement for WMS standard.
      I will try to convince deegree comittees for the idea. Maybe you could do the same for Geoserverand Mapserver (And then have a proposal from the 3 big OSGeo WMS implementations)

      Delete
  7. In the recommendations for Inspire download services
    http://inspire.jrc.ec.europa.eu/documents/Network_Services/Technical_Guidance_Download_Services_3.0.pdf

    the CRS is given as links like
    "http://www.opengis.net/def/crs/EPSG/0/25832"
    Unfortunately these links do not get direct answer about the axis order but the response contains elements like
    gml:cartesianCS xlink:href="urn:ogc:def:cs:EPSG::4400" or
    gml:ellipsoidalCS xlink:href="urn:ogc:def:cs:EPSG::6422"

    Only after checking from the EPSG registry what cs codes 4400 and 6422 mean it is possible to know the axis orders which seem to be
    "Cartesian 2D CS. Axes: easting, northing (E,N). Orientations: east, north. UoM: m." and
    "Ellipsoidal 2D CS. Axes: latitude, longitude. Orientations: north, east. UoM: degree", respectively.
    There seems to be 10 Cartesian CS codes, 6 Vertical CS codes, 7 Ellipsoidal CS codes and 1 Spherical CS code in the web application at
    http://epsg-registry.org/
    It makes 24 codes together. Perhaps we could program our servers and clients to understand those 24 codes directly so that making a query to address "http://www.opengis.net/def/crs/EPSG/0/xxxx" would be enough?


    ReplyDelete
  8. OWS-10 is just wrapping up, and had some great progress on this issue. A branch of the CITE Team engine can be configured as a WMS service proxy, allowing client conformance tests to be performed (finally).

    I really hope this is rolled out promptly as it would greatly improve these issues, as this problem rests as much (or more) with the client software then those kind enough to publish information.

    Right now as a server you are kind of screwed as the vast majority of clients are expecting easting/northing order when they request EPSG:4326. If they request with CRS:84 (or one of the URIs) then you know they are expecting EPSG database order and can safely provide correct information.

    I also discovered that you can detect when a server is inconsistent by comparing LatLong bounds against the EPSG:4326 envelope.

    This has two interesting consequences.

    This check has been added to the CITE Tests, if this means I can trust the servers to provide content in a consistent fashion then I could consider recommending WMS 1.3.0.

    And more importantly (as a client developer) this allows us to detect a server that is inconsistent - and then "correctly" interpret the GetCapabilities information (regardless of if they are strictly following the EPSG database, or have the axis order in easting/northing order).

    We should be able to add this inconsistent GetCapabilities check to the GeoTools WMS support classes as a temporary bandaid around this issue, perhaps a similar fix could be provided for OpenLayers? The ability for existing clients to smoothly work with WMS 1.3.0 (regardless of this mistake) would be a big improvement.

    ReplyDelete
  9. Merci Daniel pour cet article (le seul d'ailleurs sur le net à ce sujet) ! Un partenaire qui utilisait nos WMS avec MapServer et avait des problèmes pour visualiser nos couches. Après avoir regardé les requêtes, j'ai pu comprendre que son outil (MapInfo : http://reference.mapinfo.com/software/mapxtreme_windows/english/7_1_0/MapXtreme_710_ReleaseNotes.pdf) utilisait la version 1.3.0 du WMS et qu'il utilisait la projection 4326, donc l'inversion des axes. En faisant une recherche dans Internet, j'ai trouvé ton billet et c'était exactement cela le problème. Cela démontre que ce genre de connaissance mérite d'être publiée, sinon on se serait retrouvé dans le néant. Merci, ce billet a été très utile :-) Vive le partage d'information.
    Nicolas Gignac

    ReplyDelete