SOAP and REST in the OIIE

Over the past year, a few software vendors have expressed interest to MIMOSA in exploring an alternate web service technology stack based on REST and JSON rather than traditional SOAP and XML that we typically find used in Enterprise Application Integration. This is somewhat due to the perception, particularly driven from the open-source web application development community, that SOAP and XML carry a lot of overhead and baggage. In this article, I’ll compare the incumbent approach based on SOAP, which MIMOSA uses with the OSA-EAI, OSA-CBM and more recently, the OpenO&M Web Service Information Service Bus Model (ws-ISBM) to one based on REST. I’ll also discuss what the implications are of adding a RESTful stack may have on the work that we are doing with the OIIE (Open Industrial Interoperability Ecosystem).

What is SOAP?

SOAP (Simple Object Access Protocol) was developed as an object-access protocol in 1998 with the latest version, SOAP 1.2, released in 2003. It is based on HTTP and XML as underlying standards and is often used in combination with Web Services Description Language (WSDL) for specifying Web Services.

What is REST?

REST (Representational State Transfer) is an architectural style that was developed in 1996 in conjunction with HTTP 1.1. It usually involves operating a verb (e.g. GET, POST, DELETE) on one-to-many resources identified by a URL. There is no official standard for RESTful web APIs as it describes an approach rather than a protocol; however, most RESTful implementations are based HTTP, JSON and XML.

MIMOSA technology adoption

While MIMOSA does not have an official framework on which it bases its technology adoption, we generally evaluate the follow characteristics before adopting different technologies into our standards portfolio:

• Demand by members
• Attitude toward change by members
• Meet functional requirements
• Level of standardization and formalization
• Level of industry adoption, support and tooling
• Ease of use for developers
• Meet performance requirements

I’ll use these characteristics as the basis of analyzing SOAP and RESTful approaches in the following sections.

Demand and attitude toward change by members

While it is only a handful of organizations that have expressed interest in exploring a RESTful stack, these have exclusively been software vendors rather than end users. It is not unsurprising that end users show little interest in specifying technological requirements, as their focus is on supporting the line of business and its requirements. If a particular technology assists in supporting business requirements in a more effective and efficient manner, then there would be a strong value proposition for technological change. If it is change for its own sake, then this becomes a less compelling reason since legacy systems using older technologies still require support (it is uncommon to see a major uplift on brownfield sites).

Software vendors on the other hand, are intrinsically tied to technological progression, and from experience, software developers like new and shiny things. From a business perspective, vendors derive revenue through sales and support contracts – new products and features means additional revenue streams. Concurrently, older or neglected technologies also mean increased support costs to vendors with a diminishing talent pool.

From the software vendors, only one is a current MIMOSA member, while the others are in the process of becoming members. Like end users, existing vendor members have investment tied up in supporting existing implementations, while new entrants can be more agile with a blank slate.

Meet functional requirements

The current work priority for the MIMOSA Technical Committee is to support the OIIE and associated standards. The place within the OIIE where a RESTful approach applies is in the spot where the OpenO&M ws-ISBM is positioned. The ws-ISBM provides a standard interface based on SOAP web services that facilitates the exchange of data between applications.

The purpose of the OpenO&M ws-ISBM (and the abstract model that was extracted to ISA 95 Part 6 Message Service Model) is to provide a fixed set of web services using common messaging patterns supporting an unlimited number of message types. The ISBM would eliminate each standards organization from having to specify a data transport standard to be used with their data exchange standards, and therefore allow the standards organization to focus on the data content. This would then give software vendors some comfort to support a limited set of web services natively within their applications, and software vendors could implement support for messages that corresponded to relevant data exchange use cases and scenarios. SOAP was naturally chosen as an implementation specification, since the vast majority of standards-based integration uses SOAP if not XML.

That said, a RESTful approach can be applied to the ISBM and was in fact, envisioned by the editors of the ISBM and MSM specifications. An example proposal of a RESTful ISBM is shown in the appendix using a direct translation of the concepts from SOAP. There are, however, some non-trivial issues that the OpenO&M Technical Committee would need to address:

• Whether both XML and JSON (and others) are supported as message formats on a single channel
• Replacing WS-Security UsernameToken as a base level of security token format
• Replacing SOAP Faults for returning error information to clients
• Whether content-based filtering (currently using XPath) is to be supported if messages can be specified using multiple formats (see first point)

Expanding on the first point, at present, all of the data specifications specified by the OIIE Use Cases and Scenarios are XML-based or have XML representations. OAGi (Open Applications Group), one of the OpenO&M Imitative members, more recently published a JSON representation in addition to their XML representation for their data specifications, presumably to support demand by their members. While the ws-ISBM based on SOAP can technically support JSON message structures by embedding the JSON in the SOAP Body, the ws-ISBM would require a minor additional specification to do so in an interoperable manner.

Thus the primary functional driver for developing a RESTful implementation of the ISBM specification will be the support for non-XML messages formats.

Level of standardization and formalization

While technologically REST/JSON appears as a superior alternative to SOAP, where it falls down is at a standards level. Most RESTful services offered by organizations will follow conventions, but there isn’t a well-adopted standard that can be followed. Initiatives like JSON API are trying to fill that gap, but these are currently immature, and having organically grown to fulfill a need, currently aren’t part of the W3C or IETF. JSON Schema, published by the IETF, is still a draft standard. RDF/JSON, published by the W3C, is still a draft standard. And then there are no well-adopted, standardized, drop-in replacements for WS-Security, WS-AtomicTransaction, WS-ReliableMessaging.

Level of industry adoption, support and tooling

The fact that organizations are expressing interest in supporting RESTful interactions clearly indicates that there is a need for us as a standards organization to be watchful in this space. We can also see from a Google Trends comparison between the terms “web service soap” and “web service rest” that there is an inflexion point during 2014-2015 where the searches made on Google are starting to favor REST:

This inflexion point is hit in 2013 if the search terms “soap xml” and “rest json” are compared:

The two dominant enterprise application development frameworks of Java and .NET have long supported SOAP Web Services including contract-first code generation from WSDL. Microsoft added support for JSON serialization in .NET 3.5 (released in 2007) while Oracle added JSON support to Java EE 7 (released in 2013). Particularly in the case of Java, third-party libraries seem to be preferred compared to using the core libraries, and hence the late arrival into the Java core libraries.

It should be noted that due to the asset-intensive industries that we work in where legacy systems can mean working with information technologies 10-20 years old (if not older), we often do not have the luxury to use modern technology stacks if we need to support brownfield sites. The case for greenfield sites is usually different, but is influenced and restricted by the requirements of brownfield sites against the corporate strategic technology roadmap.

WSDL code generation

The use of SOAP web services is closely tied with the use of WSDL, which provides metadata about a service and allows software developers to generate code, dealing with higher level data transfer objects, rather than lower level constructs. While WADL (Web Application Description Language) was proposed as a WSDL equivalent for REST web services, it has not been standardized by the W3C. Initiatives such as Swagger will automatically document metadata and even perform client code generation, and while open-source, it is not based on a published industry standard.

Ease of use by developers

As software developers need to implement code to facilitate the exchange of data, let’s have a look at the differences in complexity to generate and consume the SOAP and REST HTTP payloads shown above. I’ll use Ruby for the examples, as one of its tenets is to optimize for developer happiness (and therefore simplicity and readability). Like Java and .NET, it contains libraries to help with heavy lifting.

I’ll use a very simple scenario of an application requesting tag data for pumps from another application hosting a tag data web service. While it doesn’t use an intermediate ISBM Service Provider, it is still illustrative of the comparison.

To generate the SOAP HTTP request and output the response:

tag_type = "pump"
require "savon"
client = Savon.client(wsdl: "http://example.com/tags?wsdl")
response = client.call(:get_tags, message: {"Type" => tag_type })
puts response.body

To generate the REST HTTP request and output the response:

tag_type = "pump"
require "httparty"
response = HTTParty.get("http://example.com/tags?type=#{tag_type}")
puts response

While this example is simplistic as it is hardcoded to a “Pump” type, it illustrates that there is little difference in code complexity between a SOAP and RESTful approach. The majority of work when developing data exchange adapters is in marshalling and unmarshalling data from and to an application data model rather than building the request or receiving the response.

Meet performance requirements

CPU Performance

REST/JSON is generally touted as a more performant solution than SOAP and others have conducted studies that confirm the assertion, particularly when dealing with less powerful devices such as mobile devices. This is primarily due to the complexities in parsing XML vs JSON rather than the HTTP call itself – XML and particular XML Schema, is a more complex specification than JSON, with the complexity allowing for greater functionality. This complexity, when implemented in parsing libraries, will result in slower performance, and tangentially, increase the maintenance cost of those libraries.

Message Size

Transport layer communications with both SOAP and REST takes place using HTTP (Hypertext Transfer Protocol) using a request and response model. Using the scenario above of requesting tag data, we can put together some examples of what this might look like when it gets to the HTTP layer.

HTTP Request

Here is an example HTTP request that would be sent in a SOAP exchange:

POST /tags HTTP/1.1
Content-Type: application/soap+xml; charset=utf-8
Content-Length: 128

<s:Envelope s:xmlns="http://www.w3.org/2003/05/soap-envelope">
  <s:Body>
    <GetTags>
      <Type>Pump</Type>
    </GetTags>
  </s:Body>
</s:Envelope>

Here is an example HTTP request that would be sent in a REST exchange:

GET /tags?type=Pump HTTP/1.1

The RESTful approach is considerably smaller than the SOAP approach and for this particular example, while the SOAP request can be cut down to 219 bytes, the RESTful approach is 28 bytes. Apart from the size, there are several differences:

	SOAP	REST
HTTP Method	SOAP only uses the POST HTTP method.	REST uses the entire vocabulary of HTTP methods. In this case, since the application is querying information related to a tag, the GET method is the most appropriate since it is used for retrieving information.
Request Data	The request data for a SOAP message is placed within the SOAP Envelope’s <body> XML tag. A SOAP Envelope also allows the use of a <header> XML tag for additional functionality, which can be omitted if unused (as with this example).	A GET request contains no body, and therefore all data must be included in the URL as a query string.

HTTP Response

Here is an example HTTP response that would be sent in a SOAP exchange:

HTTP/1.1 200 OK
Content-Type: application/soap+xml; charset=utf-8
Content-Length: 290

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Body>
    <GetTagsResponse>
      <Tag>
        <Id>0f8d4ced-8b4e-46da-9a6b-db7d68f35aa1</Id>
        <Name>P-11001</Name>
        <Type>Pump</Type>
      </Tag>
    </GetTagsResponse>
  </s:Body>
</s:Envelope>

Here is an example HTTP response that would be sent in a REST/XML exchange:

HTTP/1.1 200 OK
Content-Type: application/xml; charset=utf-8
Content-Length: 134

<Tags>
  <Tag>
    <Id>0f8d4ced-8b4e-46da-9a6b-db7d68f35aa1</Id>
    <Name>P-11001</Name>
    <Type>Pump</Type>
  </Tag>
</Tags>

Here is an example HTTP response that would be sent in a REST/JSON exchange:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 117

{
  "tags": {
    "id": "0f8d4ced-8b4e-46da-9a6b-db7d68f35aa1",
    "name": "P-11001",
    "type": "Pump"
  }
}

Similar to the HTTP request, the SOAP HTTP response is larger at 313 bytes, compared to 171 bytes for the REST/JSON HTTP response and 192 bytes for the REST/XML HTTP response.

Again, there is a lot of ceremony that is included with the SOAP Envelope structure leading to the larger size. Another difference can be seen between JSON vs XML in that the object type in the JSON tags array is implied by the serializer/deserialzer on either side of the exchange, while the XML examples make the <Tag> node explicit.

Larger size example

While a JSON approach will always result in smaller payload sizes than XML, I wanted to explore the difference with a larger example, where the overhead of the SOAP Envelope would become relatively inconsequential. Using the MIMOSA CCOM (Common Conceptual Object Model) representation of a Debutanizer Fractionator P&ID (Piping and Instrumentation Diagram) from the OGI (Oil and Gas Interoperability) Pilot, we see the following payload sizes:

Approach	Uncompressed (bytes)	Gzip compressed (bytes)	Uncompressed (% difference from REST/JSON)	Gzip compressed (% difference from REST/JSON)
SOAP	576,753	70,450	36.93	3.50
REST/XML	576,656	70,378	36.91	3.39
REST/JSON	421,188	68,069	0	0

I have included gzip compressed payload sizes, as most modern technology stacks support this as an option. I have also included the percentage difference from the smallest payload (REST/JSON).

We see that there is little difference between either the SOAP and REST/XML approaches, confirming that the SOAP Envelope adds next to no overhead on larger sized messages. However, there is a 37% difference between XML and JSON approaches appears quite significant. For a large capital project with 300 P&IDs, this difference amounts to an additional 47 MB. This is not a bewildering amount given today’s technology, and considering that gzip compression reduces this to less 1 MB, there is minimal advantage in using a JSON approach for reducing payload sizes.

SOAP and REST simultaneously?

So a RESTful approach looks good from a technology perspective compared to SOAP: it allows the transfer of smaller payloads over the wire, provides a (marginally) simpler programming model, and performs significantly better on lower-powered, typically mobile devices – which is a hot area of opportunity with the current Internet of Things movement. Where it significantly falls down is at the standardization level, and with MIMOSA being a proponent for standardization, our preference for technology standards is to wait until final standards are published and look like they will have broad adoption by industry.

However, the proposition is not an either-or scenario, and both SOAP and REST can potentially be used simultaneously. Similar to how SOAP 1.1 and SOAP 1.2 endpoints can co-exist simultaneously from the one service, it is conceivable that both SOAP and RESTful endpoints can co-exist in a single ISBM Service Provider implementation. Thus applications that prefer a RESTful approach can use the REST interface, while applications that prefer a SOAP approach can use the ws-ISBM interface.

To support the concurrent use of SOAP and RESTful approaches with the ISBM issues such as message formats and security tokens will need to be resolved when interchanging between the two approaches. There will also need to be some level of specification in the scenario where a message is posted using REST/JSON interface and how it is made available to subscribers using a SOAP/XML interface.

Like we have done in the past with the adoption of DTDs then XML Schema and UUIDs, MIMOSA is always open to adding different technologies in our standards portfolio, but at a time when they make sense.

Appendix – A RESTful ISBM

As we saw in the initial description about REST, a RESTful approach will apply methods to resources. The resources associated with the ISBM are:

• Channels
• Security Tokens
• Sessions
• Messages (Publications, Requests, Responses)

If we take the Channel Management Service operations and convert this to a RESTful approach, the following table shows one potential approach:

ISBM Channel Management Service Operation	Current behavior	RESTful implications
Create Channel	HTTP method: POST URL: /ChannelManagement Data: Specified in HTTP body	HTTP method: POST URL: /channels Data: Specified in HTTP body
Add Security Tokens	HTTP method: POST URL: /ChannelManagement Data: Specified in HTTP body	HTTP method: POST URL is /channels/{channel-id}/security-tokens Token data is specified in HTTP body
Remove Security Tokens	HTTP method: POST URL: /ChannelManagement Data: Specified in HTTP body	HTTP method: DELETE URL is /channels/{channel-id}/security-tokens/{token-id} Data: Specified in URL
Delete Channel	HTTP method: POST URL: /ChannelManagement Data: Specified in HTTP body	HTTP method: DELETE URL: /channels/{channel-id} Data: Specified in URL
Get Channel	HTTP method: POST URL: /ChannelManagement Data: Specified in HTTP body	HTTP method: GET URL is /channels/{channel-id} Data: Specified in URL
Get Channels	HTTP method: POST URL: /ChannelManagement Data: N/A	HTTP method: GET URL: /channels Data: N/A

This conversion is based on the rules:

• Querying resources results in a GET method.
• Query filters are embedded in the URL.
• Creating resources results in a POST method. Note that security tokens are a nested resource of a parent channel resource.
• Destroying resources results in a DELETE method. The token type will determine whether the token-id can be passed in the URL or in the HTTP body.