IDS REST

Linked Data Platform (LDP) is the W3C recommended architecture to allow read/write interactions on distributed data using Semantic Web technologies. It combines the well-established identification methods of Web resources (HTTP URIs/URLs) and vocabularies from RDF with the access concepts from Linked Data (data retrieval using HTTP GET, Content-Negotiation). This document presents a binding of IDS interactions to the LDP Recommendation in order to merge the advantages of both approaches: LDP allows the dynamic interaction with unknown entities while the IDS defines patterns to ensure a sovereign and controlled data ecosystem.

The merged approach in this specification presents a union of both LDP and IDS concepts, called IDS-REST. As such, the resulting binding limits the interaction opportunities of the original LDP Recommendation regarding the requirements of the IDS specifications and vice versa. The benefits of the combined approach lie in (a) the simplification of IDS interactions using the LDP best practices, while (b) the trustworthiness of the IDS ecosystem is introduced into LDP-conform applications.

Terminology and Roles

Table 1: Relevant terms and their roles.

Resources of Interest

The LDP defines a special type of resources that are exposed by servers: Containers. Containers are defined as Resources that can contain further Resources or Containers. This creates a tree-like structure which can be explored following hypermedia links. Therefore, user agents are able to discover the data content of a LDP server on-the-fly and without any external information.

On the other side, an IDS Connector and the IDS Resources it provides must follow the scheme of the IDS Information Model. The rich semantics and the thereby imposed implications of the different IDS classes are crucial for any IDS conform interaction. The thereby imposed complex operation semantics however is explicitly not regarded by the LDP Recommendation. That implies the following consequences for a LDP binding in the IDS:

• IDS Connectors are the single Origin Server of an IDS Resource.

• The root container must be an instance of the class ids:Connector or one of its subclasses.

• Offered data is collected in an instance of ids:Catalog, which at the same time appears as an ldp:Container.

• ids:/ldp:Resources inside an ids:Catalog can have further child resources.

• Allowed operations, for instance the creation or deletion of a resource, depend on the location of the target resource in the data tree. Certain resources (instance of ids:Connector at the root level and its catalog) can be deleted or updated only by the Connector Operator.

Motivation

At the time of writing this document, the IDS interactions follow a message-driven pattern. The IDS attributes are partitioned and encoded in defined message types according to the Multipart format. With this specification all IDS interactions are supported. Therefore, the obvious questions are: Why is it necessary to define yet another interaction pattern? This contradiction is crucial to answer as every additional layer of specifications, requirements and standardization increases the onboarding efforts of developers and thereby hinders the adoption of the IDS in general.

It has been shown that the proposed Multipart and message-driven communication is significantly complex to implement for a huge proportion of developers. The success of the IDS however is directly dependent on a vivid and motivated developer community. Therefore, the IDS must provide the patterns that developers are familiar with instead of insisting on rarely used technologies. In addition, the tool support for Multipart messages is rather limited, while there is a massive trend towards REST APIs on the Web but also in any kind of distributed architectures.

First and foremost, REST APIs restrict the number of assumptions a consuming application needs to make about a remote endpoint. The protocol, possible states, state transformations, exchange sequences, error behavior, and so on need to be understood by all parties. REST is already a well-known and commonly accepted paradigm providing recommendations (not necessarily answers!) to a lot of these topics. Reusing them simplifies the life of the developers, increases the maintainability, and last but not least presents the condensed lessons learned of a huge community.

Apart from these design advantages, the tool support for REST APIs is significantly better - especially in terms of open source projects - than for any other communication pattern.

Following this argumentation, the IDS community agreed that a REST binding is required. Whether it - at some point - might even replace the currently used interaction patterns is not in the scope of this document. The hereby presented binding is intended as an coequal binding with the same relevance and commitment as the other bindings.

Operations and Interactions

An instance of an IDS Message, as defined in the IDS Information Model, encodes i.a. the intended command, content, target, expected response. For example, an ArtifactRequestMessage contains its operational semantics ('return an IDS Artifact'), the target resource, expected response (ArtifactResponseMessage), data format (Multipart) etc. This implies that both the sender and the receiver of the message need to share all these assumptions and deeply understand the IDS requirements and side-effects. Obviously, this is especially hard for people joining the IDS community but also for experienced IDS developers.

The REST paradigm on the other hand separates the command semantics from the content and deeply integrates the communication pattern with HTTP methods. This reduces the necessary assumptions but also restricts the expressiveness of the interaction. In addition, REST interactions are stateless. In terms of the IDS communication and data exchange, this means that neither the server nor the client is expected to remember previous interactions or how a current request fits into an ongoing sequence of requests. As a consequence, all requests must be complete so that they provide all necessary information for their correct execution. Hence, User-Agents interact with IDS Connectors only through the URI pointing at the target resource, the eight HTTP methods (GET, PUT, POST, DELETE, HEAD, OPTIONS, PATCH, TRACE), the HTTP Headers and the optional request body. Therefore, an explicit mapping of the defined message types to the resource- and state-based view of REST is required. The following sections outline this mapping in detail.

IDS Message Mapping

Table 2: IDS Message Types and their representation in the IDS REST protocol.

Encryption

Currently, TLS encryption of all messages is required in the IDS. Unencrypted requests, for instance plain HTTP, must always be rejected. Furthermore, certificates and keys used for the communication encryption must be valid and signed by a trustworthy CA. A client must not expect a reasonable response for non-encrypted requests, even if the server responds with a redirect to the HTTPS URL.

Identification and Location

Following the IDS Information Model, IDS entities, such as Participants, Connectors, or Resources, are represented by information documents containing a self-description. In addition, each IDS entity has its own unique URI identifier.

IDS components can use this URI to find a document representing the entity. For instance, Connector A (https://example.org/ConnectorA) asks the IDS Metadata Broker for information about a Connector B (https://example.org/ConnectorB). Subsequently, the Broker responds with a document containing a representation of the current state of the Connector B (ConnectorB.jsonld) as far as the Metadata Broker has stored it.

Note that in this example, the Identifier URI of Connector B (https://example.org/ConnectorB) is not necessarily equal to the URL at which it's endpoints (https://companyB.com/ids/connector/data/) are located. Furthermore, Connector A must know the Broker's endpoint URL (https://metadata-broker.org/infrastructure) in advance. Connector A can then lookup the received document representing Connector B (ConnectorB.jsonld), discover the endpoint URL, and send a request to it.

The IDS-REST approach is significantly different. The identifying URI is bound to the location of the entity itself. Therefore, Connector B must not use https://example.org/ConnectorB as its identifier but https://companyB.com/ids/connector/ instead and provide at least a self-description document at this location. This enables Connector A by learning about a Connector B (identified by https://companyB.com/ids/connector/) to directly communicate with it without further lookups on endpoints. The identification URI must always be the same as the root URL under which the entity is hosted. However, this simplification on the client connector-side increases the effort on the server connector-side as the later must always synchronize identities with hosting locations.

Authentication

The IDS transfers and validates identity claims using the Dynamic Attribute Token (DAT). The DAT contains, among other attributes, the signed identity of an IDS Connector.

Each IDS Connector involved in an interaction, both the origin server and the user agent, must verify the respective DAT of the counterpart by the following sequence

1. Compare the signature with the public key of the referenced Dynamic Attribute Provisioning Service (DAPS)

2. Compare the announced public keys with the certificate used for the TLS channel

3. Verify the iat, exp, nbf time stamps.

4. Check if the security claims match their own security specifications.

Protocol

The operation semantics follows the typical REST semantics for CRUD operations. The following items state the usual interpretation of the eight HTTP methods as remote commands.

• GET: Read the target resource

• PUT: Overwrite the target resource state. If it did not exist, create it with the sent state. The desired resource state is in the request body.

• POST: Create a new resource (if the target resource is an LDP Container). The desired resource state is in the request body. Undefined for non-LDP Containers.

• DELETE: Delete the target resource.

• HEAD: Read the meta-information of the target resource.

• OPTIONS: Read the allowed HTTP operations on the target resource.

• PATCH: Attach content to the target resource.

• CONNECT: undefined method for IDS-REST

The IDS REST binding follows this pattern. The following sections explain the generally used parts like headers, body and response structure, the error codes, and maps the IDS Messages to these interaction items.

Request Headers

Table 3: Request headers.

Full list of http-header attributes to use see here.:

Request Body

For every class except an ids:Artifact, the body of the HTTP request must always contain a valid RDF serialization. An ids:Artifact can also contain non-RDF or even binary data. In any case, the serialization of the body must be announced in the corresponding value of the "Content-Type" header. The default serialization in the IDS is JSON-LD (media type: 'application/ld+json'). Systems not able to operate on RDF can also treat the content as plain JSON and parse it like that (media type: 'application/json').

Still, every IDS compliant message exchange must use valid JSON-LD serializations, in particular by using the JSON key "@context" with either a proper local context definition or by linking to an appropriate remote context document. The default context element for the IDS is "@context": "https://w3id.org/idsa/contexts/context.jsonld".

The context key-value pair must always be the first one in any IDS compliant JSON or JSON-LD object.

Access Control

Access control (AC) to a given resource is inspired by the W3C work on the "Web Access Control" (WAC) language. However, the permissions and restrictions that can be expressed by WAC are not sufficient for the IDS scenarios, mainly due to the missing support for Contract Classes. For instance, the context 'time' is not considered by WAC.

In general, an IDS-REST client must add an DAT to its request. Without it, or if the DAT is not valid or contains conflicting statements, for instance issued-at before not-after, the IDS-REST server must directly reject the request. The IDS-REST server must return a 4xx status code and may return an additional indication in the response body. Independent of incorrect DATs, an IDS-REST server may also require custom Access Tokens. The recommended behavior is to use the ids-authorizationToken request header. The tokens can be maintained through well-established standards, like OAuth2, or through proprietary processes. This gives the Providers of Connectors the ability to create own data spaces where only dedicated Participants can access protected resources.

Usage Control

The IDS-REST binding solely targets the interactions and operations at the interface of a Connector to the outside network. Both client (Data Consumer) and server (Data Provider) therefore only focus on Access restrictions for enforcement. The explicit Usage Control tasks happen inside the Data Consumer and are not directly affecting the client-server interaction anymore.

Nevertheless, the IDS-REST binding contains the means to express usage restrictions as instances of Contracts. Contracts are represented as any other Web resource and can retrieved and manipulated accordingly. Their distinct meaning, to outline and control operations on data, is defined by the IDS Information Model and further specifications. The IDS-REST binding is therefore an agnostic layer to pave the way for Usage Control but does not by itself add further specifications.

Response

Every request message must be answered with the according HTTP response message. In case the receiving IDS Connector assumes a misuse, for instance, a denial of service attack, it could also block the requester. In any other case, the response message must contain a reference to the originating message, an HTTP status code, a set of response headers and optional data in the response body.

Response Header

The response headers are the same as the request header.

For the full list of http-header attributes to use, see here.

Table 4: Response headers.

Status Codes

Although the IDS Information Model contains its own status codes, the commonly known HTTP codes are used as much as possible. The following table lists the available ones:

Table 5: Status codes of responses.

Complex Operations and Workflows

Composed workflows or more elaborate interactions that can not be directly mapped to the previously outlined plain HTTP operations have to be managed by the User-Agent and the Origin Server respectively. While each Rest interaction - containing exactly one request/response pair - is exhaustive by definition, typical IDS workflows require more complex communication patterns.

That implies that even though the single interactions are stateless, and therefore only defined by the current state of the target resource and the HTTP message, the Origin Server and the User-Agent must remember the results of previous interactions to some degree. The respective obligations depend on the type of intended operation and is specified through the sub-classes of the IDS Message class. The following table outlines the binding of the Message semantic with the respective REST binding for the IDS:

Table 6: Interaction patterns and their mapping in the IDS-REST binding.

A connector conform to IDS REST must, in addition to the general criteria for IDS Connectors, implement the following features:

• It must have a self-description hosted at its root path. Note that the root path is not necessarily the first segment of an URL path but the first dedicated section for the connector. For instance, for https://example.org/ids/connector/some/ids/endpoint/ the root path can start after 'connector/'

• All entities have at least one URL which points to their location on a Connector server.

• All Resources/Representations/Artifacts must be able to handle related Contract interactions sent to them. At least an error message must be returned.

Notifications

Notifications and publish/subscribe or push-based communication are not yet parts of the IDS REST protocol binding. While this functionality may be introduced in the future, the Linked Data Notification may serve as a preliminary reference. See here IDS Linked Data Notification (IDS-LDN)

Identification and Naming of Resources

Even though the structure of the URL of a resource must not necessarily reflect its location on the server, it is recommended to apply this pattern by the Origin Server. For instance, creating a resource called https://bad-example.org/some-resource can be created at an IDS Connector at https://example.org/connector/:

@prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix owl:  <http://www.w3.org/2002/07/owl#> .
@prefix ids:  <https://w3id.org/idsa/core/> .
@prefix ldp:  <http://www.w3.org/ns/ldp#> .

<https://example.org/connector/>
    a ids:BaseConnector, ldp:BaseContainer ;
    ids:catalog [
        a <https://example.org/connector/Catalog> ;
        ids:offers <https://bad-example.org/some-resource> ;
        ldp:contains <https://bad-example.org/some-resource> ;
    ] ;
    ldp:contains <https://example.org/connector/Catalog> ;
    #...
.

However, without further specifications, the Origin Server has the complete naming authority. It could therefore respond instead:

201

Link: https://example.org/connector/Catalog/some-resource

And update its state as:

@prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix owl:  <http://www.w3.org/2002/07/owl#> .
@prefix ids:  <https://w3id.org/idsa/core/> .
@prefix ldp:  <http://www.w3.org/ns/ldp#> .

<https://example.org/connector/>
    a ids:BaseConnector, ldp:BaseContainer ;
    ids:catalog [
          a <https://example.org/connector/Catalog> ;
          ids:offers <https://example.org/connector/Catalog/some-resource> ;
          ldp:contains <https://example.org/connector/Catalog/some-resource> ;
    ] ;
    ldp:contains <https://example.org/connector/Catalog> ;
    #...
.

Therefore, the User Agent must not expect its originally suggested name to be used by the Origin Server. However, the Origin Server should take the User Agent's suggestions into account, and serve them as long as possible. The client can use the "Slug" header to indicate the desired path extension.

Equality of Resources

Despite ids:Contracts, the equality or non-equality of resources can be expected. For instance, the same ids:DataResource can be hosted at several IDS Connectors with no obligation to announce this fact. This is different for Contracts - and especially ids:ContractAgreements - as those imply legal consequences. A Contract published at one place must not have an extended copy at some other place. While the Contract itself can of course be copied, it must always the same URI.

Furthermore, no IDS Connector is allowed to rename, extend, update or shorten an ids:ContractAgreement.

REST API Documentation (non-normative)

• IDSA Swagger API

Community-provided Descriptions

• https://app.swaggerhub.com/apis/jlangkau/IDS-LDP/

• PolicyInformationPoint API Description

Example IDS REST Request (non-normative)

The example below shows a typical Multipart Message, sent with a POST request:

--msgpart
Content-Type: application/json; charset=utf-8
Content-Disposition: form-data; name="header"

{
  "@context" : "https://w3id.org/idsa/contexts/context.jsonld",
  "@type" : "ids:ConnectorAvailableMessage",
  "@id" : "http://industrialdataspace.org/connectorAvailableMessage/34d761cf-5ca4-4a77-a7f4-b14d8f75636a",
  "issued" : "2019-12-02T08:25:08.245Z",
  "modelVersion" : "4.1.0",
  "issuerConnector" : "https://companyA.com/connector/59a68243-dd96-4c8d-88a9-0f0e03e13b1b",
  "securityToken" : {
    "@type" : "ids:DynamicAttributeToken",
    "tokenFormat" : "https://w3id.org/idsa/code/tokenformat/JWT",
    "tokenValue" : "eyJhbGciOiJSUzI1NiIsInR5cCI..."
  }
}


--msgpart
Content-Type: application/json
Content-Disposition: form-data; name="payload"

<Payload: Self-description of the Connector>

This Multipart Message can be transformed to the following REST request:

GET https://www.example.org/server/cpu_3
ids-modelVersion : "4.1.0"
ids-issuerConnector: "https://wall-e.nicos-rd.com/"
ids-securityToken: "eyJ1c2VyIjoiam90dCJ9...42"
ids-issued: "2019-12-02T08:25:08.245Z"
Accept: text/turtle

References and Related Resources

Authors

Contributers

Version History

---