multipart
IDS over Multipart Messages
1. Messages
The IDS Information Model defines a generic Message class and properties as well as specific message subclasses, which are used for communication in the IDS.
The generic Message class itself is usually not used for communication in the IDS, but to model similar properties of the different messages. The Information Model provides a set of specific message types for different scenarios. The message types can be grouped into three main message classes:
RequestMessages
ResponseMessages
NotificationMessages
The message type and properties are part of the Message header. Further describing the individual message types would go beyond the scope of this document. All available message types with additional information about their usage, payloads, property restrictions and new properties can be found in the following table: Link to Message table (updated continuously).
The Information Model does not provide an explicit modeling about the payload of a message, except the formalization in the description fields of the message types and the Message documentation mentioned above.
2. Structure of a Multipart Message
For HTTP Multipart communication, each request and response in the IDS is represented as a message of content type multipart/mixed or multipart/form-data with at least one part. The first part MUST contain the instantiated message class of the information model (e.g., SelfDescriptionRequest), serialized as JSON-LD (named "header"). The second part of the request contains the associated payload of the message type as defined in the information model (consequently named "payload"). For instance for a BrokerQueryMessage, this part contains the textual query (e.g., SPARQL SELECT or any other supported query format) that should be processed by the Broker. Messages structured in that way are independent of the chosen communication method and protocol, i.e., they may be transferred synchronously by using HTTPS, asynchronously by using MQTT or by other session or stream-based protocols (IDSCP). For IDSCP, however, metadata is exchanged just once for each connection and this only is true if multipart messages are tunneled.
As an example, let's consider an IDS message of type ids:DescriptionRequestMessage which does not require any payload part. In order to send this message to a connector that supports the synchronous HTTP API, the serialized instance of the SelfDescriptionRequest class needs to be sent through an HTTP POST request to the /infrastructure endpoint of the receiving connector. Note that this request message type does not require a payload section, as all necessary information is supplied through the header attributes. The POST body itself MUST be of content-type multipart/form-data or multipart/mixed, e.g., multipart/form-data; boundary=--msg-boundary with this exemplary raw content:
However, in this minimal example, the receiving connector cannot verify the authenticity of the requesting connector, i.e., it cannot determine if the sender is (still) a valid participant on the IDS ecosystem or if the claimed facts in its self-description are actually true or not. In order to provide this kind of security, each request needs to be accompanied by a Dynamic Attribute Token (DAT) (see XXX). A connector that is in possession of this token SHOULD/MUST present it at each outgoing message exchange so that the receiver can base access control decisions on it. Note that it is up to the receiving connector to handle and interpret the security token DAT. The header part of the above example with included security token looks like this:
The message below shows the receiving connector's response to the incoming message. When the response is received using HTTP, the content-type header would be "multipart/form-data; boundary=mPLw1UTMYjqqYqh1Bb_ttWBKcdSPfB9FBgz3;charset=UTF-8". Note that the boundary is just an example and is usualy auto-generated by the communication tool or library that is used by the sending connector. As can be seen in the "header" part of the message, the message type is DescriptionResponse. It is linked to the previous message through the correlationMessage property.
JSON-LD is the required format for any header content. In the example, it has been generated by the publicly available Java library that implements the Information Model. Using this library is not mandatory but eases the parsing of valid IDS statements and messages. Implentations in other programming languages may also come in the future. As the message below is a response to a self-description request, it contains the requesting connector's self description in its payload section. Note that the content of the payload depends on the message type: if, for instance, the message type is a general OperationResult, then the payload might be binary data or any other format, depending on the operation that was invoked by the original request.
3. Self-Description
Information about the Self-Description can be found here.
4. Interactions with Infrastructure Components
4.1 Metadata Broker Communication
The IDS Broker acts as a registration and discovery service for the other connectors in the IDS. As such, it receives information about the availability of connectors, their characterisitics and offers. The Broker persists these information and enables queries. As not only connectors would like to search for resources, a Broker should also host a website enabling the necessary interactions in a user-friendly manner.
The default interaction protocol for the IDS Broker is HTTPS. Any IDS Broker should therefore serve the following endpoints:
Table 1: Endpoints of an IDS Metadata Broker
The path segment of the URL is optional. In cases the Broker is one service of many of a certain authority, the path is necessary in order to link to the correct server.
4.1.2 Hosts and Endpoints
An endpoint is defined in each resource (ids:Resource) and contains information regarding the access to a Resource’s contents, e.g., the Artifact. There are two types of endpoints:
Static Endpoints (ids:StaticEndpoint), which are used for exposing an Artifact
Interactive Endpoints (ids:InteractiveEndpoint), which do not reference a specific Artifact, but an operation binding (ids:OperationBinding). Operation bindings refer to a specific operation and an API document that describes the protocol details Additionally, an endpoint contains the following fields / properties:
* mandatory fields
The endpointHost maps to a network host of a Connector, from which the data consumer can get access to the contents of the resource. The following example contains a partial snippet of a Connector’s self-description for the network host specification as well as a snippet of a Static Endpoint in a Resource (ids:Resource)
Partial Snippet of Connector's self-description (ids:host property)
Partial Snippet of a Resource's metadata (ids:resourceEndpoint property)
4.1.4 Connector Registration
A Connector is registered at an IDS Metadata Broker through a IDS ConnectorUpdateMessage.
4.1.5 Connector Update
The Connector Update Workflow is the same as the one for registering a new Connector. Note that the old Connector Self-Description is completely overwritten by the new request.:
Listing 1: Multipart message to register a new Connector.
4.1.6 ConnectorUnavailableMessage
To remove a Connector registration from the Broker, a ConnectorUnavailableMessage is used.
Note that no payload is required. The mentioned connector is always the issuning connector. That implies that no third party can send this type of message. An IDS Broker may however allow other options to remove a connector's descriptions, for instance through its Web UI.
4.1.7 Browsing/Querying Broker Data
A Broker might provide query functionality by accepting SPARQL messages like the one below. However, a Broker instance might support different query languages as e.g. SQL or Cypher.
4.1.8 QueryMessage
A IDS QueryMessage is used to send complex queriies, for instance to search for all registered Connectors and the information model versions they support:
The Broker might answer with a ResultMessage or reject the request (RejectionMessage with a helpful rejection reason).
4.1.8 Requesting explicit Connectors and Resources
Send a DescriptionRequestMessage to receive the Self-Description of known Connectors or Resources.
4.2 Clearing House Interactions
The Clearing House is a central component that provides access to a trustworthy persistence store. Connectors may log messages in the Clearing House and read messages from it. Using the Clearing House is optional for all connectors. A valid DAPS token is required in all requests to the Clearing House.
4.2.1 Endpoints
The default interaction protocol for the IDS Clearing House is HTTPS. Any IDS Clearing House should therefore serve the following endpoints:
*Table 1: Endpoints of an IDS Clearing House
4.2.2 Creating a Processes
The IDS Clearing House structures its logs into processes and requires that all messages are logged under a process id. Processes are meant to represent agreed upon data exchanges that include the storage of the contract agreement and all messages that document the data exchange. Processes can be created by any IDS connector with a valid DAT, who becomes the owner of the process. Additional owners can be defined in the parameters of the request. Only owners of the process may read or write log entries for the process. The Clearing House expects a RequestMessage in the header part of the multipart message.
Parameters
Response Codes
4.2.3 Logging messages
The information that should be logged in the Clearing House (logData) is sent in the payload of a LogMessage. The log entry stored in the Clearing House consists of the payload and meta-data from the LogMessage and is stored under the given process pid. Returns a signed receipt of the logged data in form of a JSON Web Token (JWT) as payload of the response.
Parameters
Response Codes
Example Http Multipart Request
4.2.4 Query multiple messages of a process
Retrieves multiple log entries that are stored under the given pid in the Clearing House. Pagination is enforced, so retrieving all log entries might require multiple requests. When querying the Clearing House the query may contain the requested page of data, the size of the requested page, and the sorting order of the results. The Clearing House answers the request with a ResultMessage that contains in the payload the log entries found, as well as the pagination information page, size and order that were used in the request. Each log entry is returned as a LogMessage, i.e., the payload of the ResultMessage contains a json
array of LogMessage.
Parameters
Response Codes
The following example illustrates an answer of the Clearing House to a multipart request with a QueryMessage:
4.2.5 Query a single message of a process
Retrieves the log entry that is stored under the given id and the given process pid in the Clearing House. The Clearing House answers the request with a ResultMessage that contains as the payload the log entry found. The log entry is returned as a LogMessage, i.e., the payload of the ResultMessage contains a LogMessage.
Parameters
Response Codes
Last updated