Comment on page
This document outlines the key elements of the transfer process protocol. The following terms are used:
- A message type defines the structure of a message.
- A message is an instantiation of a message type.
- The transfer process protocol is the set of allowable message type sequences and is defined as a state machine (TP-SM).
- A transfer process (TP) is an instantiation of the CNP-TP.
- A provider is a participant agent that offers an asset.
- A consumer is a participant agent that requests access to an offered asset.
- A Connector is a
- An Asset is data or a service a provider grants access to.
A transfer process (TP) involves two parties, a provider that offers one or more assets under a usage policy and consumer that requests assets. A TP progresses through a series of states, which are tracked by the provider and consumer using messages. A TP transitions to a state in response to a message from the counter-party.
A TP is managed by a
Connector. The connector consists of two logical components, a
Control Planeand a
Data Plane. The control plane serves as a coordinating layer that receives counter-party messages and manages the TP state. The data plane performs the actual transfer of asset data using a wire protocol. Both participants run control and data planes.
It is important to note that the control and data planes are logical constructs. Implementations may choose to deploy both components within a single process or across heterogeneous clusters.
Asset transfers are characterized as
pulltransfers and asset data is either
non-finite. This section describes the difference between these types.
A push transfer is when the provider data plane initiates sending of asset data to a consumer endpoint. For example, after the consumer has issued an
TransferRequestMessage,the provider begins data transmission to an endpoint specified by the consumer using an agreed-upon wire protocol.
A pull transfer is when the consumer data plane initiates retrieval of asset data from a provider endpoint. For example, after the provider has issued an
TransferProcessStart,message, the consumer can request the data from the provider-specified endpoint.
Asset data may be
non-finite.Finite data is data that is defined by a finite set, for example, machine learning data or images . After finite data transmission has finished, the transfer process is completed. Non-finite data is data that is defined by an infinite set or has no specified end, for example streams or an API endpoint. With non-finite data, a TP will continue indefinitely until either the consumer or provider explicitly terminates the transmission.
The TP states are:
- REQUESTED - An asset has been requested under an
Agreementby the consumer and the provider has sent an ACK response.
- STARTED - The asset is available for access by the consumer or the provider has begun pushing the asset to the consumer endpoint.
- COMPLETED - The transfer has been completed by either the consumer or the provider.
- SUSPENDED - The transfer has been suspended by the consumer or the provider.
- TERMINATED - The transfer process has been terminated by the consumer or the provider.
Sent by: Consumer
Resulting State: REQUESTED
TransferRequestMessageis sent by a consumer to initiate a transfer process.
consumerPidproperty refers to the transfer id on consumer side.
agreementIdproperty refers to an existing contract agreement between the consumer and provider.
dct:formatproperty is a format specified by a
Assetassociated with the agreement. This is generally obtained from the provider
dataAddressproperty must only be provided if the
dct:formatrequires a push transfer.
callbackAddressis a URI indicating where messages to the consumer should be sent. If the address is not understood, the provider MUST return an UNRECOVERABLE error.
Providers should implement idempotent behavior for
TransferRequestMessagebased on the value of
dspace:consumerPid. Providers may choose to implement idempotent behavior for a certain period of time. For example, until a transfer processes has completed and been archived after an implementation-specific expiration period. If a request for the given
dspace:consumerPidhas already been received and the same consumer sent the original message, the provider should respond with an appropriate
Once a transfer process have been created, all associated callback messages must include a
Providers must include a
dspace:providerPidproperty in the
dataAddresscontains a transport-specific endpoint address for pushing the asset. It may include a temporary authorization via the
- Valid states of a
Sent by: Provider
Resulting State: STARTED
TransferStartMessageis sent by the provider to indicate the asset transfer has been initiated.
dataAddressis only provided if the current transfer is a pull transfer and contains a transport-specific endpoint address for obtaining the asset.. It may include a temporary authorization via the
Sent by: Provider or Consumer
Resulting State: SUSPENDED
TransferSuspensionMessageis sent by the provider or consumer when either of them needs to temporarily suspend the data transfer.
Sent by: Provider or Consumer
Resulting State: COMPLETED
TransferCompletionMessageis sent by the provider or consumer when asset transfer has completed. Note that some data plane implementations may optimize completion notification by performing it as part of its wire protocol. In those cases, a
TransferCompletionMessagemessage does not need to be sent.
Sent by: Provider or Consumer
Resulting State: TERMINATED
TransferTerminationMessageis sent by the provider or consumer at any point except a terminal state to indicate the data transfer process should stop and be placed in a terminal state. If the termination was due to an error, the sender may include error information.
TransferProcessis an object returned by a consumer or provider indicating a successful state change happened.
TransferErroris an object returned by a consumer or provider indicating an error has occurred. It does not cause a state transition.
Last modified 1mo ago