Guide for Preparing and Validating the Preconfigured Setup
This file explains how to configure connector A and B for the preconfigured setup expected in the user guide. Additionally, it contains instructions how to ensure the correct configuration of those connectors.
All the steps included in this document are automated in a postman collection called "TestBed_Guide.postman_collection.json" available at the IDS_testbed repository.
First, install Postman tool in your environment to automatically execute the provided postman collection.
sudo snap install postman
On success, an output similar to this should be displayed:
It is installed postman (v9/stable) 9.16.0 por Postman, Inc. (postman-inc✓)
In the Activities search bar type Postman and click on the icon to launch the application.
Inside the application click on Import. In the section File click on Upload Files and select the TestbedPreconfiguration.postman_collection.json that is included in the downloaded directory IDS-testbed.
This Postman collection provides the end user with different calls in order to create a full offer at DataspaceconnectorA with all the required dependencies, register connector A and B at Metadata Broker and perform a successful contract negotiation between provider (connector A) and consumer (connector B).
Preconfiguration
Registering Data at connector A
First of all, this section will explain the necessary steps that need to be follow in order to have a data offered at the Connector A of the reference Testbed (Connector A acting as Provider).
The following steps have been extracted from the official documentation guide (https://international-data-spaces-association.github.io/DataspaceConnector/CommunicationGuide/v6/Provider) and have been modified in order to work for the reference Testbed deployment setup.
Setup the Reference Testbed and follow the next steps on Connector A to create a complete resource. Access in a browser https://localhost:8080 and enter in the Swagger UI of connector A.
Register Resource: POST /api/offers
Create a base resource with the following Request Body
{"title":"DWD Weather Warnings","description":"DWD weather warnings for Germany.","keywords": ["DWD" ],"publisher":"https://dwd.com","language":"DE","license":"","sovereign":"https://dwd.com","endpointDocumentation":"","paymentModality":"undefined"}
The response body should give code 201 and should have this structure:
Create a base resource with the following Request body
{"title":"IDS Catalog","description":"This catalog is created from an IDS infomodel catalog."}
The response body should give code 201 and should have this structure:
{"creationDate":"2024-05-29T10:37:36.346+0000","modificationDate":"2024-05-29T10:37:36.346+0000","title":"IDS Catalog","description":"This catalog is created from an IDS infomodel catalog.","additional": {},"_links": {"self": {"href":"https://localhost:8080/api/catalogs/824d97c3-d9e8-404f-8659-e89f3f7bf3fa" },"offers": {"href":"https://localhost:8080/api/catalogs/824d97c3-d9e8-404f-8659-e89f3f7bf3fa/offers{?page,size}","templated":true } }}
Add Offer to Catalog: POST /api/catalogs/{id}/offers
Generate Usage Control Policy: POST /api/examples/policy
Obtain a usage policy for provide access following the documentation (https://international-data-spaces-association.github.io/DataspaceConnector/Documentation/v6/UsageControl#example-endpoint)
Insert the following Request body
{"title":"Example Usage Policy","description":"Usage policy provide access applied","type":"PROVIDE_ACCESS"}
The response body should give code 200 and should have this structure:
Create a rule for provide access usage policy with the following Request body (it is used the provide access usage policy obtained from the previous call).
Create a contract defining the provider url, the start and end dates at the following Request body
{"title":"Contract","description":"This is an example contract","provider":"https://connectora:8080/","start":"2023-10-22T07:48:37.068Z","end":"2028-10-22T07:48:37.068Z"}
The response body should give code 201 and should have this structure:
{"creationDate":"2024-05-29T10:40:57.025+0000","modificationDate":"2024-05-29T10:40:57.025+0000","title":"Contract","description":"This is an example contract","start":"2023-10-22T07:48:37.068+0000","end":"2028-10-22T07:48:37.068+0000","consumer":"","additional": {},"_links": {"self": {"href":"https://localhost:8080/api/contracts/247ef8d0-d053-4069-90bf-e8aab7908435" },"rules": {"href":"https://localhost:8080/api/contracts/247ef8d0-d053-4069-90bf-e8aab7908435/rules{?page,size}","templated":true },"offers": {"href":"https://localhost:8080/api/contracts/247ef8d0-d053-4069-90bf-e8aab7908435/offers{?page,size}","templated":true } }}
Add Rule to Contract Template: POST /api/contracts/{id}/rules
Create an artifact that contains DWD weather data with the following Request body
{"title":"Example artifact with weather data","description":"This is an example artifact that contains information about weather data", "value": "{‘type’:’FeatureCollection’,’features’:[{‘type’:’Feature’,’id’:’Autowarn_Analyse.fid-46a27240_180cc7856b1_-1fb6’,’geometry’:null,’geometry_name’:’THE_GEOM’,’properties’:{‘ID’:1667125,’CREATED’:’2022-05-02T21:55:05.285Z’,’ID_ALERT’:’1651528474710.2’,’SOURCE’:’NowCastMIXAnalysis’,’CATEGORY’:’Met’,’EVENT’:’HBN’,’SEVERITY’:’Unknown’,’EC_II’:’HBN’,’EC_GROUP’:’HEARTBEAT’,’EC_AREA_COLOR’:’255 255 255’,’EFFECTIVE’:’2022-05-02T21:54:34Z’,’ONSET’:’2022-05-02T21:50:00Z’,’EXPIRES’:’2022-05-02T21:55:00Z’,’SENDERNAME’:’propHB’,’HEADLINE’:’Heartbeat Event (NowCastMIX)’,’ALTITUDE’:0,’CEILING’:9842.52}}",
"automatedDownload":true}
The response body should give code 201 and should have this structure:
{"creationDate":"2024-05-29T10:43:02.929+0000","modificationDate":"2024-05-29T10:43:02.929+0000","remoteId":"genesis","title":"Example artifact with weather data","description":"This is an example artifact that contains information about weather data","numAccessed":0,"byteSize":456,"checkSum":1078006432,"additional": {},"_links": {"self": {"href":"https://localhost:8080/api/artifacts/95b43d79-9560-4ca5-8bdf-06dd5d109182" },"data": {"href":"https://localhost:8080/api/artifacts/95b43d79-9560-4ca5-8bdf-06dd5d109182/data" },"representations": { "href": "https://localhost:8080/api/artifacts/95b43d79-9560-4ca5-8bdf-06dd5d109182/representations{?page,size}",
"templated":true },"agreements": {"href":"https://localhost:8080/api/artifacts/95b43d79-9560-4ca5-8bdf-06dd5d109182/agreements{?page,size}","templated":true },"subscriptions": { "href": "https://localhost:8080/api/artifacts/95b43d79-9560-4ca5-8bdf-06dd5d109182/subscriptions{?page,size}",
"templated":true },"route": {"href":"https://localhost:8080/api/artifacts/95b43d79-9560-4ca5-8bdf-06dd5d109182/route" } }}
Representations: POST /api/representations
Create a representation with the following Request body
Register Connector A at MetaData Broker: POST /api/ids/connector/update
At the Swagger UI of Connector A insert the following recipient url for message POST /api/ids/connector/update:
https://broker-reverseproxy/infrastructure
The server response should give code 200.
Register Connector B at MetaData Broker: POST /api/ids/connector/update
At the Swagger UI of the Connector B insert the following recipient url for message POST /api/ids/connector/update:
https://broker-reverseproxy/infrastructure
The server response should give code 200.
Validating Preconfigured Setup: Interaction between Connectors
This section will explain the necessary steps that need to be follow in order to request data from Connector A (acting as data provider) using Connector B (acting as data consumer). The following steps have been extracted from the official documentation guide (https://international-data-spaces-association.github.io/DataspaceConnector/CommunicationGuide/v6/Consumer) and have been modified in order to work for the reference Testbed deployment setup.
With the Reference Testbed setup, follow the next steps on Connector B to obtain data from Connector A. Access in a browser https://localhost:8081 and conduct the following steps in the Swagger UI of connector B.
Request Self-description from Connector A: POST /api/ids/description
Request the self-description of Connector A utilizing POST /api/ids/description.
Insert the recipient url of Connector A
https://connectora:8080/api/ids/data
Left the elementId empty because the id of the requested resource is not known.
The response body should give code 200 and be euqivalent to the following:
Put the connector automatically download data of an artifact to False (download)
false
The request body must contain a contract offer, this contract must match the one the resource was created with. Therefore, it is a policy with a USE permission. The rule list will be automatically turned into a contract request to then send it to the provider. This will read this contract request, compare it to the artifact’s (respectively the corresponding resource’s) contract offers, and return either a ContractRejectionMessage or a ContractAgreementMessage.
Include in it the id of the Connector A rule and as ids:target the artifact.
If you paste it in a browser and download it is obtained the corresponding DWD weather data.
Validating Preconfigured Setup: Interaction with Broker
This section will explain the necessary steps to validate correct setup of the Broker with information regarding Connector A and B and to obtain information about the dataset offered by Connector A. With the Reference Testbed setup, follow the next steps on Connector B to obtain information from the Broker concerning Connector A. Access in a browser https://localhost:8081 and conduct the following steps in the Swagger UI of connector B.
Request Broker Self-Description: POST /api/ids/description
Obtain broker self-description using POST /api/ids/description.
Insert the recipient url
https://broker-reverseproxy/infrastructure
The response body should give code 200 and should have this structure:
{"@context": {"ids":"https://w3id.org/idsa/core/","idsc":"https://w3id.org/idsa/code/" },"@type":"ids:Broker","@id":"https://broker-reverseproxy/","ids:description": [ {"@value":"A Broker with a graph persistence layer","@language":"en" } ],"ids:title": [ {"@value":"IDS Metadata Broker","@language":"en" } ],"ids:hasDefaultEndpoint": {"@type":"ids:ConnectorEndpoint","@id":"https://w3id.org/idsa/autogen/connectorEndpoint/fbcb92e9-084b-4846-a957-f7cd417ccc94","ids:path":"/","ids:accessURL": {"@id":"https://broker-reverseproxy/" },"ids:endpointInformation": [ {"@value":"Dieser Endpunkt liefert eine Selbstbeschreibung dieses IDS Connectors","@language":"de" }, {"@value":"Endpoint providing a self-description of this connector","@language":"en" } ] },"ids:hasEndpoint": [ {"@type":"ids:ConnectorEndpoint","@id":"https://w3id.org/idsa/autogen/connectorEndpoint/002793f6-8d61-468a-8d06-d035c1ed9e77","ids:path":"/infrastructure","ids:endpointDocumentation": [ { "@id": "https://app.swaggerhub.com/apis/idsa/IDS-Broker/1.3.1#/Multipart%20Interactions/post_infrastructure"
} ],"ids:accessURL": {"@id":"https://broker-reverseproxy/infrastructure" },"ids:endpointInformation": [ { "@value": "This endpoint provides IDS Connector and IDS Resource registration and search capabilities at the IDS Metadata Broker.",
"@language":"en" }, { "@value": "Dieser Endpunkt ermöglicht die Registrierung von und das Suchen nach IDS Connectoren und IDS Ressourcen am IDS Metadata Broker.",
"@language":"de" } ] } ],"ids:resourceCatalog": [ {"@type":"ids:ResourceCatalog","@id":"https://w3id.org/idsa/autogen/resourceCatalog/24263dca-47a5-4b95-980e-48f88eff32c9","ids:offeredResource": [ {"@type":"ids:DataResource","@id":"https://w3id.org/idsa/autogen/dataResource/34582161-2e82-4541-b79a-dfd8f2348761","ids:representation": [ {"@type":"ids:DataRepresentation", "@id": "https://w3id.org/idsa/autogen/dataRepresentation/577087e0-b1d5-41c0-8503-19f08b11d34c",
"ids:instance": [ {"@type":"ids:Artifact","@id":"https://broker-reverseproxy/connectors/" } ] } ] } ] } ],"ids:securityProfile": {"@id":"https://w3id.org/idsa/code/BASE_SECURITY_PROFILE" },"ids:maintainer": {"@id":"https://www.iais.fraunhofer.de" },"ids:curator": {"@id":"https://www.iais.fraunhofer.de" },"ids:inboundModelVersion": ["4.0.3" ],"ids:outboundModelVersion":"4.0.3"}
Query List of Connectors to Check Successful Registration: POST /api/ids/description
Query the MetaData Broker for available connectors in the Testbed using message POST /api/ids/description.
Insert the recipient url
https://broker-reverseproxy/infrastructure
Insert the element id of the requested resource
https://broker-reverseproxy/connectors/
The response body should give code 200 and contain both connectors (A and B) in the list of connector as follows:
{"@graph": [ {"@id":"https://broker-reverseproxy/connectors/","@type":"ids:ConnectorCatalog","listedConnector": ["https://broker-reverseproxy/connectors/2129657530","https://broker-reverseproxy/connectors/2129657531" ] }, {"@id":"https://broker-reverseproxy/connectors/2129657530","@type":"ids:BaseConnector","sameAs":"https://connector_A","curator":"https://sovity.de/","description":"IDS Connector A with static example resources", "hasDefaultEndpoint": "https://w3id.org/idsa/autogen/connectorEndpoint/e5e2ab04-633a-44b9-87d9-a097ae6da3cf",
"inboundModelVersion": ["4.2.0","4.2.1","4.1.0","4.0.0","4.2.2","4.2.3","4.2.7","4.1.2","4.2.4","4.2.5","4.2.6" ],"maintainer":"https://sovity.de/","outboundModelVersion":"4.2.7","publicKey":"https://w3id.org/idsa/autogen/publicKey/78eb73a3-3a2a-4626-a0ff-631ab50a00f9","resourceCatalog":"https://broker-reverseproxy/connectors/2129657530/1127754494","securityProfile":"https://w3id.org/idsa/code/BASE_SECURITY_PROFILE","title":"Dataspace Connector","version":"8.0.2" }, {"@id":"https://broker-reverseproxy/connectors/2129657530/1127754494","@type":"ids:ResourceCatalog","sameAs":"https://localhost:8080/api/catalogs/824d97c3-d9e8-404f-8659-e89f3f7bf3fa" }, {"@id":"https://broker-reverseproxy/connectors/2129657531","@type":"ids:BaseConnector","sameAs":"https://connector_B","curator":"https://sovity.de/","description":"IDS Connector B with static example resources", "hasDefaultEndpoint": "https://w3id.org/idsa/autogen/connectorEndpoint/e5e2ab04-633a-44b9-87d9-a097ae6da3cf",
"inboundModelVersion": ["4.2.6","4.0.0","4.2.7","4.2.0","4.1.0","4.2.1","4.2.2",