Installation Guide

IDSA Reference Testbed Installation (CA-DAPS-DSC-MDB)

Index

The installation and configuration process is explained below for each of the components. To further support this document, the links to the official installation guides will be linked.

Execution modes

You may either run the preconfigured testbed offered in this repo or follow the instructions for the manual setup below to set it up on your own and possibly adjust it to your needs.

Hardware Requirements

In this section the minimum requirements required for operating the IDS-testbed are detailed.

The current minimum requirements for the IDS-Testbed are:

  • 4 GB RAM (however 8GB RAM is recommended)

  • 50 GB storage

It is recommended to use 64bit quad core processor to provide enough processing power for all docker containers.

Take into account that if more components are included at the IDS-testbed or a huge amount of data is uploaded it is possible to run out of disk free space. In this cases it is recommended to provide more free disk storage.

Target View: Preconfigured testbed

Follow this section to automatically launch the Preconfigured set up of the Testbed.

Setting up requirements

The software required for the successful deployment of the testbed is the following:

  • OS: Ubuntu 22.04.5 LTS

  • Docker: 20.10.7

  • Docker-compose: 1.27.4

First, verify your ubuntu version

the output should be similar to this

Then update your system with

Install docker and docker-compose

verify install with

The output should look similar to

The output should look similar to

If your docker-compose version is not the required one, execute the following command:

Download the IDS-testbed to your local environment.

Move to the downloaded directory and execute the docker-compose.yml script.

If you face problems with docker or user right, execute the following commands and log out and back so that your group membership is re-evaluated.

Re-execute the docker-compose.yml script.

The process of downloading the images and launching the containers of the different components (DAPS, DSC and MDB) will begin.

The IDS-testbed will be correctly deployed. The components that are part of the IDS-testbed can be reached at the URLs mentioned below.

DAPS:

  • can be reached at https://localhost:443

  • needs to be preconfigured to know connector A, B and the Broker

Connectors:

  • connector A

    • can be reached at https://localhost:8080

    • needs to be preconfigured with a self-description and offering a dataset ("hallo world")

  • connector B

    • can be reached at https://localhost:8081

    • needs to be preconfigured with a self-description and offering a dataset ("goodbye world")

Broker:

  • can be reached at https://localhost:444

  • needs to be aware of connector A, connector B and store their self-descriptions

Target View: Manual testbed set up

Follow this section to manually launch the Testbed.

Setting up requirements

The software required for the successful deployment of the testbed is the following:

  • OS: Ubuntu 20.04.1 LTS

  • Docker: 20.10.7

  • Docker-compose: 1.27.4

  • Java: 11

  • Maven: 3.6.3

  • Ruby: 2.7.0

  • Python3

First, verify your ubuntu version

the output should be similar to this

Then update your system with

docker and docker-compose

As we need to run different components at the same time, install docker and docker-compose

verify install with

The output should look similar to

In some environments, e.g. WSL2, you might have to start the docker daemon manually by calling dockered .

The output should look similar to

Java and maven

Some components like the Data Space Connector require Java 11. Install it with

verify install with

The output should look similar to

To avoid problems while building components you should set the JAVA_HOMEenvironment variable on your system.

HINT: you might want to check with jrunscript -e 'java.lang.System.out.println(java.lang.System.getProperty("java.home"));'

To enable the build process of the Data Space connector we install maven

verify install with

The output should look similar to

Python

The CA script provided with the IDS testbed requires python. Install it with

Ruby

The Omejdn Daps runs on Ruby. Setup Ruby by calling

Other tools

Some additional tools that might be useful

Installation of the components

First, let us set up the network with

Download the Testbed

CERTIFICATE AUTHORITY

Move to right directory, and make the files executable:

Follow the documentation detailed at the "README.md" file which covers the scope, dependencies and usage of the component.

The preconfigured setup includes certificates for:

  • a root CA called "ReferenceTestbedCA"

  • a subCA called "ReferenceTestbedSubCA"

  • certificates for devices called "connectorA", "connectorB", "broker", "daps" and "connectorA_revoked"

which are located at CertificateAuthority/data-cfssl folder.

Continue here after the official documentation has been followed

The Certificate Authority provides {CERT_FILENAME}.crt and {CERT_FILENAME}.key formats. Keep in mind that other formats will be required for the different components. Those have to be created.

Now convert the generated certificates in data/cert using openssl to then use in the Connector and DAPS.

The output should look similar to

Obtain a .p12 file format from the current .crt and .key formats:

You should now have two additional files in data-cfssl/cert

The certificate chain (CA, SubCA, Certs) has been created and the user should be able to create as many certificates as they need for their environment.

DAPS

The official documentation of the Omejdn DAPS is here: https://github.com/International-Data-Spaces-Association/omejdn-daps

Adding the keys to the DAPS

Every client that wants to use the local Omejdn DAPS must place their {CERTFILE}.cert file in the keys directory.

The directory can be found in

Add the certificate provided by the local CA, newly created by the local CA or provided by Fraunhofer AISEC. Place the certificate at the folder DAPS/keys/omejdn/ with name omejdn.key to avoid dependency issues later on.

Adding the clients to the DAPS

Note: The user must execute the register_connector.sh file in order to add the client to the Omejdn DAPS. Once executed, the certificate will be included in the DAPS's list of clients.

To execute the script

It could look something like this

The certificate will be added to the list of DAPS's clients. You can check it at the file DAPS/config/clients.yml

Required changes in the configuration

Change the configuration file .env with your favorite editor, e.g. nano.

Note The file could be hidden. Select the option show hidden files and it should be placed at IDS-testbed root directory.

Replace the following lines with the necessary configuration. It could look something like this

Configure the docker-compose.yml file with your configuration. Then run the Omejdn DAPS server.

The docker-compose.yml could look something like this

Place the local CA created certificate at the folder DAPS/keys/TLS/ and name it as daps.crt and daps.key to match the above mentioned docker-compose.yml file configuration.

DATASPACE CONNECTOR:

The testbed will have two built-in Connectors. They will be referred to as ConnectorA and ConnectorB. They will have different configurations, so they will each have their own directory. These directories are going to be referred to as DataspaceConnectorA and DataspaceConnectorB.

It is recommended to follow the guide with one Connector at a time to avoid configuration issues.

Make sure you are in the right directory:

or

Component Documentation

The official documentation will cover the introductions, deployment, documentation and communication guide of the component.

Official documentation: https://github.com/International-Data-Spaces-Association/DataspaceConnector/tree/v8.0.2

Continue here after reading the official documentation

Official configuration documentation: https://international-data-spaces-association.github.io/DataspaceConnector/Deployment/Configuration#configuration

The Dataspace Connector must be configured to work in this environment.

Define the PostgreSQL containers

Define the PostgreSQL container to be used by DataspaceConnectorA and DataspaceConnectorB. For the IDS-testbed deployment it is configured at the docker-compose.yml file.

It could look something like this (ConnectorA)

It could look something like this (ConnectorB)

Changes to the application.properties file

The configuration necessary for the application properties is located at the src/main/resources/application.properties folder of the official DSC repository.

For the IDS-testbed deployment it is configured at the docker-compose.yml. Here it is detailed the port, daps configuration and the server ssl keystore. It is also defined the PostgreSQL database setup.

The server server.ssl.key-store=file:///config/{TLS_FILENAME}.p12, where {TLS_FILENAME} is to be replaced with the certificate created previously by the local CA. The Dataspace Connector expects the TLS certificate in .p12 format.

Note Make sure the created certificates have the correct permissions. For the Dataspace Connector this .p12 format certificate must be configured with read and write rights for the user permissions and group permissions. The file permissions can be viewed and changed using the following commands:

Changes to the config.json file

Use nano or your most favourite editor

Deployment Mode

Edit connectorDeployMode from TEST_DEPLOYMENT to PRODUCTIVE_DEPLOYMENT for the connector to request and validate incoming DATs

Dataspace Connector KeyStore

{CERT_FILENAME} will be a certificate from the local CA or external to this testbed, provided by Fraunhofer AISEC (Contact Gerd Brost).

Ensure {CERT_FILENAME} are different for ConnectorA and ConnectorB

Note: Local CA certs will be available. Users can use those, create new ones or bring their own FH cert to replace {CERT_FILENAME}.

Open the conf directory

Ensure the {CERT_FILENAME}.p12 file used for ids:keyStore is placed in this directory for the config.json to access it

Modify the truststore.p12 for the Connector to accept these new TLS certificates. Make sure the {TLS_FILENAME}.crt is in this directory and then

It could look something like this (ConnectorA)

It could look something like this (ConnectorB)

It could look something like this (Metadata Broker)

It could look something like this (Omejdn DAPS)

You will be asked the following in the terminal:

  • Enter keystore password: , type password

  • Trust this certificate? [no]: , type yes

It should return:

Ensure both connectorA.crt and connectorB.crt are imported into the truststore.p12

When using the DSC for clarity reasons modify the following lines

Put a meaningful description to your connector

Put a meaningful URL that uniquely identifies your connector towards the IDS Metadata Broker.

It could look something like this (ConnectorA)

It could look something like this (ConnectorB)

Additional Changes

For the use of this testbed, the Dataspace Connector must be built via docker-compose.

The testbed is run in a docker network defined earlier in this document called testbed_local.

Configure the docker-compose.yml file with your configuration. The docker-compose.yml could look something like this for the DataspaceConnectorA.

DSC will not fly without a DAPS token now. Make sure the DAPS runs first.

METADATA BROKER

Component Documentation

The official documentation will cover the pre-requisites, installation and deployment of the component.

Official documentation: https://github.com/International-Data-Spaces-Association/metadata-broker-open-core

Continue here after reading the official documentation

Download the component from the official repository

Use the downloaded component to build the broker-core image.

Changes to the application.properties file

Use nano or your most favourite editor.

DAPS

This will make use of the locally installed DAPS.

Add the local DAPS to the trusted hosts

Changes to the component's keystore

At the folder broker-core/src/main/resources/ add the certificate provided by the local CA, newly created by the local CA or provided by Fraunhofer AISEC. If it is NOT provided by the local CA, make sure it is correctly added to the local DAPS.

It could look something like this

Expected outcome:

To check the content of the created keystore, use the following command:

It could look something like this

Build the broker-core image

Go to the main directory and build the project with maven:

This will create a .jar file in broker-core/target that will have to be copied into docker/broker-core.

Once the file is copied, move to the docker/broker-core directory and place there the TLS certificate that corresponds to the DAPS. For the IDS-testbed it is located at DAPS/keys/TLS/daps.cert and use the following command to change the certificate format to daps.crt

Then build the core image locally using the following command.

Adding the TLS certificates

At the IDS-testbed/MetadataBroker/ folder place the TLS certificates created by the local CA together with the keystore.

  • server.crt

  • server.key

  • isstbroker-keystore.jks

Usage

Take the content from the file metadata-broker-open-core/docker/composefiles/broker-localhost/docker-compose.yml and copy it at your docker-compose.yml file. Use nano or your most favourite editor.

Use the TLS certificates and ensure the container names are consistent with other dependencies by adding container_name:.

If port 443 is already in use, the reverseproxy container will exit with code 1. Follow the steps in the next block to get around this:

Go to the compose file and build the Metadata Broker

Stop and delete testbed set up

In this section it is detailed how to automatically stop and delete IDS-testbed set up.

Be aware that following this section will stop and delete all the containers launched by the IDS-testbed. It will also remove the docker-compose.yml file configuration regarding the images of the different components and the associated docker volumes and network.

Move to your IDS-testbed downloaded directory and execute the following command

This is the expected outcome

As seen above, the containers have been stopped and deleted. The network, volumes and images used by the docker-compose.yml file have also been deleted.

Last updated