Installation Guide
IDSA Reference Testbed Installation (CA-DAPS-DSC-MDB)
Index
The installation and configuration process is explained below for each of the components. To further support this document, the links to the official installation guides will be linked.
Execution modes
You may either run the preconfigured testbed offered in this repo or follow the instructions for the manual setup below to set it up on your own and possibly adjust it to your needs.
Hardware Requirements
In this section the minimum requirements required for operating the IDS-testbed are detailed.
The current minimum requirements for the IDS-Testbed are:
4 GB RAM (however 8GB RAM is recommended)
50 GB storage
It is recommended to use 64bit quad core processor to provide enough processing power for all docker containers.
Take into account that if more components are included at the IDS-testbed or a huge amount of data is uploaded it is possible to run out of disk free space. In this cases it is recommended to provide more free disk storage.
Target View: Preconfigured testbed
Follow this section to automatically launch the Preconfigured set up of the Testbed.
Setting up requirements
The software required for the successful deployment of the testbed is the following:
OS: Ubuntu 20.04.1 LTS
Docker: 20.10.7
Docker-compose: 1.27.4
First, verify your ubuntu version
the output should be similar to this
Then update your system with
Install docker and docker-compose
verify install with
The output should look similar to
The output should look similar to
If your docker-compose version is not the required one, execute the following command:
Download the IDS-testbed
to your local environment.
Move to the downloaded directory and execute the docker-compose.yml
script.
If you face problems with docker or user right, execute the following commands and log out and back so that your group membership is re-evaluated.
Re-execute the docker-compose.yml
script.
The process of downloading the images and launching the containers of the different components (DAPS, DSC and MDB) will begin.
The IDS-testbed will be correctly deployed. The components that are part of the IDS-testbed can be reached at the URLs mentioned below.
DAPS:
can be reached at https://localhost:443
needs to be preconfigured to know connector A, B and the Broker
Connectors:
connector A
can be reached at https://localhost:8080
needs to be preconfigured with a self-description and offering a dataset ("hallo world")
connector B
can be reached at https://localhost:8081
needs to be preconfigured with a self-description and offering a dataset ("goodbye world")
Broker:
can be reached at https://localhost:444
needs to be aware of connector A, connector B and store their self-descriptions
Target View: Manual testbed set up
Follow this section to manually launch the Testbed.
Setting up requirements
The software required for the successful deployment of the testbed is the following:
OS: Ubuntu 20.04.1 LTS
Docker: 20.10.7
Docker-compose: 1.27.4
Java: 11
Maven: 3.6.3
Ruby: 2.7.0
Python3
First, verify your ubuntu version
the output should be similar to this
Then update your system with
docker and docker-compose
As we need to run different components at the same time, install docker and docker-compose
verify install with
The output should look similar to
In some environments, e.g. WSL2, you might have to start the docker daemon manually by calling
dockered
.
The output should look similar to
Java and maven
Some components like the Data Space Connector require Java 11. Install it with
verify install with
The output should look similar to
To avoid problems while building components you should set the
JAVA_HOME
environment variable on your system.HINT: you might want to check with
jrunscript -e 'java.lang.System.out.println(java.lang.System.getProperty("java.home"));'
To enable the build process of the Data Space connector we install maven
verify install with
The output should look similar to
Python
The CA script provided with the IDS testbed requires python. Install it with
Ruby
The Omejdn Daps runs on Ruby. Setup Ruby by calling
Other tools
Some additional tools that might be useful
Installation of the components
First, let us set up the network with
Download the Testbed
CERTIFICATE AUTHORITY
Move to right directory, and make the files executable:
Follow the documentation detailed at the "README.md" file which covers the scope, dependencies and usage of the component.
The preconfigured setup includes certificates for:
a root CA called "ReferenceTestbedCA"
a subCA called "ReferenceTestbedSubCA"
certificates for devices called "connectorA", "connectorB", "broker", "daps" and "connectorA_revoked"
which are located at CertificateAuthority/data-cfssl
folder.
Continue here after the official documentation has been followed
The Certificate Authority provides {CERT_FILENAME}.crt and {CERT_FILENAME}.key formats. Keep in mind that other formats will be required for the different components. Those have to be created.
Now convert the generated certificates in data/cert
using openssl to then use in the Connector and DAPS.
The output should look similar to
Obtain a .p12
file format from the current .crt
and .key
formats:
You should now have two additional files in data-cfssl/cert
The certificate chain (CA, SubCA, Certs) has been created and the user should be able to create as many certificates as they need for their environment.
DAPS
The official documentation of the Omejdn DAPS is here: https://github.com/International-Data-Spaces-Association/omejdn-daps
Adding the keys to the DAPS
Every client that wants to use the local Omejdn DAPS must place their {CERTFILE}.cert
file in the keys
directory.
The directory can be found in
Add the certificate provided by the local CA, newly created by the local CA or provided by Fraunhofer AISEC. Place the certificate at the folder DAPS/keys/omejdn/
with name omejdn.key
to avoid dependency issues later on.
Adding the clients to the DAPS
Note: The user must execute the register_connector.sh
file in order to add the client to the Omejdn DAPS. Once executed, the certificate will be included in the DAPS's list of clients.
To execute the script
It could look something like this
The certificate will be added to the list of DAPS's clients. You can check it at the file DAPS/config/clients.yml
Required changes in the configuration
Change the configuration file .env
with your favorite editor, e.g. nano
.
Note The file could be hidden. Select the option show hidden files
and it should be placed at IDS-testbed root directory.
Replace the following lines with the necessary configuration. It could look something like this
Configure the docker-compose.yml
file with your configuration. Then run the Omejdn DAPS server.
The docker-compose.yml
could look something like this
Place the local CA created certificate at the folder DAPS/keys/TLS/
and name it as daps.crt
and daps.key
to match the above mentioned docker-compose.yml
file configuration.
DATASPACE CONNECTOR:
The testbed will have two built-in Connectors. They will be referred to as ConnectorA and ConnectorB. They will have different configurations, so they will each have their own directory. These directories are going to be referred to as DataspaceConnectorA
and DataspaceConnectorB
.
It is recommended to follow the guide with one Connector at a time to avoid configuration issues.
Make sure you are in the right directory:
or
Component Documentation
The official documentation will cover the introductions, deployment, documentation and communication guide of the component.
Official documentation: https://github.com/International-Data-Spaces-Association/DataspaceConnector/tree/v8.0.2
Continue here after reading the official documentation
Official configuration documentation: https://international-data-spaces-association.github.io/DataspaceConnector/Deployment/Configuration#configuration
The Dataspace Connector must be configured to work in this environment.
Define the PostgreSQL containers
Define the PostgreSQL container to be used by DataspaceConnectorA and DataspaceConnectorB. For the IDS-testbed deployment it is configured at the docker-compose.yml
file.
It could look something like this (ConnectorA)
It could look something like this (ConnectorB)
Changes to the application.properties file
The configuration necessary for the application properties is located at the src/main/resources/application.properties
folder of the official DSC repository.
For the IDS-testbed deployment it is configured at the docker-compose.yml
. Here it is detailed the port, daps configuration and the server ssl keystore. It is also defined the PostgreSQL database setup.
The server server.ssl.key-store=file:///config/{TLS_FILENAME}.p12
, where {TLS_FILENAME}
is to be replaced with the certificate created previously by the local CA. The Dataspace Connector expects the TLS certificate in .p12
format.
Note Make sure the created certificates have the correct permissions. For the Dataspace Connector this .p12
format certificate must be configured with read and write rights for the user permissions
and group permissions
. The file permissions can be viewed and changed using the following commands:
Changes to the config.json file
Use nano or your most favourite editor
Deployment Mode
Edit connectorDeployMode
from TEST_DEPLOYMENT
to PRODUCTIVE_DEPLOYMENT
for the connector to request and validate incoming DATs
Dataspace Connector KeyStore
{CERT_FILENAME} will be a certificate from the local CA or external to this testbed, provided by Fraunhofer AISEC (Contact Gerd Brost).
Ensure {CERT_FILENAME} are different for ConnectorA and ConnectorB
Note: Local CA certs will be available. Users can use those, create new ones or bring their own FH cert to replace {CERT_FILENAME}.
Open the conf
directory
conf
directoryEnsure the {CERT_FILENAME}.p12 file used for ids:keyStore
is placed in this directory for the config.json
to access it
Modify the truststore.p12
for the Connector to accept these new TLS certificates. Make sure the {TLS_FILENAME}.crt is in this directory and then
It could look something like this (ConnectorA)
It could look something like this (ConnectorB)
It could look something like this (Metadata Broker)
It could look something like this (Omejdn DAPS)
You will be asked the following in the terminal:
Enter keystore password:
, typepassword
Trust this certificate? [no]:
, typeyes
It should return:
Ensure both connectorA.crt and connectorB.crt are imported into the truststore.p12
When using the DSC for clarity reasons modify the following lines
Put a meaningful description to your connector
Put a meaningful URL that uniquely identifies your connector towards the IDS Metadata Broker.
It could look something like this (ConnectorA)
It could look something like this (ConnectorB)
Additional Changes
For the use of this testbed, the Dataspace Connector must be built via docker-compose.
The testbed is run in a docker network defined earlier in this document called testbed_local
.
Configure the docker-compose.yml
file with your configuration. The docker-compose.yml
could look something like this for the DataspaceConnectorA.
DSC will not fly without a DAPS token now. Make sure the DAPS runs first.
METADATA BROKER
Component Documentation
The official documentation will cover the pre-requisites, installation and deployment of the component.
Official documentation: https://github.com/International-Data-Spaces-Association/metadata-broker-open-core
Continue here after reading the official documentation
Download the component from the official repository
Use the downloaded component to build the broker-core image.
Changes to the application.properties file
Use nano or your most favourite editor.
DAPS
This will make use of the locally installed DAPS.
Security-related
Add the local DAPS to the trusted hosts
Changes to the component's keystore
At the folder broker-core/src/main/resources/
add the certificate provided by the local CA, newly created by the local CA or provided by Fraunhofer AISEC. If it is NOT provided by the local CA, make sure it is correctly added to the local DAPS.
It could look something like this
Expected outcome:
To check the content of the created keystore, use the following command:
It could look something like this
Build the broker-core image
Go to the main directory and build the project with maven
:
This will create a .jar
file in broker-core/target
that will have to be copied into docker/broker-core
.
Once the file is copied, move to the docker/broker-core
directory and place there the TLS certificate that corresponds to the DAPS. For the IDS-testbed it is located at DAPS/keys/TLS/daps.cert
and use the following command to change the certificate format to daps.crt
Then build the core
image locally using the following command.
Adding the TLS certificates
At the IDS-testbed/MetadataBroker/
folder place the TLS certificates created by the local CA together with the keystore.
server.crt
server.key
isstbroker-keystore.jks
Usage
Take the content from the file metadata-broker-open-core/docker/composefiles/broker-localhost/docker-compose.yml
and copy it at your docker-compose.yml file. Use nano or your most favourite editor.
Use the TLS certificates and ensure the container names are consistent with other dependencies by adding container_name:
.
If port 443 is already in use, the reverseproxy
container will exit with code 1. Follow the steps in the next block to get around this:
Go to the compose file and build the Metadata Broker
Stop and delete testbed set up
In this section it is detailed how to automatically stop and delete IDS-testbed set up.
Be aware that following this section will stop and delete all the containers launched by the IDS-testbed. It will also remove the docker-compose.yml
file configuration regarding the images of the different components and the associated docker volumes and network.
Move to your IDS-testbed
downloaded directory and execute the following command
This is the expected outcome
As seen above, the containers have been stopped and deleted. The network, volumes and images used by the docker-compose.yml
file have also been deleted.
Last updated