# Testsuite

## Set-up

**The Testbed must be set-up and running**. Please follow the instructions in the [installation guide](https://docs.internationaldataspaces.org/ids-knowledgebase/ids-reference-testbed/getting-started-with-ids-reference-testbed/installationguide) and come back.

Please ensure, you have already installed. If not, please use the following command (depending on you OS):

`sudo apt install htmldoc`

In addition, you need a running [NodeJs installation](https://nodejs.org/en/download/).

You install the missing modules by running the following commands (probably with `sudo`):

`npm install -g newman`

`npm install -g newman-reporter-html`

## Configuration file

There are a couple of environmental variables to be set before running the test suite. Because the configuration differs between IDS components, it is necessary to change at least the following values (whereas the others might be common between different IDS components).

### Connector environment variables

Navigate to the file `Testsuite/env/Applicant_IDS_Connector_Test_Configuration.postman_environment.json` and open it in an editor of choice.

Replace the **value of the JSON "value"** at least of the following environment variables - if necessary - so it fits to your IDS Connector configuration:

```json
{
  "key": "APPLICANT_CONNECTOR_PROTOCOL",
  "value": "https",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_CONNECTOR_IP_DNS",
  "value": "connectorb",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_CONNECTOR_IP",
  "value": "localhost",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_CONNECTOR_PORT",
  "value": "8081",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_MAINTAINER",
  "value": "https://sovity.de/",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_SECURITY_PROFILE",
  "value": "https://w3id.org/idsa/code/BASE_SECURITY_PROFILE",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_CONNECTOR_ID",
  "value": "https://connector_B",
  "type": "default",
  "enabled": true
}
```

In addition, please check whether you need to adjust the following values according to your component. The IDS Testsuite uses some default paths to resources known in the IDS, e.g. `/api/ids/data`. If you need to change those too, please change the environment variables accordingly.

Normally, every resource is represented by one environment variable as shown in following extract:

```json
{
  "key": "APPLICANT_FIRST_LEVEL_RESOURCE_NAME",
  "value": "/api",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_IDS_RESOURCE_NAME",
  "value": "/ids",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_DATA_RESOURCE_NAME",
  "value": "/data",
  "type": "default",
  "enabled": true
},
{
  "key": "APPLICANT_CONNECTOR_ACCESS_URL",
  "value": "{{APPLICANT_FIRST_LEVEL_RESOURCE_NAME}}{{APPLICANT_IDS_RESOURCE_NAME}}{{APPLICANT_DATA_RESOURCE_NAME}}",
  "type": "default",
  "enabled": true
}
```

## Execution

Currently, it is possible to run tests again an IDS Connector and IDS Broker. Simply run the following commands and replace $COMPONENT with either "Connector" or "Broker":

`cd Testsuite`

`sh execute.sh $COMPONENT`

The PDF report can be found in the results folder.

## Covered IDS certification criteria (Connector) for Testbed v1.1

Last update: **May 30th, 2022**

|  Criterion  | Covered |                                                     Comment                                                    |
| :---------: | :-----: | :------------------------------------------------------------------------------------------------------------: |
|   COM\_01   |   Yes   |                              implicitly through allowing only TLS v.1.2 and v.1.3                              |
|   COM\_02   |   Yes   |                implicitly through successful connection establishment (requesting DAT everytime)               |
|   COM\_03   |   Yes   |                              implicitly through allowing only TLS v.1.2 and v.1.3                              |
|   USC\_01   |   Yes   |                                                                                                                |
|   USC\_02   |   Yes   |                                                                                                                |
|   INF\_01   |   Yes   |                                     SUT is able to send a self-description                                     |
|   INF\_03   |  Partly | Currently, the sent self description do not define connector operator and log format of data endpoints offered |
|   INF\_04   |   Yes   |                Implicitly covered by successful message exchange (additional error cases later)                |
|   INF\_05   |   Yes   |                                      SUT is able to send messages with DAT                                     |
|   IAM\_01   |   Yes   |             The DAT sent by the SUT is valid and requested everytime the connection is established             |
|   BRK\_01   |   Yes   |                                                                                                                |
|   BRK\_02   |  Partly |                                                                                                                |
|   CR\_3\_1  |   Yes   |                              implicitly through allowing only TLS v.1.2 and v.1.3                              |
| CR\_3\_1\_1 |   Yes   |                              implicitly through allowing only TLS v.1.2 and v.1.3                              |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.internationaldataspaces.org/ids-knowledgebase/ids-reference-testbed/getting-started-with-ids-reference-testbed/testsuite.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.