Specification Deployment

i3-MARKET Artifacts overview

So far, the i3-MARKET artifacts involved in the deployment have been classified into the following categories:

  • Third-party artifacts (black): these artifacts are open-source dependencies. In I3-MARKET R1 have been identified the following:
    • Hyperledger Besu (blockchain network)
    • CockroachDB (distributed storage)
    • RocksDB (ledger)
    • Loopback4 (i3-MARKET backplane API)
  • I3-MARKET artifacts (green): SW pieces to be developed in the context of i3-MARKET, mainly provided in Wp3 and Wp4.
  • I3-MARKET SDK artifacts (orange): SDK software to be developed in the context of i3-MARKET and deployed in the external actor’s side.

A detailed list of i3-MARKET artifacts including artifact ID, artifact name, artifact deployment responsible, artifact deployment host/s, artifact dependencies, artifact deployment status for R1, artifact deployment status for Mid Review and additional info as public deployment endpoints is shown in Table 1. i3-MARKET artifacts:

Artifact IdArtifactHostsDependencies
A1Blockchain FrameworkallDecentralized Storage
A2CockroachDB (Distributed Storage)all
A3Decentralized StorageallIncluded with the Blockchain framework
A4User-centric Authenticationall
A5Service-centric Authenticationall
A6HW WalletPilot Host
A7Cloud WalletallCloud Wallet Client, Backplane API (Cloud Wallet Server, User-centric Authentication), Data Access SDK, I3-Market SDK
A8Smart Contract Managerall
A9SLA/SLE Smart Contractall
A10Conflict Resolutionall
A11Explicit-User ConsentallBackplane API (Smart Contract Manager, Distributed Ledger, Distributed Storage)
A12Auditable Accountingall
A13Standard PaymentallBackplane API (Auditable Accounting, Conflict Resolution, Smart Contract, SLA/SLE Smart Contract)
A14TokenizationallBackplane API (User-centric Authentication, Smart Contract, SLA/SLE Smart Contract)
A15Micro Paymentall
A16Data Access SDKPilot Hosts
A17Data Acess APIall
A18Semantic Data Manager (triplestore)Node4
A19Semantic Model ManagementNode4
A20Semantic Orchestration (Offering and Discovery)Node4Backplane API (users ids), Decentralized storage
A21Backplane APIall
A22i3-Market SDK-GeneratorNode1
A23Ansible AWXNode2
A24Docker Swarm Clusterall
A26SDK-RI (Reference Implementation)all

A29Secure Server (keycloak)Node4
A30 Notification ManagerAll nodesSDK-RI, sdk-core
Table 1 – i3-MARKET artifacts

Deployment architecture view

Cluster Infrastructure

Figure 1. i3-MARKET deployment diagram (R1) shows the deployment diagram associated to i3-MARKET ecosystem for the first report period. The main deployment decisions behind are the following:

  • The i3-Market cluster consists of 4 nodes in which the following artifacts have been deployed:
    • Hyperledger Besu (blockchain network node). I3-MARKET private blockchain network requires 4 Besu nodes to support BFT consensus approach.
    • CocroachDB (distributed storage instance). CocroachDB has been selected as technology to support distributed storage in I3-MARKET.
    • Loopback4 (backplane API instance). Looopback4 is the technology supporting backplane API.

In addition to that, it should be considered that:

  • The existing pilot’s demonstrator will adhere to one of the four nodes of the cluster. The loopback instances, deployed in the cluster, will be mapped to each one of the existing data markets which are: AGORA, MindSphere and IBM DM.
  • All semantics artifacts will be deployed to each one 4 nodes conforming the cluster.
  • Security artifacts behind the security server will be deployed on a physically decoupled node.
  • Finally, I3-MARKET R2 and R3 will be updated properly to consolidate the strategy of progressively moving the main SW artifacts alongside the data market.
Figure 1 – i3-MARKET deployment diagram (R1)

Taking into account the existing funding for the i3-MARKET common infrastructure provisioning, and trying to optimize said funding as well as balance the load of the cluster nodes, the original deployment diagram was updated, which resulted in the diagram shown in Figure 2. Deployment diagram R1 update.

Figure 2 – Deployment diagram R1 update

To sum up, it is important to mention that the number of nodes was reduced from 4 to 6 in order to optimize funding and be able to maintain the infrastructure until the end of the project. NGINX has been adopted as a reverse proxy and load balancer.

System requirements

To elaborate the deployment plan and identify the SW artifacts list, dependencies, SW and HW requirements and restrictions, a template was circulated to the partners. For further details about this template go to : SW and HW requirements.

An overview of the main capabilities required for the i3-MARKET SW artifacts are shown in Figure 3. i3M physical nodes tentative approach as a tentative approach.

Figure 3 – i3M physical nodes tentative approach

Final approach to be considered for i3-MARKET R1 is shown in Figure 4. i3M physical nodes final approach. A detailed description of capabilities for each one of the required artifacts for R1 is provided in : SW and HW requirements.

Figure 4 – i3M physical nodes final approach


The I3-MARKET Cloud SW stack is represented in Figure 5. I3M Cloud SW stack and it is composed of four layers: Cloud provisioning and management layer, DevOps SW layer, Third-party SW layer and I3M SW layer.

Figure 5 – i3M Cloud SW stack

Cloud provisioning and management layer oversees providing and managing all physical nodes that the i3-MARKET common infrastructure is composed of. ATOS will be leading the provision of the cloud layer.

For the management of physical resources in a homogeneous way, an Ansible Tower1 instance will be deployed for the administration of said physical resources, thus having their management centralized from Ansible.

The DevOps layer will be led by SIEMENS and the concrete DevOps approach as well as the concrete DevOps stack for i3-MARKET will be specified in internal deliverable I2.41 Error: No se encuentra la fuente de referencia. I3-MARKET DevOps will be a set of practices that will combine software development and IT operations and it will aim to shorten the I3-MARKET system development life cycle and provide continuous delivery with high software quality.

Third-party SW layers will be managed by GFT, UPC and Guardtime and it will be mainly in charge of providing the SW stack identified as SW requirements by i3-MARKET system. These SW requirements are: Hyperledger Besu, CocroachDB, Loopback4 and Keycloak.

Finally, i3-MARKET SW layer will be composed by all i3-MARKET artifacts. Section I3-MARKET SW layers provides a detailed view about SW components, associated build blocks, physical resources assigned, deployment owners, artifacts type and technologies.

1 Ansible Tower: https://www.ansible.com/products/tower

i3-MARKET SW Layers

Tables in current section collect all the relevant information associated with the deployment of all the artifacts that are part of the four layers of the i3-MARKET cloud stack and that were identified in section I3-MARKET Artifacts Overview. Therefore, for each i3-MARKET artifact the following information is provided:

  • Artifact name or SW component,
  • Associated building block (see internal deliverable I2.41 Error: No se encuentra la fuente de referencia),
  • Assigned-VM or physical resource (PR),
  • Artifact deployment owner,
  • Artifact type, and finally,
  • Technology supporting artifact.

Generally, meaning of the colours in the tables are the following:

  • Artifacts developed, deployed, and running for R1: blue colour,
  • Artifacts developed, deployed, and running for the first report period (R1 + half R2): green colour,
  • Artifacts developed, deployed, and running for the following periods: orange colour.

i3-MARKET proprietary Software

i3-MARKET SW layer is composed by the following i3-MARKET artifacts shown in Table 2. SW artifacts deployment information.:

User-centric AuthenticationTrust, Security and Privacyi3-MARKET SWKeycloak
Service-centric AuthenticationTrust, Security and Privacyi3-MARKET SWOIDC, VC
Cloud WalletTrust, Security and Privacyi3-MARKET SW
SW WalletTrust, Security and Privacyi3-MARKET SW
HW WalletTrust, Security and Privacyi3-MARKET SW
Smart Contract ManagerTrust, Security and Privacyi3-MARKET SW
Conflict ResolutionTrust, Security and Privacyi3-MARKET SW
Explicit user-concernTrust, Security and Privacyi3-MARKET SW
Auditable AccountingTrust, Security and Privacyi3-MARKET SW
Standard paymentsTrust, Security and Privacyi3-MARKET SWLibrary to be integrated and deployed in Data Access SDK
Micro-paymentsTrust, Security and Privacyi3-MARKET SW
TokenizationTrust, Security and Privacyi3-MARKET SW
Data Access SDK—-i3-MARKET SWSdk-Generator
Data Access APIData Accessi3-MARKET SW
Semantic Storage (Triple store)Semanticsi3-MARKET SW + Third P. SWVirtuoso
Metadata ManagementSemanticsi3-MARKET SWVirtuoso
Semantic Orchestration (Offering and Discovery)Semanticsi3-MARKET SWVirtuoso
Backplane APIBackplanei3-MARKET SWLoopBack4
i3-MARKET SDKReference Implementationi3-MARKET SWSdk-Generator
Notification MangerBackplanei3-MARKET SWRabittMQ
Table 2 – SW artifacts deployment information

Third-party Software

Third-party SW layer managed mainly by UPC, Guardtime and NUIG is composed by the following software artifacts shown in Table 3. Third-party i3-MARKET artifacts:

Blockchain FrameworkBlockchain networkThird party SWHyperledger Besu
Distributed StorageData StorageThird party SWCocroachDB
Decentralized StorageData StorageThird party SWRocksDB.
Security ServerTrust, Security and PrivacyThird party SWKeycloack
Semantic StorageSemanticsi3-MARKET SW + Third P. SWOpen Virtuoso
Table 3 – Third-party i3-MARKET artifacts

DevOps Software

The DevOps layer leaded by ATOS and SIEMENS combines software development and IT operations and it aims to shorten the I3-MARKET system development life cycle and provide continuous delivery with high software quality by means of the following artifacts listed in Table 4. DevOps related SW artifacts.:

Ansible AWXDeploymentThird party SWAnsible AWX
Docker SwarmDeploymentThird party SWDocker Swarm
Gitlab CI/CD (Runners)Ci/CDThird party SWGitLab
NexusCi/CDThird party SWNexus
NGINXManagement/ SecurityThird party SWNGinx
MKDocsDocumentationThird party SWMkDocs
Table 4 – DevOps related SW artifacts

Cloud Management and Monitoring

Finally, for the management and monitoring of physical resources in a homogeneous way the following artifacts shown in Table 5. Cloud artifacts. have been considered:

Ansible AWXDeploymentThird party SWAnsible AWX
ZabbixMonitoringThird party SWZabbix
Table 5 – Cloud artifacts

Deployment Guides

Artifact Deployment Guides

The target audience are the i3-MARKET project developers who are participating in the development and deployment of the i3-MARKET backplane.

The i3-MARKET operative considers four possible deployment scenarios, categorized into manual and automatized deployments. These scenarios are the following:

  • Manual deployment scenario one (MDS1)
  • Automatized deployment scenario with Ansible (ADS1)
  • Automatized deployment scenario with Ansible and GitHub CI/CD (ADS2)
  • Automatized deployment scenario with Docker Compose (ADS3)

Considering an i3-MARKET user role perspective, the main roles involved in the different deployment scenarios are:

  • i3M Root Instance Admin,
  • i3M SW developer, and
  • i3M Third-party SW admin.
  • i3M Pilot Instance Admin

Pilot Environment Deployment Guide

The target audience is the i3-MARKET marketplaces which want to join the i3-MARKET ecosystem and need to deploy the decentralized services (“Pilot Environment” infrastructure) to interact with other marketplaces.

To this purpose, i3-MARKET has defined an operative consisting of an automated deployment scenario with Docker Compose (ADS3). This operative is fully described in 5.1.4 and it is the last role, i3M Pilot Instance Admin, who can benefit the most of this operative.

Tagging releases strategy

i3-MARKET has evolved into a complex system where a large number of pieces must interact together for a comprehensive and integrated performance. Therefore, the different versions released by each single component/microservice should be managed and controlled to avoid incompatibilities in the deployments. 

A strategy based on tagging and a compatibility matrix has been defined to deal with the release’s compatibility.

End-User Usage Operative Specification

Developers Guide

  • Identity Provider API
  • Verify Credentials API
  • Wallet API
  • Data Access API
  • Backplane API
  • SDK-RI


  • Auditable Accounting Library
  • Wallet Client Library
  • i3-Market’s SDK Core

Pilots Administration Guide

During the last year, the pilots have deployed their own instances of i3-MARKET in their  respective environments. Therefore, pilots administration is rather specific. More details on the pilots deployment can be found in deliverable D5.4. With respect to security, each pilot infrastructure has its own OIDC, VC and Keycloak services which will perform all the user authentication and authorization process under the own policies defined by a marketplace. Web-RI as a reference implementation of i3-MARKET functionalities facilitates the setup of OIDC.

Central Administration Guide

Cloud management

In this section an approach is presented for successfully deploying, configuring, and monitoring centralized core services of i3-MARKET. This approach is based on the usage of Ansible Tower[1] as key pillar for managing the cloud resources. With Ansible Tower we can control the i3-MARKET central infrastructure (see section 4.1 for more details) with a visual dashboard, role-based access control, job scheduling, integrated notifications, and graphical inventory management. The Ansible Tower dashboard is shown in the figure below.

Regarding last version of i3-MARKET, the proposed approach is based on the definition of a physical resource inventory in Ansible, in order to be able to automate the deployments of central artifacts (listed in section 4.1). In line with the deployment plan presented in section 4, the i3-MARKET physical inventory will be composed of physical resources, whose nomenclature based on section 4.1 tables (column physical resource) is the following:

  • I3M-PH-Node1, I3M-PH-Node2, I3M-PH-Node3: These three nodes contain three different instances of i3-MARKET that act as development environments and testing purposes for the i3-MARKET developers.
  • I3M-PH-Node4: Physical node 4 contains Master Besu Node, Cockroach data base which hosts the “Seed Index” for federating queries, Rock Data Base central instance of the blockchain, Security Services for allowing authentication and authorization capabilities to the central node and Notification Manager

Infrastructure Monitoring

The idea behind was to take advantage of Ansible Tower and the metrics provided via the API and feed them into Grafana by using Node exporter and Prometheus

Ansible Tower must be configured to provide metrics for Prometheus to be viewed via Grafana. In addition to that, Node Exporter is used to export the operating system metrics to an operating system (OS) dashboard in Grafana.Grafana looks for data in Prometheus. Prometheus itself collects the data in its database by importing them from Node Exporters and from the Ansible Tower APIs.

Free* Open Source Software Tools for SMEs, Developers and Large Industries Building/Enhancing their Data Marketplaces.

For more detail and source code please refer below link.

Go to the beginning of the page

Other resources of interest


We are a community of experienced developers who understand that it is all about changing the perception of data economy via marketplaces support.

i3-MARKET Architecture

Take a look at the main building blocks and their hierarchy.

Data Access API

The secure Data Access API enables data providers secure registration…

Data Storage

The Decentralised storage shall provide highest available security guarantees…

Identity and Access Management

The SSI & IAM subsystem is in charge of providing both “User-centric Authentication” and…

Smart Contracts, Wallets & Accounting

i3-M Wallet is a set of technologies that facilitate the management of their identity to…

Crypto Token and Monetization System

The Data Monetization subsystem is in charge of providing “Standard Payments”…

Semantic Engine

We developed and implemented dedicated software components for Semantic Engine System as…

Developers Quickstart SDK

Once a marketplace is part of i3-MARKET, it can issue credentials to its consumers, providers, and…


Full Developers Documentation