Specification Deployment

i3-MARKET Artifacts overview

So far, the i3-MARKET artifacts involved in the deployment have been classified into the following categories:

  • Third-party artifacts (black): these artifacts are open-source dependencies. In I3-MARKET R1 have been identified the following:
    • Hyperledger Besu (blockchain network)
    • CockroachDB (distributed storage)
    • RocksDB (ledger)
    • Loopback4 (i3-MARKET backplane API)
  • I3-MARKET artifacts (green): SW pieces to be developed in the context of i3-MARKET, mainly provided in Wp3 and Wp4.
  • I3-MARKET SDK artifacts (orange): SDK software to be developed in the context of i3-MARKET and deployed in the external actor’s side.

A detailed list of i3-MARKET artifacts including artifact ID, artifact name, artifact deployment responsible, artifact deployment host/s, artifact dependencies, artifact deployment status for R1, artifact deployment status for Mid Review and additional info as public deployment endpoints is shown in Table 1. i3-MARKET artifacts:

Artifact IdArtifactHostsDependencies
A1Blockchain FrameworkallDecentralized Storage
A2CockroachDB (Distributed Storage)all
A3Decentralized StorageallIncluded with the Blockchain framework
A4User-centric Authenticationall
A5Service-centric Authenticationall
A6HW WalletPilot Host
A7Cloud WalletallCloud Wallet Client, Backplane API (Cloud Wallet Server, User-centric Authentication), Data Access SDK, I3-Market SDK
A8Smart Contract Managerall
A9SLA/SLE Smart Contractall
A10Conflict Resolutionall
A11Explicit-User ConsentallBackplane API (Smart Contract Manager, Distributed Ledger, Distributed Storage)
A12Auditable Accountingall
A13Standard PaymentallBackplane API (Auditable Accounting, Conflict Resolution, Smart Contract, SLA/SLE Smart Contract)
A14TokenizationallBackplane API (User-centric Authentication, Smart Contract, SLA/SLE Smart Contract)
A15Micro Paymentall
A16Data Access SDKPilot Hosts
A17Data Acess APIall
A18Semantic Data Manager (triplestore)Node4
A19Semantic Model ManagementNode4
A20Semantic Orchestration (Offering and Discovery)Node4Backplane API (users ids), Decentralized storage
A21Backplane APIall
A22i3-Market SDK-GeneratorNode1
A23Ansible AWXNode2
A24Docker Swarm Clusterall
A25ZabbixNode3
A26SDK-RI (Reference Implementation)all
A27SDK-core
SDK-Generator
A28Nexus

A29Secure Server (keycloak)Node4
A30 Notification ManagerAll nodesSDK-RI, sdk-core
Table 1 – i3-MARKET artifacts

Deployment architecture view

Cluster Infrastructure

Figure 1. i3-MARKET deployment diagram (R1) shows the deployment diagram associated to i3-MARKET ecosystem for the first report period. The main deployment decisions behind are the following:

  • The i3-Market cluster consists of 4 nodes in which the following artifacts have been deployed:
    • Hyperledger Besu (blockchain network node). I3-MARKET private blockchain network requires 4 Besu nodes to support BFT consensus approach.
    • CocroachDB (distributed storage instance). CocroachDB has been selected as technology to support distributed storage in I3-MARKET.
    • Loopback4 (backplane API instance). Looopback4 is the technology supporting backplane API.

In addition to that, it should be considered that:

  • The existing pilot’s demonstrator will adhere to one of the four nodes of the cluster. The loopback instances, deployed in the cluster, will be mapped to each one of the existing data markets which are: AGORA, MindSphere and IBM DM.
  • All semantics artifacts will be deployed to each one 4 nodes conforming the cluster.
  • Security artifacts behind the security server will be deployed on a physically decoupled node.
  • Finally, I3-MARKET R2 and R3 will be updated properly to consolidate the strategy of progressively moving the main SW artifacts alongside the data market.
Figure 1 – i3-MARKET deployment diagram (R1)

Taking into account the existing funding for the i3-MARKET common infrastructure provisioning, and trying to optimize said funding as well as balance the load of the cluster nodes, the original deployment diagram was updated, which resulted in the diagram shown in Figure 2. Deployment diagram R1 update.

Figure 2 – Deployment diagram R1 update

To sum up, it is important to mention that the number of nodes was reduced from 4 to 6 in order to optimize funding and be able to maintain the infrastructure until the end of the project. NGINX has been adopted as a reverse proxy and load balancer.

System requirements

To elaborate the deployment plan and identify the SW artifacts list, dependencies, SW and HW requirements and restrictions, a template was circulated to the partners. For further details about this template go to : SW and HW requirements.

An overview of the main capabilities required for the i3-MARKET SW artifacts are shown in Figure 3. i3M physical nodes tentative approach as a tentative approach.

Figure 3 – i3M physical nodes tentative approach

Final approach to be considered for i3-MARKET R1 is shown in Figure 4. i3M physical nodes final approach. A detailed description of capabilities for each one of the required artifacts for R1 is provided in : SW and HW requirements.

Figure 4 – i3M physical nodes final approach

i3-MARKET V1

The I3-MARKET Cloud SW stack is represented in Figure 5. I3M Cloud SW stack and it is composed of four layers: Cloud provisioning and management layer, DevOps SW layer, Third-party SW layer and I3M SW layer.

Figure 5 – i3M Cloud SW stack

Cloud provisioning and management layer oversees providing and managing all physical nodes that the i3-MARKET common infrastructure is composed of. ATOS will be leading the provision of the cloud layer.

For the management of physical resources in a homogeneous way, an Ansible Tower1 instance will be deployed for the administration of said physical resources, thus having their management centralized from Ansible.

The DevOps layer will be led by SIEMENS and the concrete DevOps approach as well as the concrete DevOps stack for i3-MARKET will be specified in internal deliverable I2.41 Error: No se encuentra la fuente de referencia. I3-MARKET DevOps will be a set of practices that will combine software development and IT operations and it will aim to shorten the I3-MARKET system development life cycle and provide continuous delivery with high software quality.

Third-party SW layers will be managed by GFT, UPC and Guardtime and it will be mainly in charge of providing the SW stack identified as SW requirements by i3-MARKET system. These SW requirements are: Hyperledger Besu, CocroachDB, Loopback4 and Keycloak.

Finally, i3-MARKET SW layer will be composed by all i3-MARKET artifacts. Section I3-MARKET SW layers provides a detailed view about SW components, associated build blocks, physical resources assigned, deployment owners, artifacts type and technologies.

1 Ansible Tower: https://www.ansible.com/products/tower

i3-MARKET SW Layers

Tables in current section collect all the relevant information associated with the deployment of all the artifacts that are part of the four layers of the i3-MARKET cloud stack and that were identified in section I3-MARKET Artifacts Overview. Therefore, for each i3-MARKET artifact the following information is provided:

  • Artifact name or SW component,
  • Associated building block (see internal deliverable I2.41 Error: No se encuentra la fuente de referencia),
  • Assigned-VM or physical resource (PR),
  • Artifact deployment owner,
  • Artifact type, and finally,
  • Technology supporting artifact.

Generally, meaning of the colours in the tables are the following:

  • Artifacts developed, deployed, and running for R1: blue colour,
  • Artifacts developed, deployed, and running for the first report period (R1 + half R2): green colour,
  • Artifacts developed, deployed, and running for the following periods: orange colour.

i3-MARKET propietary Software

i3-MARKET SW layer is composed by the following i3-MARKET artifacts shown in Table 2. SW artifacts deployment information.:

SW COMPONENTBUILDING BLOCKTYPETECHNOLOGY
User-centric AuthenticationTrust, Security and Privacyi3-MARKET SWKeycloak
Service-centric AuthenticationTrust, Security and Privacyi3-MARKET SWOIDC, VC
Cloud WalletTrust, Security and Privacyi3-MARKET SW
SW WalletTrust, Security and Privacyi3-MARKET SW
HW WalletTrust, Security and Privacyi3-MARKET SW
Smart Contract ManagerTrust, Security and Privacyi3-MARKET SW
Conflict ResolutionTrust, Security and Privacyi3-MARKET SW
Explicit user-concernTrust, Security and Privacyi3-MARKET SW
Auditable AccountingTrust, Security and Privacyi3-MARKET SW
Standard paymentsTrust, Security and Privacyi3-MARKET SWLibrary to be integrated and deployed in Data Access SDK
Micro-paymentsTrust, Security and Privacyi3-MARKET SW
TokenizationTrust, Security and Privacyi3-MARKET SW
Data Access SDK—-i3-MARKET SWSdk-Generator
Data Access APIData Accessi3-MARKET SW
Semantic Storage (Triple store)Semanticsi3-MARKET SW + Third P. SWVirtuoso
Metadata ManagementSemanticsi3-MARKET SWVirtuoso
Semantic Orchestration (Offering and Discovery)Semanticsi3-MARKET SWVirtuoso
Backplane APIBackplanei3-MARKET SWLoopBack4
i3-MARKET SDKReference Implementationi3-MARKET SWSdk-Generator
Notification MangerBackplanei3-MARKET SWRabittMQ
Table 2 – SW artifacts deployment information

Third-party Software

Third-party SW layer managed mainly by UPC, Guardtime and NUIG is composed by the following software artifacts shown in Table 3. Third-party i3-MARKET artifacts:

SW COMPONENTBUILDING BLOCKTYPETECHNOLOGY
Blockchain FrameworkBlockchain networkThird party SWHyperledger Besu
Distributed StorageData StorageThird party SWCocroachDB
Decentralized StorageData StorageThird party SWRocksDB.
Security ServerTrust, Security and PrivacyThird party SWKeycloack
Semantic StorageSemanticsi3-MARKET SW + Third P. SWOpen Virtuoso
Table 3 – Third-party i3-MARKET artifacts

DevOps Software

The DevOps layer leaded by ATOS and SIEMENS combines software development and IT operations and it aims to shorten the I3-MARKET system development life cycle and provide continuous delivery with high software quality by means of the following artifacts listed in Table 4. DevOps related SW artifacts.:

SW COMPONENTBUILDING BLOCKTYPETECHNOLOGY
Ansible AWXDeploymentThird party SWAnsible AWX
Docker SwarmDeploymentThird party SWDocker Swarm
Gitlab CI/CD (Runners)Ci/CDThird party SWGitLab
NexusCi/CDThird party SWNexus
NGINXManagement/ SecurityThird party SWNGinx
MKDocsDocumentationThird party SWMkDocs
Table 4 – DevOps related SW artifacts

Cloud Management and Monitoring

Finally, for the management and monitoring of physical resources in a homogeneous way the following artifacts shown in Table 5. Cloud artifacts. have been considered:

SW COMPONENTBUILDING BLOCKTYPETECHNOLOGY
Ansible AWXDeploymentThird party SWAnsible AWX
ZabbixMonitoringThird party SWZabbix
Table 5 – Cloud artifacts

Go to the beginning of the page

Other resources of interest

Home

We are a community of experienced developers who understand that it is all about changing the perception of data economy via marketplaces support.

i3-MARKET Architecture

Take a look at the main building blocks and their hierarchy.

Data Access API

The secure Data Access API enables data providers secure registration…

Data Storage

The Decentralised storage shall provide highest available security guarantees…

Identity and Access Management

The SSI & IAM subsystem is in charge of providing both “User-centric Authentication” and…

Smart Contracts, Wallets & Accounting

i3-M Wallet is a set of technologies that facilitate the management of their identity to…

Crypto Token and Monetization System

The Data Monetization subsystem is in charge of providing “Standard Payments”…

Semantic Engine

We developed and implemented dedicated software components for Semantic Engine System as…

Developers Quickstart SDK

Once a marketplace is part of i3-MARKET, it can issue credentials to its consumers, providers, and…

Documentation

Full Developers Documentation