i3-MARKET Architecture

The Overall Picture

The Overall Picture is the description of the system including main building block or subsystems. The image bellow shows the overall component diagram for i3-MARKET.

Level 1

Backplane Gateway System

The Backplane Gateway System is the building block in charge of offering to all participants and marketplaces access to the Backplane system. The goal of the Backplane API is therefore twofold: on one hand, it serves an integrated API endpoint for all the i3-MARKET services offered by i3-MARKET and implemented in the respective building blocks. And on the other hand, it provides secure mechanisms for preventing not allowed accesses.

In term of public interfaces, the functionality integrated by the Backplane Gateway is exposed thought the Backplane API.

In terms of internal connections with other i3-MARKET building block, Backplane Gateway system has secure communication with the rest of subsystems to integrate their services into the Backplane API. For this integration any service much have a complete specification following the OpenAPI Specification 3.0.

Data Access System

The Data Access System is the building block in charge of allowing data consumer obtain access to the data offered by the data providers.

It exposes its functionality, publicly, thorough the Secure Data Access API.

This data access system is securely linked with the Backplane API in order to guarantee two main issues:

Ensuring all the involved stakeholder have signed the required contracts,
Monitoring the quantities of exchanged data assets for the token-based monetization service.

Trust, Security and Privacy System

The Trust, Security and Privacy (TSP) is the building block in charge of providing the self-sovereign identity, access management, contracting, consents, accounting and payments blockchain-based solutions managed in the i3-MARKET system in order to guarantee the desired levels of trust, security and privacy for federated data markets.

The TSP system expose its functionally, publicly, thorough the Backplane API.

In term of dependencies with other existing building block, the TSP interacts with the decentralized ledger of the Data Storage system and with the Data Access System for the monetization of the data assets.

Semantic Engine System

The Semantic Engine System is the building block in charge of providing the needed semantic data models for make possible the consumers and applications understand the meaning of the data exchanged between different stakeholders. Apart from that, the semantic engine will allow the participants to taking advantage of this semantic data models by means of providing a metadata management in charge of registering offering and performing queries for discovery purposes.

All this metadata management and query functionality is exposes, publicly, thorough the Backplane API.

In terms of dependencies with other building blocks, the semantic engine mainly interacts with the Storage system for storing the offering descriptions.

Data Storage System

The Data Storage system is the building block in charge of storing common data shared across all participants instances.

It interacts with mostly all main building blocks, especially with the Semantic Engine System for performing the synchronization between semantic repositories and distributed storage and with the Trust, Security and Privacy for instantiating and executing smart contract in the blockchain-based decentralized storage.

Level 2

Backplane Gateway System – General description

The Backplane Gateway has two main purposes:

Single entrypoint

The Backplane offers a single set of endpoints, allowing clients to interact with all the services offered by the i3-MARKET project through a single API. This allows the whole system to have a modular and distributed architecture, while providing the ease of use of a single common interface.

Auth management

All authentication and authorization flows are centralized and managed by the Backplane Gateway so that the subsystems are protected without them needing to handle their own auth flows. The actual authentication/authorization is delegated to specialized subsystems. This centralization also allows the support of several auth flows (and the addition of new ones) transparent to the several subsystems.

Besides these main purposes, the presence of the Backplane Gateway provides several benefits:

Allows the subsystems to be isolated and not exposed publicly, so that they can only be accessed by the Backplane or other subsystems.
External connections are all handled by the Backplane, making connection security and encryption simpler and more straightforward.
The nature of the Backplane functionalities makes it easily scalable and replicable, making the addition of new Backplane instances transparent for both the subsystems and the clients.

Inner building blocks

The Backplane Gateway consists of 3 main components:

Backplane API

The Backplane API is the set of endpoints exposed by the Gateway. It comprises all the publicly available endpoints of the subsystems integrated with the Backplane, as well as a few other endpoints, belonging to the Backplane itself, used in the authentication/authorization flows.

The API follows the OpenAPI Specification 3.0, and the endpoints corresponding to each subsystem are generated automatically based on the subsystem’s own OpenAPI Specification.

Backplane Auth Manager

The Backplane auth manager is the responsible of handling the authentication and the authorization required for the endpoints of the different subsystems. The actual auth processes are delegated to the corresponding subsystem (User/Service centric authentication subsystem), depending on the requirements of the endpoint and client.

Backplane Router

The Backplane router is the component of the Backplane responsible for the forwarding of the incoming requests to the several subsystems. It also checks whether the endpoint requires authentication/authorization, invoking the Auth Manager if it does.

Data Access System – General description

The Data Access system consists of four main subsystems for implementing the transfer of data. The picture below shows the subsystems.

Inner building blocks

Authentication and Authorization

The Data Access system will assist i3-MARKET will following capabilities for the exchanged data assets:

• Authentication – Verifies the identity of the user against the i3-MARKET Backplane

• Authorization – Verifies the permissions the authenticated user has in the i3-MARKET platform allowing to perform authorized actions and grants access to resources

Authentication & Authorization subsystem has the following scope management:

• Policy management: Policy is a set of rules that defines how to protect the assets in order to provide trust, security and privacy. Policy Management component is in charge of enforcing the rule set provided by i3-MARKET Backplane inside of the Data Access system. The responsibilities of the Policy management module are:
  • Intercept access attempts
  • Check attempt against rule set
  • Grant access to permitted assets

• Role management: A Role is a set of Policies attached to an entity in order to define the access that entity has within the i3-MARKET Data Access system. The Role management component is in charge with fetching the list of policies and verifying them against the Data Access system. The responsibilities of the Role management module are:
  • Get the list of policies associated with Role from Backplane
  • Verify Role access by invoking Policy management
  • Allow or Deny functionalities

Secure Data Transfer & Anonymization

Secure Data Transfer & Anonymization subsystem has the following components:

• Data Encryption: The responsibilities of the Data encryption module are:
  • Key generation and Exchange
  • Transfer data in an encrypted way between endpoints
  • Decrypt data on the consumer side

• Proxy: The proxy needs to be used when the identity of the Data Provider needs to be hidden. This feature is optional, no needed to implement it if the there is no specific requirement referring to the anonymity of the Data Provider. The responsibilities of the Proxy module are:
  • Activate the proxy
  • Configure the parameters to hide the identity
  • Data transfer goes through the proxy

Data Transfer Transparency

Data Transfer Transparency subsystem has the following components:

• Data Transfer Management: This component is responsible with the management of the connection between provider and consumer and implements the following functionalities:
  • Initialize the connection
  • Resume the connection
  • Finalize the connection

• Data Transfer Tracking: This component implements the following operation:
  • Measure the amount of transferred data

• Data Transfer Monitor: The information about how much data was transferred, when the data transfer was initiated and when it was completed are monitored and the following operations are triggered:
  • Inform the i3-MARKET Backplane that the data transfer was performed and reports how much data was transferred
  • Invoke the linked smart contract

Data Management

Two methods for data transfer are supported by Data Access API which are supported by the following modules:

• Batch Data Transfer Management: One-time data transfer for one chunk of data in a session with the following methods:
  • Request data
  • Transfer data

• Data Stream Management: Continuous transfer of data based on a subscription, e.g. publish/subscribe mechanism:
  • Subscribe to an offering
  • Trigger data transfer – on the producer side
  • Get data – on the consumer side
  • Unsubscribe

Trust, Security and Privacy System – General description

One of the pillars of the i3-Market Backplane is the “Trust, Security and Privacy” System which leverages the blockchain technologies to ensure trust, security and privacy by design providing the following building blocks:

• An Identity and Access Management system based on Decentralized/Self Sovereign Identity and Verifiable Credentials
• Smart Wallets with different level of security (Cloud/HW Wallet)
• Smart Contracts to record, operate and manage in a trusted and transparent way the agreements between the different stakeholders and particularly the explicit user consent of the Data Owner
• A Data monetization system based on crypto currency for secure, trusted and cost-effective peer-to-peer payments

Inner building blocks

SSI&IAM

The SSI&IAM building block provide an Authentication and Authorization mechanism to access i3-Market Backplane and Stakeholders resources

The User-centric Authentication component implements the Self Sovereign Identity paradigm ensuring that:

• Identity and personal data are stored with the user
• Claims and attestations can be issued and verified between users and trusted parties
• Users selectively permission access to data and Data only needs to be verified a single time

The Service-centric Authentication component make the Data marketplaces of the network able to provide their users a Distributed Identity they own and can use with other stakeholders.

Both the authentication mechanisms follow the OpenID Connect standard to allow wide commercial acceptance.

Smart Contracts

The Smart Contract Manager enforces certain contractual parameters of the Data Sharing Agreement between data provider and consumer using pre-defined Smart Contracts, which are based on the legal agreements. The Smart Contract Manager component incorporates the conflict resolution, the explicit user-consent and the auditable accounting component.

Smart Wallet

i3-MARKET wallets are key components that enable interaction between the different stakeholders and services in the i3-MARKET ecosystem. A wallet just stores and uses cryptographic material that in i3-MARKET are used to achieve the following features: authentication and authorization (by proving ownership of DIDs and verifiable credentials), and non-repudiation of data exchange (by digitally signing different operations).

The Smart Wallet building block is designed to be as secure as user-friendly and in a way that existing technologies can be easily added as i3-MARKET-enabled wallets. In this project we are going to integrate three types of wallets:

• HW Wallet. Hardware wallets are in charge of storing the user’s private keys using a physical device as storage. It performs cryptographic operations to reduce the key exposition. This specific wallet satisfies the highest security policies since it is based on ‘something you have’ and you are the owner of the data.

• SW Wallet. Software wallets store the user’s private keys on the storage of a general-purpose device (e.g. computer or smartphone). It combines the security polices of ‘something you have’ and ‘something you know’. Despite no specific hardware is needed, the loss of the device might imply losing the keys if no backups are made.

• Cloud Wallet. Cloud wallets store the user private keys on secure cloud databases. Even it has the less strict security policies, it can offer much more functionalities than the other wallet subsystems, such as easier access and simpler key recovery.

Data Monetization

Data Monetization building block provide a crypto currency solution that allows instant currency exchange among the participating data spaces/marketplaces, and also support full audibility of all transactions. This is vital for a fully decentralized solution, as it provides the basis for building trust in the federation backplane. The payment mechanism shall support micro-payments and occur minimal cost.

The Standard Payment component permits in advance or a posteriori payment for a specific dataset or piece of data with traditional payment systems, ensuring trust, security and full auditability of data transfers through an ad-hoc Non Repudiable Protocol.

The Tokenization component provide the creation of a crypto token for instant currency exchange among the participating data spaces/marketplaces ensuring the full auditability of payment transactions provided by the blockchain technology.

The Micro payments component provide a mechanism for reducing the cost of crypto payment transactions especially for small amounts of tokens.

Semantic Engine System – General description

One of the core pillars in i3-MARKET is the semantic engine which play an important role in terms of registration and querying the offerings in a distributed and interoperable way. Semantic engine exposes different API endpoints for various tasks for example registration and querying.

Inner building blocks

Metadata Management

The Metadata Management subsystem encompasses three components:

• Offering registration: This component allows users to register offerings. Semantic engine exposes different endpoints for offering registration. Examples are;(i) register data provider, (ii) register offering of a data provider, (iii) update offerings

• Semantic mapping: This component does semantic mappings and transform (JSON to RDF) data received from API endpoints.

• Vocabulary management: This component keeps and manages all the vocabularies, defined as i3-semantic model, used in different components of the semantic engine.

Semantic Storage

This component communicates with the RDF triplestore and pushes and retrieves data from that store.

In i3-MARKET, open-source virtuoso is used as a triplestore, which allows us to store RDF data and query using SPARQL query language.

Querying

Two main components will be provided:

• Offering Discovery: This component allows users, either data provider or consumers, to query already registered offerings.
  
• Semantic orchestrator: The role of this component is to manage synchronization with the distributed data storage component and the query processing for instance, local or distributed query.

Data Storage System – General description

Inner building blocks

The Storage system consists of two main subsystems for implementing, respectively, the decentralised storage and distributed storage features, as shown in figure below. The subsystems are, at least in the initial architecture, relatively independent of other systems and, also, independent of each other.

Decentralised Storage

The diagram of decentralised storage subsystem is show in the figure bellow. The decentralised storage subsystem is implemented as blockchain-based distributed ledger network. The software implementation is Hyperledger Besu in a permissioned setup using IBFT 2.0 consensus [10]. Hyperledger Besu uses internally an embedded RocksDB instance for storing linked blocks (the journal of transaction) and world state (the ledger). Hyperledger Besu can instantiate and execute smart contracts for supporting the use cases of i3-MARKET framework.

Distributed Storage

The diagram of Distributed storage subsystem is shown in figure bellow. The subsystem consists of a distributed cluster of database nodes and an optional interface layer (will not be implemented for V1). The database provides an SQL interface to other i3-MARKET framework components. The software implementation of the internal database is CockroachDB that can be accessed via PostgreSQL-compatible wire protocol for which a large number of client libraries exist for different languages and platforms. CockroachDB is a highly scalable and resilient distributed database. Only secure access to the database will be enabled, hence all clients need to use private keys and valid certificates to access the database.

Deployment View

The deployment or physical view “describes the mapping(s) of the software onto the hardware and reflects its distributed aspect”

the i3-MARKET deployment view is depicted in the picture below. Four nodes constituted the i3-MARKET R1 cluster. On each node it will be deployed a Backplane Gateway System and an instance of all the rest i3-MARKET main building blocks (Trust, Security and Privacy System, Storage System, and Data Access System) giving backend support to the Backplane Gateway System. In addition to that, node 4 will host all the components related with the Semantic Engine Building Block

Free* Open Source Software Tools for SMEs, Developers and Large Industries Building/Enhancing their Data Marketplaces.

For more detail and source code please refer below link.

Go to the beginning of the page

Other resources of interest

Documentation

Full Developers Documentation