Developers QuickStart – SDK

SDK Approach

The layered SDK approach defined in the context of task 4.5 is the mechanism that allows to adapt and extend existing data marketplaces to interface with the i3-MARKET backplane.

Specifically, the layers that are part of the proposed solution for the SDK are the following:

• SDK-Core: this layer aims to simplify the i3-Market SDK building process by generating client stubs for any i3-Market backend endpoint/API, defined with the OpenAPI (formerly known as Swagger) specification. In this way, therefore, the development team can better focus on the implementation and adoption of these backend endpoints or APIs.
• SDK-Reference Implementation (SDK-RI): aims to identify and provide a set of common services to be implemented for consuming available Backplane functionalities.
• SDK-Execution Patterns (SDK-EP): it is including the atomic functions which make use of Backplane API (via SDK) adding some business logic.
• SDK Web – RI: it is supporting the front-end or GUI integrating the common services provided by the SDK-RI and that can be reused and customized as part of the pilot specification and implementation defined in the context of WP5.

Further details of each one of the SDK layers are provided in dedicated sections SDK Core, SDK Reference Implementation, and SDK Web – Reference Implementation.

SDK Core

SDK Core Specification

General Objectives

• Backplane API SDK. The main goal of the SDK is boasting the Backplane API to create applications for the i3-MARKET platform. It will assist the data marketplaces and stakeholder developers with a set of tools, examples and documentation which will reduce the developing effort to be part of the i3-MARKET ecosystem. The Backplane API SDK content is divided into different logical modules which correspond to each of the i3-MARKET modules integrated in the Backplane API. Following, the different modules identified for the first version of the requirement specification can be seen:
  • User Centric Authentication SDK
  • Cloud Wallet SDK module
  • Data Access SDK module
  • Standard Payments SDK module
  • Tokenization SDK module

• Enhanced Backplane API SDK. For some cases, the SDK will complete the Backplane API services with its own logic to support the developers in the use of the i3-MARKET capabilities. These will be done through a set of workflows.
• Automatically-build Backplane API SDK. In addition to the inner SDK functionality, i3-MARKET will provide mechanisms to automatically build the SDK component and it will be offered in different programming languages.

Context

From “Figure 1 – SDK position in the i3-MARKET system”, it can be seen how the i3-MARKET SDK interacts with following building blocks:

• Backplane API. The Backplane API becomes the most important interactor of the Backplane SDK. In fact, the essence of the SDK is precisely allowing stakeholder´s developers to create software (App Client) based on the (Backplane) API, in an easy and efficient way.
• Cloud Wallet. The SDK will interact with the Cloud Wallet Client to guarantee the security on the iterations between the stakeholders and i3-MARKET Backplane i. First, the Backplane SDK connects with the Cloud Wallet Server to create a wallet identity. Once the user has a wallet identity, the Backplane SDK will connect with the Backplane API with the correct access token.
• App Client. The piece of software, created by the stakeholder´s developers, which allows them to be part of the i3-MARKET ecosystem.

In addition to these interactions previously listed, i3-MARKET SDK might interact with other Backplane SDKs to put in contact directly different stakeholders with different roles. For example, it could be necessary that the Data Owner gives consent to the Data Provider for sharing its data, to perform that actions such as “Give Data Owner Consent” will be invoked from Data Provider SDK to Data Owner SDK. This operative will be included as part of the SDK release 2.

At the top of the image it can be seen the three types of actors which interact with the i3-MARKET system. Some of them will interact directly with the Backplane SDK and some others will do indirectly through a Data Marketplace:

• Data Marketplaces. The only actor planned to interact directly with the Backplane API through the Backplane SDK was initially the data marketplace but in subsequent discussions was agreed that the stakeholders such as consumers, providers and owners could also interact with the Backplane API. Few more details can be seen in the next paragraph.
• Stakeholders such as: Consumer, Provider and Owner. The three of them will interact not only with their respective data marketplaces but also directly with the i3-MARKET Backplane API through the Backplane SDK. In order to do that, the stakeholder should give proof of his/her identity and act with the proper role. Some of the actions which the stakeholder will be able to do directly with the SDK/Backplane are the authentication, offering creation/discovery, subscription, etc.
• Data spaces. The data spaces will act as data providers therefore they will be able to interact with a concrete data marketplace or even directly with the i3-MARKET Backplane API through the Backplane SDK, with the correct role and security token.

Big Picture

It is important to highlight that SDK-Core is supported as main pillar for the SDK-Generator which is one of the outcomes of task 4.5.

The following picture shows the subsystem in charge of the SDK generation. Several components are involved in the process.

Some of the components depicted in “Figure 2 – SDK Generation diagram components” are external to the SDK scope, such as:

• Backplane API/Backplane API Gateway, provided in the context of Task 4.4,
• OpenAPI Generator, an open-source tool useful for the generator process.

On the other hand, following components will be provided in the context of the SDK Generation:

• SDK Generator, the entry point to the Backplane API SDK creation and client of the OpenAPI Generator API.
• Backplane SDK Enhancer. It will oversee extending the Backplane API functionality when required. It will use as input the result from the OpenAPI Generator and will give as result an augmented SDK which is able to complement the functionality exposed by the Backplane API.

SDK Core Implementation

The SDK Core implementation is based on the usage of SDK Generator and it is described in detail in next subsections.

Core Technology

The SDK Core is supported by means of a) the SDK Generator REST API and b) an ansible playbook in charge of generating:

• An SDK-Core Java artifact that contains client stub for Backplane API (semantic engine, notification manager and smart contract manager), OIDC (Open Id), VC (Verifiable credentials) and Data Access API.
• An SDK-Core JavaScript artifact contains client stub for Backplane API (semantic engine, notification manager and smart contract manager), OIDC, VC and Data Access API.

SDK Generator

The SDK Generator is the main pillar of the SDK Core. The SDK Generator is based on SDK as a service approach. SDK Generator aims to automatically generate the client stubs needed to interact and consume all the functionalities exposed in a REST API. The SDK as a service approach is shown in “Figure 3 – SDK Generator approach”.

The workflow behind SDK Generator is based on the provision of a programming language specification next to an OAS file and making use of the OpenAPI generator server which is able to produce as output SDK client stubs next to associated documentation about how to use it.

The languages supported by the SDK Generator are shown in “Figure 4 – SDK Generator supported programming languages”:

Continuous Integration and Delivery

The SDK-Core artifact is automatically provided by means of an CI/CD pipeline based on Ansible AWX. A conceptual view of SDK-Core pipeline is shown in “Figure 5 – SDK-Core CI/CD pipeline”.

As initial step in the pipeline the SDK-Core artifact is triggering the compilation and deployment of a new version of the SDK-Generator once a commit into master branch of SDK-Generator project happens. As second step (represented as green area in Figure 6. SDK-Core CI/CD pipeline.), the generation and publishing of a new version of the SDK-Core artifact it is triggering each one a new version of backplane API it is deployed. The CI/CD behind backplane API includes a triggering to SDK-Core pipeline. In this way, SDK-Core covers a set of tasks mainly in charge of generating SDK-Core artifacts for Java and JavaScript versions taking a set of relevant OAS files associated to the following artifacts:

• Backplane API (including Semantic Engine, Notification Manager and Smart Contract Manager)
• OIDC API
• Verifiable Credentials API
• Data Access API

Finally, the pipeline includes a couple of tasks in charge of publishing the generated Java and JavaScript versions of SDK-Core into i3-MARKET Nexus repository.

SDK Core Installation

SDK Core is a library which is installed by simply importing from Nexus.

SDK Reference Implementation (SDK-RI)

SDK-RI Specification

Objectives

• Provide the mechanisms in terms of SW pieces for testing the i3-MARKET Backplane services/artifacts.
• Follow the approach SDK-RI as a service: SDK-RI will be a set of services needed for simulating a i3-MARKET-ized data marketplace behaviour.
• SDK-RI will let to the pilots check this reference implementation as a guide/example for developing their own integration with i3-MARKET.

Context

SDK-RI contextualization was already introduced in section Context as part of the SDK Core context.

SDK-RI Implementation

Core Technology

In an initial stage of SDK-RI implementation the technology options presented in “Figure 6 – Implementation technologies for SDK-RI” were considered:

To sum up, the candidate technologies to support the implementation of SDK-RI were the followings:

• Node.js
• Node.js + Express
• Java + RPM
• Java + Swagger + Tomcat

Finally, option 4 was selected but substituting Jetty for Tomcat as web application server. Therefore, we can conclude by saying that SDK-RI is a web app deployed within Jetty and encapsulated in a docker container.

Continuous Integration and Deployment

The SDK-RI artifact is automatically provided by means of an CI/CD pipeline based on Ansible AWX. A conceptual view of SDK-Core pipeline is shown in “Figure 7 – SDK-RI pipeline”.

As initial stage the SDK-RI is imported as a library in the last version of the SDK-Core published in i3-MARKET Nexus maven repository. As second stage, once a commit is done into master branch of SDK-RI GitLab project, a compilation and deployment of a new version of SDK-RI is carried out.

SDK-RI Installation

The setup instructions and docker based deployment of SDK-RI is covered in detail in next subsections.

Setup

Clone the repository and download the dependencies:

git@gitlab.com:i3-market/code/sdk/i3m-sdk-reference-implementation.git

Keycloak Setup

The SDK-RI default configuration file is placed in /src/main/wabapp/WEB-INF/keycloak.json. The default configuration is shown below:

To provide your Keycloak setup you should update providing a realm, auth-server-url, resource and credentials.

Running the SDK-RI with docker

Use docker to run the SDK-RI. To do so, follow the same setup instructions as above.

Then, just build and run using:

SDK-RI container is built over a Jetty image and the SdkRefIMpl war file is deployed into Jetty.

Finally, just go to http:/$deploy_host/SdkRefImpl for accessing to SDK-RI REST API.

Configuring and using SDK-RI

To configure SDK-RI instance, the following steps should be covered:

• The marketplace will have all the common services exposed in an SDK-RI/ endpoint.
• Each marketplace end user, which pursues making use of the SDK-RI, should configure the SDK-RI by means of:
  • Pointing to the Backplane endpoint(s) hosted in a concrete i3-Market node (i.e: Backplane API node1, OpenID Connect Provider API node1, Verifying Credential service API node1)
  • Pointing to the Wallet endpoint hosted locally.
• This configuration should be defined in the SDK-RI properties file placed at “src/resources/sdk_ri_config.properties”

Prerequisites

Marketplaces must be accepted to join the federation. Currently, the rules of the federation have not yet been decided. Once a marketplace is part of i3-MARKET, it can issue credentials to its consumers, providers, and data owners. But as far as marketplaces are concerned, the authorisation flow of a data marketplace has not been determined.

Deploy and Configure the SDK-RI

Requisites

As it is shown in Figure 25. Hardware consumption., the current requirements of the SDK-RI in terms of machine capacity are quite low, however, these requirements will have to be analysed in more depth later when the library is fully functional.

Usage SDK-RI

Currently the configuration of the SDK-RI is carried out as follows:

Keycloak Setup

Once the project has been downloaded, the configuration file indicated by the Keycloak server must be modified.

Default config file is placed in /src/main/wabapp/WEB-INF/keycloak.json:

{ “realm”: “i3-MARKET”, “auth-server-url”: “http://83.149.125.78:8080/auth/”, “ssl-required”: “none”, “resource”: “SDK-RI_Client”, “credentials”: { “secret”: “xxxxxxxxxxxxxxxxxxxxxxxxx” }, “use-resource-role-mappings”: true, “confidential-port”: 0 }

To provide your keycloak setup you should update providing a realm, auth-server-url, resource and credentials for your keycloak server.

In case you do not have your own keycloak server you can request credentials from I3-Market members.

Running the SDK-RI with docker

You can use docker to run the SDK-RI. To do so, follow the same setup instructions as above.

Then, just build and run using:

docker build –no-cache -t i3m/i3-MARKET-sdk-ri:latest . docker run –name sdk-ri -p 8181:8080 i3m/i3-MARKET-sdk-ri

SDK-RI container is built over a jetty image and deploy the SdkRefIMpl war file into jetty.

Finally just go to http:/$deploy_host/SdkRefImpl for accessing to SDK-RI REST API.

Configuring and using SDK-RI

To use a deployed instance of SDK-RI the following indications should be considered:

• The marketplace will have all the common services exposed in a SDK-RI endpoint
• Each marketplace end user, which pursues making use of the SDK-RI, should configure the SDK-RI by means of:
  • pointing to the Backplane endpoint(s) hosted in a concrete i3-Market node (i.e: Backplane API node1, OpenID Connect Provider API node1, Verifying Credential service API node1)
  • pointing to the Wallet endpoint hosted locally.
• This configuration should be defined in the SDK-RI properties file placed at “src/resources/sdk_ri_config.properties”

WEB-RI

Purpose

The WEB-RI proposes itself as reference for the implementation of a user interface to allow human users to use and interact with the functionalities provided by i3-MARKET. The WEB-RI has 3 main objectives, which they are:

• As management tool, to allow i3-MARKET developers to test their functionalities in the context of a user usage.
• By a marketing team, allowing the promotion and demonstration of i3-MARKET functionalities using a generic approach and language that can be easily translated to the available data marketplaces used by different domains.
• As a reference implementation, providing functional examples of how the i3-MARKET SDKs can be used to implement/integrate i3-MARKET functionalities into a data marketplace. As a reference implementation, WEB-RI is also a useful tool to help i3-MARKET pilots on the implementation of their use case scenarios and on testing of backplane technologies by providing specifications and code that can be used

WEB-RI Architecture

A consumer or a provider can access WEB-RI[1] via internet browser and proceed with the authentication for which the wallet must be installed and running on his personal computer. The authentication process is executed on WEB-RI frontend by calling the OIDC service which will call the wallet to perform the authentication itself.

To manage the user sessions, the WEB-RI backend saves the user session in a session storage called connect-mongo

To interact with the functionalities provided by i3-MARKET, a library was implemented, called Connector-RI.

Sitemap

Free* Open Source Software Tools for SMEs, Developers and Large Industries Building/Enhancing their Data Marketplaces.

For more detail and source code please refer below link.

Go to the beginning of the page

Other resources of interest

Documentation

Full Developers Documentation