Importing ODP Data using R

Omni Data Platform (ODP), an enterprise data management platform by CAPIOT, is a cornerstone in the digital enterprise. It provides users with  unprecedented simplicity to design and distribute data in real-time. The platform gives developers and analysts a shot-in-the-arm during their implementation phase as it takes care of all service layer and data layer build and deployments. 

Why do we need to pull from ODP?

In today’s world, data is leveraged to derive actionable insights which  provides the company with data points that will help forge its business strategy and long-term plans. . Here ODP acts as a datawarehouse  that stores information important for  the company.

Use Case :

ODP being used as a Data Management System which holds Employee data.

In this blog, we will pull a data entity (Employee Details) from ODP  into R studio and Spotfire for performing analysis.

The intended audience of this data set will be data analysts who can create actionable insights from the data set.

Step 1: Schema Creation  in ODP Author

Create three attributes in Employee Details.  Name, Age, City.

On save, ODP creates the CRUD APIs for these entities and the services are deployed as docker containers..

Step 2 : Adding Records into Employee Details

Login to ODP Client and add the records for the Employee Details

Once records have been inserted, users can use the CRUD APIs that were generated at schema creation to act on these data records.  intantly provides .  In our case, we are going to pull data from ODP and use that data in R and Spotfire..

Step 3 : Importing ODP Data in R studio

Using the below R script we managed to pull data from ODP to R Studio.

                                                   R script :

#Import the Libraries

library(httr)

library(jsonlite)

library(purrr)

library(data.table)

## 1.Create a Token

#Framing JSON Request to create a token

loginreq = ‘{ “username” : “jack @xyz.com”, “password” : “jack@123”}’

#Fetch the Token using HTTP POST API

req=httr::POST(“https://xyzabc123456.com/api/a/rbac/login”,httr::add_headers( “Content-Type” = “application/json;charset=UTF-8” ),body = loginreq  );

#Parse the HTTP JSON response to R object

odptoken = content(req , as=”parse”)

#Creating the token by Concat JWT with a token value

oktoken = paste( “JWT” , odptoken$token, sep=” “)

## 2. GET  records count from Employee Details

# Get the total count of records for the Employee Details service

resp<-GET(“https:// xyzabc123456.com/api/c/CapiotInternalDemo/dddd/count”,add_headers(accept = “application/json”,authorization = oktoken))

# Set the total count value to variable

totalcount =content(resp,as=”text”)

##  3. Pull all the Employee Details data from ODP

odpGet<-GET(“https://xyzabc123456.com/api/c/CapiotInternalDemo/dddd/”,query=list(count=totalcount ,expand=”false”,sort=”_id”,select=”name,age,city”),add_headers(accept=”application/json” ,authorization = oktoken))

odpGet1 = content(odpGet , as=”text”)

# Convert each line from json to a list

json_list <- map(odpGet1, fromJSON)

# Convert each list to a data.table

dt_list <- map(json_list, as.data.table)

#Harness the power of rbind list

dt <- rbindlist(dt_list, fill = TRUE)

Step 4 : Importing ODP Data in SpotFire  

Create a Register Data Function by including  the above R script.

Create a the output parameter as table Data Type

Click OK and visualize the created table in spotfire design layer