Data Stores

Introduction

In today’s data-driven world, efficiently managing and storing data is crucial for businesses to thrive. Data stores are centralized repositories that allow organizations to store, manage, and access their data effectively. In this article, we will explore the basics of Data stores, their types, and how DataSunrise, a leading data security platform, empowers users to create and manage instances for data stores seamlessly using its Remote Procedure Call (RPC) functionality.

What are Data Stores?

A data store is a location where data is stored and managed for easy retrieval and manipulation. Data stores can be real or digital, and they can be databases, files, or cloud storage services.

DataSunrise offers great coverage of modern databases, warehouses, storages and file systems. We are going to discuss how the data store is included in DataSunrise to monitor, secure the data and conduct the compliance discovery.

Data Stores play a vital role in supporting business operations, enabling data analysis, and facilitating decision-making processes. By centralizing data storage, organizations can eliminate data silos, reduce redundancy, and streamline data management tasks.

Instances for Data Stores in DataSunrise

DataSunrise offers a comprehensive set of tools for managing diverse types of data storage.

DataSunrise implements the Data Store as a Database Instance. To create the one you should move to ‘Configuration’ -> ‘Databases’ -> ‘+ Add Database’.

Below you can see the Instance setup UI. We truncated proxy setup for clarity. As you can see in the picture, you set up host parameters, username, and password. You can also test the database connection.

DataSunrise doesn’t let you save a new Instance before testing the connection.

All the databases, warehouses, services and file storages covered by DataSunrise are the same in the setup of Instances. With the only exception of credentials for cloud-based storages and file systems.

Let’s explore the different types of Data Stores available in DataSunrise:

1. Relational Databases: DataSunrise supports popular relational databases such as MySQL, PostgreSQL, Oracle, and SQL Server. These databases store structured data in tables with predefined schemas, enabling efficient querying and data manipulation using SQL.
2. NoSQL Databases: For handling unstructured or semi-structured data, DataSunrise integrates with NoSQL databases like MongoDB, Cassandra, and Redis. NoSQL databases offer flexibility and scalability, making them suitable for handling large volumes of diverse data.
3. Cloud Storage: DataSunrise seamlessly integrates with cloud storage platforms like Amazon S3, Google Cloud Storage, and Azure Blob Storage. These data stores provide scalable and cost-effective solutions for storing and accessing data in the cloud.
4. File Systems: DataSunrise helps you organize data stored in regular file systems like folders on your computer or network storage. File systems are ideal for storing unstructured data like documents, images, and videos.

Creating Data Store Instances with RPC

DataSunrise has a feature called Remote Procedure Call (JSON RPC) that lets users create and manage Data Store instances using code. RPC enables automation and flexibility in data management tasks. Let’s explore an example of how to create a Data Store instance using RPC in DataSunrise.

Example: Creating a PostgreSQL Data Store Instance with RPC

./executecommand.bat connect -host 192.168.10.205 -port 11000 -login admin -password 123123 -protocol https

The session will expire in 10 minutes after connection if not used.

./executecommand.sh callRPC -file ./myfolder/Request.json

The JSON file RPC JSON – Request.json is as follows:

{
  "data": {
    "login": "postgres",
    "dbType": 4,
    "name": "PostgreSQL@192.168.10.230:5432",
    "connectType": 1,
    "enableSNI": false,
    "dbName": "postgres",
    "awsRegion": "",
    "withFKMetadata": false,
    "asSysdba": false,
    "host": "192.168.10.230",
    "port": 5432,
    "ipVersion": 0,
    "cryptoType": 0,
    "kerberosServiceName": "postgres",
    "sslCertVerificationType": 0,
    "clientSslKeyGroupID": -1,
    "serverSslKeyGroupID": 0,
    "serverID": 1,
    "loginType": 1,
    "isSeparateMsgHandler": false,
    "msgHandlerThreads": 1,
    "isSeparateAuditJournal": false,
    "auditJournalThreads": 1,
    "trafficSrcType": 1,
    "networkInterfaceIP": "0.0.0.0",
    "localPort": 5433,
    "queryGroupFilter": "{\"groups_id\":[]}",
    "dynamicSqlProcessing": false,
    "dynamicSqlHandlerPort": 0,
    "dynamicSqlHandlerHost": "",
    "acceptSslConnectionsOnly": false,
    "passwd": "AES256_CBC_fYTdX42TQX5a/FKb4jawfQ==",
    "protocolType": 0,
    "additionOption": "",
    "dsEnvironmentName": "DS_ENVIRONMENT",
    "autoCreateEnvironment": false,
    "checkExistInterface": true,
    "enableSslOnlyForClientConnections": false,
    "useConnString": false,
    "customConnString": "",
    "passwordVaultType": 0,
    "isConnectionString": 0
  },
  "func": "createNewInstance",
  "queryID": 1220950648,
  "session_id": 967799160853902
}

This produces an output like the following in case of instance exists:

{
"errorStmt" : "",
"errorDesc" : "Already have instance with same name. PostgreSQL@192.168.10.230:5432",
"error" : 10,
"queryID" : 1220950648
}

When you work on JSON, make sure you don’t have instances with the same name or instances linked to the same database. Otherwise DataSunrise backend gives an error or additional questions. This can break the automation when you run RPC call.

In case of successful RPC call you see this in response JSON:

{
"id" : 16,
"error" : 0,
"errorDesc" : "",
"queryID" : 1739172125
}

How to Find JSON for RPC?

DataSunrise Web-based interface uses JSON RPC by default as the main method of controlling the firewall core.

You can find all the commands in JSON form using Debug Console in your web-browser.

1. For Chrome, or Edge press F12 button and move to the ‘Network Panel’.
2. Move to ‘Configuration’ → ‘Databases’ → ‘+ Add Database’, make initial Instance setup and test database connection. Do not press the ‘Save’ button.
3. Use ‘Start recording Network Log’ button to start logging and press ‘Save’ button. The log events will appear below in the console.
4. Press Stop recording button. You will find one or more requests to the address like https://192.168.10.205:11000/web_iface. The request payload is actually the JSON for RPC call. Turn on the Payload section to see the 

Try to stop writing the network log as soon as ‘Save’ is pressed. This minimizes the network log in your browser console. And it will be easier to analyze the ‘web_iface’ calls to find the JSON you need. In case Instance Saving it should include “func”: “createNewInstance” property after the data section.

DataSunrise’s RPC functionality automates the creation and management of data store instances. It streamlines the process of configuring and setting up data storage environments. This saves time and effort. This saves time and effort.

Example: Creating a PostgreSQL Data Store Instance with dedicated command

This approach is less common because DataSunrise CLI does not implement all the functionality. But it works and is worth mentioning at least. This approach works with no JSON and browser console analysis.

executecommand.bat connect -host 192.168.10.205 -port 11000 -login admin -password 123123 -protocol https

executecommand.bat addInstancePlus -dbHost 192.168.10.230 -dbPort 3305
-proxyHost 192.168.10.205 -proxyPort 5432 -dbType postgresql -database test -password 1234 -login postgres -name test

For other database types, refer to the DataSunrise CLI Guide on command ‘addInstancePlus’. You can also find more information on DataSunrise RPC in this guide.

Summary and Conclusion

Data Stores are essential components of modern data management, providing a centralized repository for storing and managing an organization’s data. DataSunrise offers various tools for managing different types of Data Stores. These include relational databases, NoSQL databases, cloud storage, and file systems.

With DataSunrise’s Remote Procedure Call (RPC) functionality, users can programmatically create and manage data store instances, enabling automation and flexibility in data management tasks. The provided example demonstrates how to create a MySQL Data Store instance using RPC in the DataSunrise CLI.

DataSunrise is a standout platform for data management, with robust features for data security, audit rules, masking, and compliance. It is also highly flexible. The easy-to-use interface and strong features make it a popular option for companies looking to simplify their data management.

To learn more about DataSunrise and its advanced data management solutions, we invite you to visit our team for an online demo. Experience firsthand how DataSunrise can transform the way you handle and protect your valuable data assets.