Realtime insert/update operation with JDBC driver on roxie

[adityac]

Hi Team,

After research I came to know that HPCC can process million of data in real time with less time. I am trying to evaluate a process to do real time data insert or modify with jdbc driver in java. But no luck to crack for real time data writing using jdbc driver. I did it for read/select operation and it works.

Can any one please help here to workout.

We are working with JDBC driver to fetch data from logical file in java.

Below are snap from our code-

String sqlQuery ="select * from tutorial::yn::transactions";
org.hpccsystems.ws.client.utils.Connection connection = new Connection("http://localhost:8510");
HPCCWsSQLClient mywssqlclient = HPCCWsSQLClient.get(connection);
System.out.println(mywssqlclient.getResults(mywssqlclient.executeSQLWUIDResponse(sqlQuery, "roxie", ""), 0, 50));

Till now we are ok with fetching data from file but we have real time requirement of INSERT/UPDATE operations on existing data of file.

Can someone please suggest us that we can support realtime INSERT/UPDATE operation on logical file?
If yes then how can we fulfill our requirements?

[rtaylor]

adityac,

Can someone please suggest us that we can support realtime INSERT/UPDATE operation on logical file?

The first thing to understand about the HPCC Systems platform is that it is NOT an RDBMS and not originally designed for OLTP use. It is a Big Data processing platform that was created to do batch processing of huge amounts of data (too big for effective use on any RDBMS). The JDBC driver is designed to allow external access, only, and does not support write operations.

If yes then how can we fulfill our requirements?

It IS possible to make an HPCC Systems platform emulate an OLTP system by adding in a "Deltabase" -- a small SQL database that contains all added/changed records until they can be rolled into the ROXIE data.

An intern in our Brazil office did a Deltabase project and submitted this to our 2020 Community Day conference Poster Competition: https://wiki.hpccsystems.com/display/hpcc/Johny+Chen+Jy+-+2020+Poster+Contest+Resources

And Fujio Turner did a TechTalk presentation in 2017 that also discussed this technique: https://hpccsystems.com/blog/download-tech-talks-hpcc-systems-community-episode-2

HTH,

Richard