The Download: Tech Talks by HPCC Systems – An Interview with Richard Chapman Part 2
Welcome to the ninth installment of the summer series of The Download: Tech Talks by HPCC Systems
This series focuses on information about the latest developments within HPCC Systems related to the cloud. HPCC Systems has been very active with projects over the past few months, getting better support for dynamic environments via container orchestration and other capabilities.
This final installment of the summer series of The Download: Tech Talks by HPCC Systems, features an interview with Richard Chapman (VP Technology, HPCC Systems Development – LexisNexis Risk Solutions), and Flavio Villanustre (VP and Chief Security Officer, LexisNexis Risk Solutions). In this interview, Richard and Flavio discuss the latest developments in the HPCC Systems “journey to the cloud.”
The full video recording of Flavio and Richard’s interview, “The Download Tech Talks by HPCC Systems – An Interview with Richard Chapman,” is available on the HPCC Systems YouTube channel, under 2020 Tech Talks.
The audio version is available on Soundcloud.
Meet our presenters
Flavio Villanustre is CISO and VP of Technology for LexisNexis® Risk Solutions. He also leads the open source HPCC Systems platform initiative, which is focused on expanding the community gathering around the HPCC Systems Big Data platform, originally developed by LexisNexis Risk Solutions in 2001 and later released under an open source license in 2011. Flavio’s expertise covers a broad range of subjects, including hardware and systems, software engineering, and data analytics and machine learning. He has been involved with open source software for more than two decades, founding the first Linux users’ group in Buenos Aires in 1994.
Richard Chapman has been with LexisNexis Risk Solutions for more than 30 years. He is the VP of Research and Development and the leader of the HPCC Systems development team. Richard wrote the code to create the HPCC Systems query cluster, ROXIE which stands for Richard’s Online XML Inquiry Engine. He was one of the original designers of ECL which was created as a data-centric programming language for easily expressing problems involving large quantities of data. He also wrote the plugins we currently provide for R, Java, MySql, Javascript and Cassandra.
This blog provides a brief summary of topics discussed in the interview, including:
- Latest Developments
- Current Projects
- Additional Information
Let’s begin by discussing the latest developments in the HPCC Systems “journey to the Cloud.”
Latest Developments
The current release of the HPCC Systems cloud native platform, version 7.10.0, is available for download. To see the full list of changes see the release notes. There is also a phenomenal blog by Lorraine Chapman, “What’s new in HPCC Systems 7.10.0 Gold,” that outlines the latest features.
The operating environment for the HPCC Systems cloud-native platform consists of Docker containers managed by Kubernetes. These containers are currently deployed by Helm charts on various cloud platforms, such as Azure (AKS), Amazon Web Services (AWS), and Google Cloud. Other Open Source deployment and management tools, such as Terraform, are currently being explored for use in the near future.
Version 7.12 of the cloud-native platform will be released in October 2020. This version will include a significant number of updates. Version 8.0, which is the “production” version,” is still on target for release in 2021.
Support also continues for existing “bare metal” HPCC Systems environments.
Now, let’s take a look at current projects.
Current Projects
Storage
Version 7.12.0 of the HPCC Systems cloud native platform is in development, and will allow more options for data storage, including cloud storage platforms like Azure blobs, Azure Data Lake, and Amazon S3 buckets.
ROXIE
Version 7.12.0 will include greater ROXIE cloud support. The team is currently working to determine the best way to deploy queries to ROXIE.
The development team is also exploring auto scaling in ROXIE, according to load. This would be accomplished in part by using automatic cache warming. Cache warming helps reduce execution times of queries, and automatically pre-warming the cache when executing a ROXIE query would make auto-scaling a lot more effective. This could be a significantly useful feature, not just in the cloud, but also in bare metal environments.
Additional Information
A list of resources related to HPCC Systems cloud native platform development can be found on the Cloud Native Wiki Page.
Acknowledgements
A special thank you to Richard Chapman and Flavio Villanustre for this informative discussion about the HPCC Systems cloud development project.