The Download: Tech Talks by HPCC Systems – An Interview with Gavin Halliday
Welcome to the third installment of the summer series of The Download: Tech Talks by HPCC Systems
This series focuses on information about the latest developments within HPCC Systems related to the cloud. HPCC Systems has been very active with projects over the past few months, getting better support for dynamic environments via container orchestration and other capabilities.
The third installment of the summer series of The Download: Tech Talks by HPCC Systems, features an interview with Gavin Halliday (Enterprise Lead Architect, HPCC Systems Development), and Flavio Villanustre (VP and Chief Security Officer, LexisNexis Risk Solutions).
The full recording of Flavio and Gavin’s interview, “The Download: Tech Talks by HPCC Systems – An Interview with Gavin Halliday,” is available on the HPCC Systems YouTube Channel, under 2020 Tech Talks. The audio version is available on SoundCloud.
Meet our presenters
Flavio Villanustre – Flavio Villanustre is CISO and VP of Technology for LexisNexis® Risk Solutions. He also leads the open source HPCC Systems platform initiative, which is focused on expanding the community gathering around the HPCC Systems Big Data platform, originally developed by LexisNexis Risk Solutions in 2001 and later released under an open source license in 2011. Flavio’s expertise covers a broad range of subjects, including hardware and systems, software engineering, and data analytics and machine learning. He has been involved with open source software for more than two decades, founding the first Linux users’ group in Buenos Aires in 1994.
Gavin Halliday – Gavin has been with LexisNexis for 25+ years. He is one of the original architects of the HPCC System, and has worked on many parts of the system. He is currently the lead developer of the HPCC Systems code generator, which is the compiler for our proprietary programming language ECL. Gavin is the lead mentor for projects which involve making improvements in the code generator.
This blog provides a brief summary of topics discussed in the interview, including:
- Latest Developments
- File Storage on the Cloud
- Future File Storage Plans
- Additional Information
Let’s begin by discussing the latest developments in the HPCC Systems journey to the cloud.
Latest Developments
HPCC Systems has a new cloud native platform in development. Beginning with the 7.8.x series, HPCC Systems began providing native support for containerization. Containerization makes the system much easier to manage while taking advantage of cloud infrastructure.
The new operating environment consists of Docker containers managed by Kubernetes, along with continued support for “bare metal” installations. Helm charts are used to deploy these containers on any cloud platform that supports Kubernetes.
Point releases have been released weekly, but the “cloud native” platform is not production-ready. At this time, everything is still “proof of concept” level. Version 8.0, which will be the “production” version, will be released in 2021.
HPCC Systems version 7.10 will soon be available for download on the website.
Next, we will discuss file storage on the new cloud native platform.
File Storage on the Cloud Native Platform
In the initial releases of the HPCC Systems cloud native platform, the standard Helm charts used data volumes that were created when the Helm charts were installed, and destroyed when they were uninstalled. This was perfect for testing, but not ideal for persisting data.
Helm charts are currently structured in way that allows the user to run the HPCC Systems platform locally, in a distributed cloud environment, or other type of system with standard configurations, but enables the user to designate storage options for data. In current releases of the cloud native platform, example configurations are provided that illustrate how to persist data locally, and on various cloud storage platforms.
Gavin Halliday wrote an excellent blog that provides step by step instructions for “Persisting data in an HPCC Systems Cloud native environment.”
Now, let’s move on to how file storage solutions will be implemented on the HPCC Systems cloud native platform.
Future File Storage Plans
Work continues to provide flexible support for different types of file storage platforms, such as Microsoft Azure, Amazon Web Services, and Google Cloud. Since each cloud storage platform configures file storage in a different way, future plans include plugging in different implementations of these various file storage platforms into Helm charts.
Additional Information
Videos
The following videos provide additional details about the HPCC Systems cloud development project:
“The Download, Tech Talks by HPCC Systems: An Interview with Shamser Ahmed” – available on the HPCC Systems YouTube channel, under 2020 Tech Talks.
“The Download, Tech Talks by HPCC Systems: An Interview with Ming Wang” – available on the HPCC Systems YouTube channel, under 2020 Tech Talks.
“The Download, Tech Talks by HPCC Systems: An Interview with Gavin Halliday” – available on the HPCC Systems YouTube channel, under 2020 Tech Talks.
“The Download, Tech Talks by HPCC Systems: An Interview with Jake Smith” – available on the HPCC Systems YouTube channel, under 2020 Tech Talks.
Blogs
More information about HPCC Systems and the cloud can be found in the following blogs:
“The Download, Tech Talks by HPCC Systems: An Interview with Richard Chapman” – This is the first installment of the summer series of The Download: Tech Talks by HPCC Systems, featuring an interview with Richard Chapman (VP Technology, HPCC Systems Development at LexisNexis Risk Solutions), and Flavio Villanustre (VP and Chief Security Officer, LexisNexis Risk Solutions).
“HPCC Systems and the Path to the Cloud,” by Richard Chapman: In this blog, Richard Chapman talks about the “journey to the Cloud” and demonstrates how to set up a simple test cluster using a default Helm Chart.
“Setting up a default HPCC Systems cluster on Microsoft Azure Cloud Using HPCC Systems 7.8.x and Kubernetes,” by Jake Smith: This blog outlines the steps required to set up and run the HPCC Systems Helm Charts using the Azure Kubernetes Service (AKS).
“Persisting data in an HPCC Systems Cloud native environment,” by Gavin Halliday: This blog covers how to persist your data in an HPCC Systems cloud native environment.
Acknowledgements
A special thank you to Gavin Halliday and Flavio Villanustre for this informative update about the HPCC Systems cloud development project.