Welcome to the seventh installment of the summer series of The Download: Tech Talks by HPCC Systems
This series focuses on information about the latest developments within HPCC Systems related to the cloud. HPCC Systems has been very active with projects over the past few months, getting better support for dynamic environments via container orchestration and other capabilities.
The seventh installment of the summer series of The Download: Tech Talks by HPCC Systems, features an interview with Anthony Fishbeck (Software Engineer, HPCC Systems Development, LexisNexis Risk Solutions), and Flavio Villanustre (VP and Chief Security Officer, LexisNexis Risk Solutions).
The full video recording of Flavio and Anthony's interview, “The Download: Tech Talks by HPCC Systems - An Interview with Anthony Fishbeck,” is available on the HPCC Systems YouTube channel, under 2020 Tech Talks.
The audio version is available on Soundcloud.
Meet our presenters
Flavio Villanustre is CISO and VP of Technology for LexisNexis® Risk Solutions. He also leads the open source HPCC Systems platform initiative, which is focused on expanding the community gathering around the HPCC Systems Big Data platform, originally developed by LexisNexis Risk Solutions in 2001 and later released under an open source license in 2011. Flavio’s expertise covers a broad range of subjects, including hardware and systems, software engineering, and data analytics and machine learning. He has been involved with open source software for more than two decades, founding the first Linux users’ group in Buenos Aires in 1994.
Anthony Fishbeck has been with LexisNexis Risk Solutions for more than 19 years. He was the original architect of the ESP platform and ESDL interfaces, and has since worked on several HPCC Systems components. He has an affinity for the implementation of standards and protocols, and has added JSON and REST support throughout the HPCC Systems platform. He has been adding support for a streamlined scriptable enterprise integration engine. Lately he’s been working on Kubernetes support for HPCC secrets, vaults, and a streamlined Kubernetes deployment mechanism for ESP.
This blog provides a brief summary of topics discussed in the interview, including:
- Latest Developments
- Current Projects
- Additional Information
Let’s begin by discussing the latest developments in the HPCC Systems “journey to the Cloud.”
The HPCC Systems Development team continues to make great strides in the development of HPCC Systems “cloud native” platform. The operating environment for this platform consists of Docker containers managed by Kubernetes. These containers are currently deployed by Helm charts on cloud platforms that support Kubernetes, such as Azure (AKS) and Amazon Web Services (AWS).
Other Open Source deployment and management tools, such as Terraform, are being explored for future use. There is also continued support for “bare metal” installations.
The latest version of the HPCC Systems “cloud native platform” is available for download on the website. The current version of the platform is not production-ready. The “production” version, which is version 8.0, will be released in 2021.
Now, let’s take a look at current projects.
As HPCC Systems moves to the cloud, one of the efforts taking place is streamlining the configuration and deployment of ESP for cloud deployment. The ESP (Enterprise Services Platform) is the container that allows multiple services to be “plugged in,” providing various types of functionality to client applications. The ESP essentially runs web services for internal and external use within an HPCC Environment. For more information on the ESP platform, check out the blog, “When is an ESP not an ESP?,” by Tony Fishbeck.
In bare metal environments, complex ESPs are deployed using the configenv tool. These configurations are static, and require various combinations of services, configurations, plugins, and security, with multiple applications running on multiple ports, for one process.
However, this does not work in a cloud-based environment. Additionally, this complex way of deploying ESPs does not take full advantage of the features that a cloud platform offers. So, the current focus is on configuring ESP applications for easy deployment on the cloud. This new mechanism requires minimal adjustments to the configurations before deployment, otherwise allowing the user to focus only on the things that they want to add or change.
There are also projects for the HPCC Systems native cloud platform that are centered around security. Secrets management and certificate management are two areas of security-related concentrations that are currently in development.
Kubernetes Secrets is currently being used for secrets management. A Secret is an object that contains sensitive data such as a password, token, or key. Storing confidential information in a Secret is safer and more flexible than putting the information in a Pod definition or a container image.
Another project in the works for the HPCC Systems cloud native platform is the streamlining of the enterprise integration layer. Enterprise integration allows for interaction and communication between applications in service-oriented architecture (SOA). In the streamlined process for enterprise integration, everything becomes scriptable. An object is able to be modified as it flows through the various stages of a query.
The user is able to:
- Obtain security settings
- Check the fields
- Verify the validity of requests and abort if they are invalid
- Augment the request with information from the account settings so that Roxie knows how to process the request
- Modify the response before it goes out to the user
- Modify and control the logging behavior on the backend
For additional information about the HPCC Systems cloud development project click here.
A special thank you to Anthony Fishbeck and Flavio Villanustre for this informative update about the HPCC Systems cloud development project.