Home > Resources > Highlights included in HPCC Systems 7.12.0

Highlights included in HPCC Systems 7.12.0

Our latest HPCC Systems Series 7 release (7.12.x), includes more progress made on our journey to providing a Cloud Native HPCC Systems platform. This project has been the main focus of development since the beginning of 2020. For those interested in finding out more about this journey, read my blog about the developer offsite meeting earlier in the year, which includes details of the planned design discussions as well as the changes included in our first Cloud Native release 7.8.x. More features were added in the second release in this series, HPCC Systems 7.10.x.

This blog brings you up to date with the newest additions as we work towards our HPCC Systems 8.0.x series, due in 2021, which will be feature complete for running a production ready Cloud Native HPCC Systems environment. But we aren’t just focused on our Cloud Native platform, there are a number of fixes, features and enhancements in HPCC Systems 7.12.x, that are just as applicable and useful to our Bare Metal platform users.

HPCC Systems Bare Metal users can download the latest version of the platform in the downloads section of our website in the usual way. Release notes including the list of changes made are also available and new users or existing users looking to build on their ECL knowledge, will find access to online training courses on the HPCC Systems website.

Cross Platform Highlights

Whether you are using our Bare Metal or Cloud Native platform, there are a number of new features and enhancements that will be of interest to you:

Improvements to the Workflow Engine

Nathan Halliday joined the HPCC Systems Intern Program in 2020 to complete a project in this area. His contributions included a large number of test cases (for more details, look at this pull request and also this pull request) and an implementation of a new workflow engine. The ability to execute multiple workflow items at the same time created situations that could not previously occur (for example multiple graphs running at once, potential deadlock on resources), so many of Nathan’s other contributions fixed some of these issues (For more information see this pull request about lock acquisition and this one about threading issues.

The code is merged into the 7.12.x branch of the HPCC Systems GitHub repository, but not yet been enabled by default.  There are still a couple of issues with running multiple graphs in ROXIE and Thor that need to be resolved first.  Once those are addressed in a future release, there should be a significant number of Thor queries that will potentially run more quickly.

Find out more about Nathan’s project by reading his blog about the Workflow engine and view his poster, which won him the top prize in the 2020 HPCC Systems Poster Contest in the Best Poster – Platform Enhancement Category.

VS Code Improvements

Throughout 2020, we have been adding new features and enhancements to our ECL Extension for VS Code and the latest version is available from the Visual Studio Marketplace. More details of the feature highlights and enhancements recently added are included in this blog by Gordon Smith, Enterprise/Lead Architect, LexisNexis Risk Solutions Group. These include simplified launch configurations, new workunit history view, a results viewer, access to additional resources such as bundles and client tools, the ability to insert record definitions, localisations to make this tool more accessible to those who speak a language other than English.

There is also a note about previously added Syntax Colouring features that were adopted by GitHub earlier this year. More details are featured in this blog by Dan Abittan, Senior Software Engineer, LexisNexis Risk Solutions Group.

Standard Library support for generating a Globally Unique ID

A new feature has been added into the ECL Standard Library (View the documentation / Standard Library GitHub Repository) that allows ECL developers to generate globally unique IDs in a base58 format. More information is available in the related JIRA issue.

Metrics information has been added to the ECL Archive view in ECL Watch

This feature highlights hotspots in your ECL code and shows the metrics already available within the graph, next to the lines of ECL Code (when possible). More information about this feature is available in the related JIRA issue.

Cloud Platform Highlights

To be an early adopter of our new Cloud Native platform, find out how to get started using the Path to the Cloud blog, written by the leader of the HPCC Systems Development Team, Richard Chapman.

A whole series of cloud related blogs and resources are available on this wiki page, to help you learn more about, for example, configuring storage and persisting data, or getting started on specific cloud providers such as AWS, Microsoft Azure and Google Cloud. You will also find links to the Helm files and some How to… Videos.

HPCC Systems 7.12.x  takes us another step closer to providing a production ready Cloud Native platform. Here are the main new features and improvements:

Persistent storage

Persistent storage options in cloud installations are more mature and Landing Zones are now supported. To support the progress made in this area, Gavin Halliday, Enterprise/Lead Architect, LexisNexis Risk Solutions Group, has provided three blogs:

More information about the development work completed in this area can be found in the following JIRA issues:

Changes to ROXIE related to scaling and cache warming

Code has been added to Roxie to support preheating caches when autoscaling, so that addition of a new node to a cluster under load does not cause an initial dip in capacity until the new node’s cache is filled. More information is available in this related JIRA issue.

Security for cloud installations

Features added in this area included ingress/egress controls and secrets management.

New DESDL Features

There are two notable features to mention in this area:

  • HPCC Systems support for integrating Hashicorps vaults and Kubernetes secrets. A mechanism has been provided for managing credentials, keys and other secrets by the HPCC Systems and ECL language. More information is available in this related JIRA issue.
  • Initial Kubernetes support for Dynamic ESDL services. Adds a new ESDL ‘application’ allowing you to incorporate ESDL bindings into containers derived from the HPCC Systems core container. It provides a more manageable and scalable solution that DALI based ESDL binding deployments. More information is available in this related JIRA issue.

Getting involved

A number of our HPCC Systems Interns became early adopters of our Cloud Native platform, providing some valuable testing and feedback to the development team.

If you are also taking a look at our Cloud Native platform, do let us know. Raise any issues you find using our Community Issue Tracker and if you have any questions for the developers, you can post in the Developer or Cloud forums on the HPCC Systems Website.

Tags
Features & Tools