Home > Resources > HPCC Systems Platform: Highlights from Versions 9.8 to 9.14

The HPCC Systems platform continues to evolve rapidly, delivering powerful new capabilities with each release. From version 9.8 through 9.14, the platform has introduced a wide range of enhancements that improve system performance, developer productivity, and operational transparency. These updates reflect HPCC Systems commitment to supporting modern data engineering needs—whether you’re running large-scale analytics in the cloud, building secure enterprise applications, or improving the developer experience.

In this blog post, we’ll walk through the key highlights from each of the last four releases. Each version builds on the strengths of the previous one, offering new tools and refinements that make the platform more robust, flexible, and secure. Therefore, it is strongly recommended to run on the latest version of the platform for optimal performance and functionality. For more information on HPCC Systems version support, reference this page. 

Version 9.8 – Regex Overhaul, Interactive Learning, and Security Fixes

Version 9.8 introduced a mix of backend improvements and user-facing features that enhance both performance and accessibility. This release is especially impactful for developers working with text-heavy data, administrators managing secure environments, and educators looking to make ECL more approachable for new learners.

Change in Regex Support Library 

One of the most significant changes in 9.8 is the switch to the PCRE2 library for regular expression operations, replacing both Boost::regex and ICU. While all three libraries are powerful, PCRE2 offers better UTF-8 support, improved performance, and more consistent behavior across platforms. The ECL functions REGEXFIND, REGEXFINDSET, and REGEXREPLACE are mostly affected by this update.

While these changes are designed to eliminate silent errors and improve regex reliability, ECL developers are encouraged to review the Changes to Regex Support in HPCC Systems 9.8.0 and update their regex patterns as needed..

Change in LDAP Support

Version 9.8 also includes a critical fix for LDAP caching-enabled systems using default file scope permissions. Previously, the security manager cached these permissions inconsistently, which could result in incorrect access being granted or denied. This release corrects that behavior, ensuring that:

  • Default file scope permissions are applied consistently and predictably.
  • Workunit results are now repeatable and no longer affected by workload ordering.
  • Any failures due to denied access are now accurate and must be addressed by updating permissions or managing file scopes explicitly.

Administrators relying on the filesDefaultUser setting in the Dali LDAP configuration should also review their configurations, as this setting is deprecated and may be removed in future releases. For more information, check out Change in LDAP Support.

Interactive ECL Examples

To support learning and experimentation, many examples in the ECL Language Reference web pages are now interactive. Runnable examples in the documentation are marked with a play button, which opens the code in an ECL Watch Playground window. Users can modify the code, experiment with different values, and submit it to see the results. This iterative process helps users understand ECL code better, test it in real time, and expand their knowledge.

Note: Please avoid submitting proprietary algorithms or personal identifiable information (PII) when using the HPCC Systems Playground

Syntax Check in ECL Watch Playground 

A new Syntax button in the ECL Watch Playground allows users to validate code before execution. If the code passes, a green checkmark is displayed. If it fails, detailed error messages are shown—helping developers catch issues early and iterate faster.

Performance & Security Enhancements

As with every release, 9.8 includes numerous under-the-hood improvements to boost performance and tighten security. These updates ensure that the platform remains fast, reliable, and secure in both on-prem and cloud environments. 

Version 9.10 – Transparent Power, Simplified Deployment, and Cloud Optimization

Version 9.10 focused on delivering performance and cost-efficiency improvements that are largely transparent to end users but critical for system administrators and developers. These enhancements help organizations better manage cloud resources, streamline operations, and improve the overall responsiveness of the platform—while also simplifying deployment and addressing key stability concerns. 

Simplified Installers for Bare Metal 

 Prior to 9.10, users had the flexibility to install plugins on an à la carte basis. While this offered customization, it also introduced complexity in managing and maintaining individual components. In response to community feedback, HPCC Systems now provides a streamlined installer that integrates commonly used plugins directly into the core platform package. This change simplifies the installation process and reduces the risk of version mismatches or missing dependencies.

Note: If you’re upgrading from a previous bare metal installation with optional plugins, you may need to uninstall those plugins individually before proceeding with the new installer. 

Workunit Cost Tracking Enhancements 

Updates to workunit messaging and cost tracking provide more accurate insights into resource consumption. These improvements help teams better understand and manage operational expenses, especially in cloud environments where cost visibility is essential. 

Thor and Roxie Performance Improvements

  • Thor: Enhancements to disk read code and stack capture capabilities reduce I/O latency and improve job and workunit processing.
  • Roxie: Query handling and data retrieval speeds have been optimized to deliver faster response times and better failure recovery.

Azure Blob API and Storage Plane Flexibility

  • Azure Blob API Support: Native integration with Azure Blob Storage allows users to define storage planes using Microsoft object storage, simplifying hybrid and multi-cloud deployments.
  • Storage Plane Enhancements: Users can now define storage planes in both cloud and bare-metal environments, offering greater flexibility in how data is stored and accessed.

Improved Logging and Observability

Enhanced logging capabilities provide more detailed and accessible logs, making it easier to monitor system behavior, troubleshoot issues, and track costs. These improvements support better observability across distributed environments.

Developer Experience

  • Mac Compiler Fixes: Compilation issues on macOS have been resolved, improving the development experience for users on Apple hardware.
  • ECL Language Enhancements: Improvements to timing and error reporting make ECL development more intuitive and informative.

Rowservice Stability Fix

A critical issue was identified in the Rowservice component affecting versions 9.10.0 through 9.10.14. Under certain conditions, this vulnerability could cause the service to crash, potentially disrupting workloads. To ensure system stability and security, users are strongly encouraged to upgrade to version 9.10.16 or later, where this issue has been resolved.

If you’re using earlier minor versions of the platform, similar fixes are available in 9.6.84 or later and 9.8.56 or later. See HPCC-33381 for more details.

Version 9.12 – Maintenance, Monitoring, and Developer Tools

Version 9.12 delivers a powerful set of enhancements focused on system maintenance, observability, and developer experience. These updates are designed to help teams manage cloud-native and containerized environments more efficiently while also improving the tools available for debugging, diagnostics, and ECL development.

Sasha Debug Plane Housekeeping

A new maintenance service has been introduced for containerized deployments that automatically manages files and directories on the debug plane. This service uses a configurable retention period to clean up post-mortem and debug files, helping to:

  • Reclaim disk space
  • Maintain a clean and efficient environment
  • Log cleanup activity for transparency and auditing

This feature is especially useful in long-running or high-volume systems where debug data can accumulate quickly. For more details, please check the Guidelines for handling the Sasha server.

Sasha Interface in ECL Watch

ECL Watch now includes a user interface for performing Sasha operations such as archiving and restoring ECL and DFU workunits. This is a major usability improvement for containerized environments where the traditional Sasha command-line interface may not be accessible.

Note: This feature is restricted to administrators on secured systems.

XREF Interface in ECL Watch (Technical Preview)

A new interface for performing XREF (cross-reference) tasks is now available in ECL Watch. This feature allows administrators to manage metadata and dependencies directly from the UI.

Note: As a technical preview, this feature should be used with caution and is intended for early feedback and testing.

Event Recording and Analysis

HPCC Systems 9.12 introduces the ability to record and analyze index and Dali requests from Roxie, both at startup and during query execution. A new command-line tool allows users to access and analyze this event data, providing deeper insights into system behavior and performance. This feature enhances observability and helps teams fine-tune their systems based on real-time operational data. For more details, please check HPCC-33642.

New String Functions

Four new functions have been added to the ECL Standard Library to simplify string comparison tasks:

  • Std.Str.CommonPrefix
  • Std.Str.CommonSuffix
  • Std.Uni.CommonPrefix
  • Std.Uni.CommonSuffix

These functions help developers identify shared patterns in strings and are useful in a variety of data processing scenarios, including text normalization and data deduplication See CommonPrefix and CommonSuffix for more details.

ARM Support for macOS

The HPCC Systems client tools now support Apple’s ARM architecture, ensuring compatibility with the latest macOS devices. This update improves the development experience for users on Apple Silicon hardware.

Bloom Filter Enhancements

The default BLOOM filter generation has been improved to offer better performance and accuracy. Previously, the default bloom filter for an index used only the first field, which could be ineffective if that field was small—like a single byte. This was especially problematic in test indexes, such as the regression suite’s word search index. To improve effectiveness, the bloom filter now includes at least three bytes, so keys with longer fields (e.g., 8-character strings) will use the first three characters instead of just one. See HPCC-33869 for more details.

Version 9.14 – Smarter Sampling, Streamlined Observability, and Regex Extraction

The most recent release in this series, Version 9.14 continues the HPCC Systems tradition of delivering meaningful enhancements that improve performance, reduce cloud costs, and elevate the developer and operations experience. This release introduces new tools for observability, data sampling, and regex processing—along with platform-level optimizations that make working with large-scale data even more efficient.

Workunit Summary Page in ECL Watch

A new Workunit Summary view is now available in ECL Watch under Operations → Topology → WU Summary. This feature aggregates workunits (WUs) within a selected date range and groups them by unique error and warning types. It’s a powerful tool for operations teams to detect emerging issues across releases and proactively address them before they impact production workloads.

New ECL Function: REGEXEXTRACT

The ECL Standard Library now includes a built-in function called REGEXEXTRACT, which allows developers to extract matching text from STRING, UTF8, or UNICODE values using a regular expression. This function returns both the matched and unmatched portions of the input, making it easier to isolate relevant data patterns while preserving context. It’s a valuable addition for text processing, data cleaning, and pattern recognition tasks.

Platform API Enhancement: Block Compression Support

The platform now supports block compression configuration when reading files via the API. Compressed files respect the blockedFileIOKB setting on the storage plane, which can significantly reduce file I/O costs—especially when using the Azure API. This enhancement provides more control over performance tuning and cost optimization in cloud environments. For more details, see this page.

Managed Observability Helm Chart

A new Managed Observability solution is now available via Helm charts. This self-contained deployment leverages the Elastic Stack (Elasticsearch, Kibana, APM Server) and OpenTelemetry Collector to provide tracing and observability data for HPCC Systems clusters. This solution offers a secure, automated way to deploy and manage observability infrastructure—making it easier to monitor system behavior and performance.

Rowservice Random Sampling Support

Data scientists and developers can now take advantage of random sampling in the Rowservice component. Instead of reading an entire dataset, users can specify a sampling rate (e.g., 0.01 for 1%) to retrieve a representative subset of records. The sampling is randomized across the file to avoid bias and includes support for setting a recordSamplingSeed to ensure reproducibility. This feature accelerates model development and testing by reducing data volume without sacrificing representativeness.

Looking Ahead

From version 9.8 to 9.14, HPCC Systems has made significant strides in performance, security, and user experience. These updates reflect our ongoing commitment to building a powerful, open-source data lake platform that grows with your needs. Whether you’re a developer, data scientist, educator, or system administrator, there’s something in these releases to help you work more efficiently and effectively.

We encourage you to explore the full release notes and the red book for each version and try out the new features in your own environment. As always, we value your feedback and welcome contributions to the HPCC Systems open source project.