Introduction

Regression testing plays a vital role in ensuring software reliability, especially in large-scale distributed systems like HPCC Systems. As test suites grow, so does the volume of artifacts generated during each run—most notably, workunits that record the results of individual test cases. These workunits, while essential for validation and debugging, can accumulate rapidly, leading to storage and memory bottlenecks that impact system performance and increase operational costs. To address this challenge, a collaborative team from RV College of Engineering, RV University, and HPCC Systems developed an Automated Cleanup Module for the HPCC Systems Regression Test Engine (RTE). This module automates the deletion of workunits based on configurable parameters, significantly reducing manual effort and improving resource utilization across both local and cloud environments.

The Challenge: Managing Workunit Overload

HPCC Systems RTE generates approximately 3,000 workunits per test run. These artifacts are stored locally or in the cloud, consuming valuable resources. Manual deletion via the ECLWatch interface is time-consuming and error-prone, especially when dealing with large-scale test suites.
While existing regression testing tools focus on test selection, prioritization, and execution, they often overlook post-test artifact management. This gap inspired the development of a solution that automates cleanup without compromising performance or reliability.

The Solution: Automated Cleanup Module

The cleanup module integrates directly into the RTE and introduces a new command-line parameter  “–cleanup” with three operational modes:

• none: No cleanup is performed.
• passed: Deletes workunits from passed test cases.
• workunits: Deletes all workunits from the current test run.

For example:

./ecl-test run --cleanup passed

This command deletes only the workunits associated with successful test cases. The module uses Python’s glob library to dynamically locate log files and extract workunit URLs. It then constructs deletion requests and sends them via HTTP POST to the ECLWatch interface. A custom logging system tracks each deletion, categorizing logs by severity for easy troubleshooting.

Implementation Overview

The cleanup module was designed with scalability and cross-platform compatibility in mind. It operates seamlessly in both Ubuntu-based local setups and Kubernetes-powered cloud environments. Key features include:

• Minimal overhead: Adds only ~15 minutes to process 3,000 workunits—just 5% of total RTE execution time.
• Robust error handling: Ensures uninterrupted operation under diverse runtime conditions.
• Custom logging: Provides detailed logs for each deletion, aiding in monitoring and debugging.

Figure 1: Overview of the automated cleanup module integrated into HPCC Systems RTE.

Operational Modes in Action

The module was tested across various HPCC Systems clusters—Thor, hThor, and ROXIE—and in both local and cloud environments. Below are examples of terminal outputs for each cleanup mode:

• “–cleanup none” mode:

Figure 2: Execution of RTE with none cleanup mode.

• “–cleanup passed” mode:

Figure 3: Execution of RTE with passed cleanup mode.

• “–cleanup workunits” mode:

Figure 4: Execution of RTE with workunits cleanup mode.

Validation and Logging

Workunit deletions were validated using the ECLWatch interface. Deleted workunits returned a “NotFound” status, confirming successful removal.

Figure 5: ECLWatch interface showing a deleted workunit.

The module also generates a cleanup log file after each RTE run, recording the IDs of deleted workunits and any errors encountered.

Figure 6: Sample log file generated by the cleanup module.

Performance Analysis

The cleanup module was benchmarked across individual clusters and all clusters combined. The results demonstrated minimal latency and consistent performance.

Figure 7: Runtime comparison of RTE with and without the cleanup module.

The module’s efficiency – 0.3 seconds per workunit deletion – indicates its scalability and suitability for large-scale deployments.

Final Thoughts

This project highlights the importance of automating maintenance tasks in distributed testing environments and the Automated Cleanup Module exemplifies how targeted innovations can enhance the efficiency and scalability of such systems. By automating a previously manual and error-prone process, this solution not only improves resource management but also contributes to the broader goal of sustainable and intelligent software engineering.

 Future enhancements may include:

• Scheduled cleanup: Allowing users to define cleanup intervals independent of test runs.
• Adaptive cleanup: Triggering deletions based on real-time disk usage thresholds.
• Platform generalization: Extending compatibility to other regression testing frameworks.

The module is now part of the HPCC Systems open-source GitHub repository, inviting community contributions and further development. Additionally, the outcomes of this initiative were presented in a conference paper at the 2025 Third International Conference on Networks, Multimedia and Information Technology (NMITCON-2025) – a proud milestone for the team!