HPCC Systems Community Summit 2024

The 11th Annual HPCC Systems Community Summit took place virtually from October 7th to 11th, 2024. It delivered an inspiring and content-rich week that connected professionals, developers, researchers, and tech enthusiasts from across the globe. With a packed schedule of speakers and presentations, the summit provided an unparalleled platform for collaboration, innovation, and learning within the vibrant HPCC Systems ecosystem.

This year’s theme, “Igniting Innovation,” set a powerful tone, encouraging participants to challenge conventions, explore new horizons, and leverage the transformative potential of data analytics and HPCC Systems. Renowned industry experts took center stage, sharing invaluable insights through keynote addresses, interactive workshops, panel discussions, and breakout sessions; equipping attendees with actionable knowledge to fuel their personal and professional growth.

The agenda was carefully curated to offer something for everyone, whether the attendees were looking to dive deep into cutting-edge technical topics or explore the real-world case studies demonstrating HPCC Systems impact across diverse industries. There is a session here for everyone no matter what technical knowledge level you have!

Sessions were conveniently divided into four focused tracks:

A major highlight was the hands-on workshop, where attendees honed their skills with practical, in-depth code examples on ECL scheduling and process automation. This session not only reinforced key concepts but empowered participants to immediately apply their new skills in real-world scenarios.

The summit also celebrated the next generation of innovators, giving students the spotlight to present their unique projects built on HPCC Systems. Through poster presentations, these very talented students shared their groundbreaking ideas, received expert feedback, and competed for the award for best poster in their category. Their work exemplified the spirit of innovation that lies at the heart of the HPCC Systems community.

This blog offers an insider’s guide to the event’s presentations, helping you explore all of the sessions. Whether you missed the live event or want to revisit standout talks, this blog ensures you won’t miss a beat of the inspirational knowledge-sharing that defines the HPCC Systems Community Summit.

Plenary Sessions

Welcome and Plenary Keynote

Gavin Halliday, SVP and Head of Platform Engineering LexisNexis® Risk Solutions
Michael Stefanick, Managing Director
EY

Watch as Gavin Halliday kicked off the 11th annual HPCC Systems Community Virtual Summit, as well as hear from our community keynote speaker Michael Stefanick, who presented, AI + Human: How AI will Disrupt Work and Allow Workers to Thrive.

Awards Ceremony and Closing Plenary

Bob Foreman, Trish McCall, Hugo Watanuki, & George Foreman
LexisNexis Risk Solutions

Join the recap of the successful collaborations and engagements across the community and then hear the long awaited announcement of the 2024 Community Awards.

Platform Evolution

Take a look at the latest improvements and innovative features in the platform.

The Latest Advancements in HPCC Systems Observability

Rodrigo Pastrana & Mark Kelly
LexisNexis Risk Solutions

Learn about the latest advancements in HPCC Systems observability, focused on the Open telemetry-based instrumentation framework and how this feature can help you streamline your ECL query workload and ensure your HPCC Systems deployments are functioning at full strength!

How to Secure Your Containerized HPCC Systems Platform Using Terraform

Godji Fortil
LexisNexis Risk Solutions

In our presentations over the last couple of years, we showed you how to deploy the containerized HPCC Systems Platform. As a following step, knowing how to secure your cluster over the internet is very important. This presentation demonstrated how that can be done in the best ways possible using Terraform, htpasswd, or Azure Active Directory.

Parquet Plugin Usage / Test Suite for the Parquet Plugin

Jack del Vecchio & Ilhan Gelle
LexisNexis Risk Solutions

The HPCC Systems Parquet Plugin has received many performance improvements and bug fixes over the past year. In this two-part session, Jack highlighted the improvements compared to native file formats and went into detail about the usage of the Plugin and how to leverage the Parquet file format. Ilhan then provided an overview of the robust test suite for the HPCC Systems Parquet plugin and how it helps enhance the reliability and efficiency of Parquet integration within HPCC Systems.

What’s New – ECL Watch and IDEs

Kunal Aswani
LexisNexis Risk Solutions

Catch up on the latest changes to the interface and appearance in the world of ECL development.

Data DNA

See how these projects enhanced the integrating processes and capabilities of the platform to streamline, enrich and visualize data.

Streamlining Business Tasks with HPCC Systems Clusters and Tombolo

Yadhap Dahal & Matthew Fancher
LexisNexis Risk Solutions

Introducing Tombolo, an innovative open-source project designed to enhance the already robust features, speed, and capabilities of HPCC Systems clusters. Our aim is to introduce a user-friendly web application that caters to both technical and non-technical users, enabling seamless interaction with HPCC Systems clusters. During our presentation, we’ll explore the capabilities of Tombolo, including creating workflows and monitoring assets. Additionally in this session, we’ll highlight the application’s ability to proactively send timely notifications. Moreover, we’ll provide a comprehensive overview of Tombolo’s intuitive dashboard.

Power BI Integration with HPCC Systems

Harsh Raj & Srinivasan Kothandam,
LexisNexis Risk Solutions

Learn more about this integration package that enables analysts and business users to read HPCC Systems native files directly from Power BI.

Using NLP++ to Build a Brazilian Address Cleaner in HPCC Systems

Guilherme da Silva
LexisNexis Risk Solutions

Enhancing Legal Assistance Through Data Enrichment with HPCC Systems

Nihar Mandahas, Skanda P R, Manvith L B, Pratheek Rao MP, Arya Hariharan, & Dr. Jyoti Shetty
RVCE

Integrating Microsoft Fabric and HPCC Systems for Security Analytics

Sowmya Myneni & Kushi Kiran
LexisNexis Risk Solutions

Integrating HPCC Systems with Microsoft Fabric and Power BI offers a seamless workflow for transforming, linking, and visualizing data. HPCC Systems, with its user-friendly ECL language, efficiently handles complex data transformations, which can then be imported into Microsoft Fabric. From there, Power BI can be used to create interactive and insightful visualizations and reports. This integration simplifies the process of turning raw data into actionable insights, making it an effective solution for data analysis and presentation.

Productivity Tools

These projects highlighted tools and best practices for maximizing efficiency.

Implementing Conditional Cleanup after Regression Testing in HPCC Systems

Goutami Sooda, Arya Vinod, Ahana Patil, & Chandana S RVCE

Data 360° View Using HPCC Systems

S Dhanush & Shreyas Shankar
RVCE

Testing Best Practices of the HPCC Systems Platform (Now and in the Future)

Christopher Lo
LexisNexis Risk Solutions

Navigating the Platform Build System

Michael Gardner
LexisNexis Risk Solutions

From In-House to Open Source: The Journey of PyHPCC

Amila de Silva & Rohith Podugu
LexisNexis Risk Solutions

PyHPCC is a Python package and wrapper built around the HPCC Systems web services that facilitates communication between Python and HPCC Systems. It was originally developed as a LexisNexis Risk Solutions internal-only tool to automate repetitive work done on HPCC Systems. Since the evangelization of PyHPCC began in 2022, there has been growing interest across the organization and the broader HPCC Systems community to adopt PyHPCC.

Fueling Success

Here are the real world use cases and success stories leveraging HPCC Systems.

Building an NLP Pipeline for Electronic Health Records and Brain MRI Classification

Vishalakshi Prabhu, Eshaan Mathur, Nikhil Vasu, & Prashant Ronad
RVCE

Learned Cache Size Setting for Roxie Clusters

Yifan Wang
University of Hawaii

Machine Learning and Cybersecurity Analytics Using the NSL-KDD Dataset

Zularbine Kamal
Kennesaw State University

Model Inversion Attacks with the
HPCC Systems Platform

Andrew Polisetty
Kennesaw State University

School Safety and Security Using RFID and Drones

Taiowa Donovan & Nick Schwartz
American Heritage School

Exploring the Capabilities of HPCC Systems in Facilitating Inter Fog Communication

Henrique Antonio Buzin Vargas
Federal University of Santa Catarina (UFSC)

Internship to Impact: Real Life Success in the HPCC Systems Community

George S Foreman, Christopher Connelly, Jack del Vecchio, Yash Mishra, Nathalia Ribas, & Fulvio Favilla Filho
LexisNexis Risk Solutions

Join us for a moderated panel style interview with former interns who have emerged from the HPCC Systems Summer Internship Program. In this session, the next generation of technologists will share their experiences and successes working with the HPCC Systems community and learn from their experiences during their internship and how they transitioned to full time RELX employees.

2024 Community Award Winners

In our closing plenary session, we announced the recipients of our David Kan Ambassador and Community Recognition awards. These awards are presented to people we feel have made a huge difference and contribution to the HPCC Systems Open Source Project. Every year, there are many successes achieved by our colleagues and community members. All contributions are valuable to our community and this is one way we can show our appreciation for the commitment and hard work achieved during the year.

2024 Community Recognition Award

Dr. Jyoti Shetty, RV College of Engineering, India

Since 2017 Dr. Shetty and colleagues from the RV College of Engineering have partnered with HPCC Systems to support international educational programs whose projects use data for good, find sustainable solutions, and provide answers to questions relevant to today’s society. Throughout the years, the work supported by Dr. Shetty and colleagues has resulted in numerous research projects, published papers, and engagements with industry partners. These initiatives significantly impacted the HPCC Systems open source community and culminated with the creation of the center of excellence for sustainable solutions at the RV College of Engineering in 2022, where Dr. Shetty currently serves as a key contact for HPCC Systems-related engagements.

2024 David Kan Ambassador Award

Attila Vamos, LexisNexis Risk Solutions, UK

Attila joined the HPCC Systems team back in 2013 and has been instrumental in ensuring the quality of the platform’s development. Beyond his technical expertise, Attila has made remarkable contributions to the HPCC Systems Academic Program, especially as a dedicated volunteer mentor. Throughout the years, Attila has guided numerous students from around the globe on research projects and summer internships directly related to the HPCC Systems platform. By sharing his knowledge and providing invaluable guidance, Attila has not only fostered personal growth among these students but also contributed significantly to advancing their skills within our community.

Wrap up

We are so thankful for everyone who attended, participated in, and helped make the 2024 HPCC Systems Community Summit a resounding success. This year’s event wasn’t just a typical conference—it was a dynamic, immersive experience that expanded collective knowledge and pushed the boundaries of innovation in big data analytics through HPCC Systems.

It was truly inspiring to witness the enthusiastic engagement throughout the summit. Your active participation, thoughtful discussions, and insightful feedback on sessions elevated the event to new heights. The positive energy and collaborative spirit were palpable, reinforcing the strength of our growing community. We are deeply grateful for the many encouraging comments from attendees, highlighting how valuable and impactful the sessions were across all tracks and topics.

But the journey doesn’t end here! Stay tuned—in the coming weeks, we will be sharing additional content, including more blogs featuring the poster displays, and links to their video presentations, as well as highlights from the interactive workshop. These resources will offer another chance to dive deeper into the ideas and innovations that shaped this year’s summit.

Once again, thank you for being an essential part of the HPCC Systems community. Your passion, contributions, and dedication continue to fuel our shared mission of advancing data analytics and innovation. We look forward to seeing where we can go together in the months and years ahead!