Our 9th annual HPCC Systems Community Day Summit was a day packed with content covering a variety of topics, with something for all members of our open source community. All sessions were recorded, so we’d like to give you this opportunity to catch up any you may have missed, using the HPCC Systems YouTube Channel.

At this event, we announced our 2022 Community Award and Poster Contest winners in the Closing Plenary Session. We also ran a series of three Hands On Learning ECL Workshops which are available for you to watch and extend your knowledge of the ECL language.

This blog gives a guided tour of our Community Day presentations, providing you with everything you need to select the sessions you want to watch based on your specific areas of interest.

Plenary Sessions

Join our opening plenary sessions to watch presentations featuring LexisNexis Risk Solutions Technology Leaders and Community Keynote Speakers.

Join our closing plenary sessions to find out how we have been taking part in Hackathons to attract young coders to engage with technology as a potential future career. We also announce the recipients of our 2022 Community Awards and Poster Contest winners, followed by closing comments from Flavio Villanustre.

Machine Learning Sessions

These sessions include contributions to our Machine Learning Library, uses cases of HPCC Systems and ML algorithms and connectors that allow you to interact with HPCC Systems more effectively.

  • Machine Learning and Analytics Update
    Roger Dev – Senior Architect, LexisNexis Risk Solutions Group

    This talk focuses on the latest developments in the Machine Learning and Analytics Library:
    Gaussian Process Regression – A non-linear regression bundle, accelerated using Random Fourier Features. This is a flexible kernel-based regression method, with enhanced scalability.
    HPCC Systems Causality Toolkit – Our official release of the Causality toolkit. Provides synthetic data generation and powerful probability analysis, as well as causal model analysis, and causal inference.
  • Visual Analysis of Data Relationships (Minute Marker: 8.01)
    Roger Dev – Senior Architect, LexisNexis Risk Solutions Group

    The Joint Probability Space of a multi-variate dataset is a remarkably complicated object, encompassing everything that is knowable from the data alone. We use the powerful analytic and visualization capabilities of the HPCC Systems Causality Toolkit to examine the nature of probabilistic and causal relationships within datasets. We start by examining synthetic data with known relationships in order to recognize essential patterns. Then we move on to natural datasets to see how those patterns can be recognized, and how they differ from those observed in synthetic data.
  • Analysis on Medical Images: Colorectal Cancer Diagnosis
    Sarvesh Prabhu – 2022 HPCC Systems Intern and Lambert High School, Georgia USA

    Sarvesh joined the HPCC Systems Intern Program in 2022 to complete this project which involves analysing images collected by a smart pill swallowed by patients who are experiencing gastro-intestinal symptoms that need more investigation. Getting a consistently accurate diagnosis of GI issues is a challenge and Sarvesh’s work aims to provide a solution to this problem. Using HPCC Systems and our Machine Learning Library, Sarvesh compared Neural Networks and Random Forest to discover which might provide the best training and inference approaches for pragmatic business use cases.
  • PyHPCC
    Amila De Silva – Software Engineer III, LexisNexis Risk Solutions Group

    PyHPCC is a Python package that allows users of HPCC Systems, particularly beginners to interact with HPCC Systems quickly and easily without having to learn the ECL language. We are always looking for ways to diversify the capabilities of our platform and tools like PyHPCC are a great way to encourage users, particular new users, to realise the potential of using HPCC Systems for their data analytics solutions.

Cloud Deployment

Our Cloud Native platform is available and ready for you to use. These sessions provide you with information on getting up and running with some handy tips on recommended approaches and settings for your Cloud Native HPCC Systems clusters, by the developers who built the platform.

Cloud Delivery

This collection of presentations focuses on transitioning to our Cloud Native platform as an ECL developer, as well as providing a detailed look at some specific topics for Roxie users.

  • Investigating ROXIE Queries
    Krishna Turlapathi – Director Software Engineering, LexisNexis Risk Solutions Group

    This talk focuses on ROXIE architecture and how to understand more about ROXIE query performance by looking at various fields in the ROXIE logs, including using some new stats that have recently been added. An example of collecting and analyzing stats for a completed query is provided. Some interesting information is shared about our own experiences with containers and an important ROXIE optimization about graph dependencies and how that affects when activities can start.

Developer’s Corner

These sessions provide information about tools and tips to help you get the most out of your ECL Development.

  • ECL Source Control with GIT
    Greg Panagiotatos – Senior Software Engineer, LexisNexis Risk Solutions Group

    For many years, HPCC Systems has used the eclccserver to compile ECL code from GitHub repositories, using the Githook mechanism. The latest version of the platform contains enhancements to improve the capabilities and make it easy to use GitHub. The new multiple repository feature allows teams to develop code in their own independent repositories but have versioned dependencies between those repositories. Come and learn how to use Git with HPCC Systems and see the recent changes to the platform that significantly improve support for compiling from the Git repositories.
  • ECL Notebooks
    Jim DeFabia -Senior Consulting Software Engineer, LexisNexis Risk Solutions Group

    HPCC Systems now supports ECL Notebooks in VS-Code with the ECL Language Extension. ECL Notebooks are useful for interactive tutorials, demos, proofs-of-concept, and visualizations. ECL Notebooks allow you to create and share documents that contain narrative text and cells with “live” ECL code that other users can edit and run. ECL Notebooks are like Jupyter Notebooks, but they support ECL, and render and run inside of VS-Code (with the ECL extension). This presentation demonstrates how to create and use ECL Notebooks.
  • ECLS Scanner Tool
    Rahul Jain – Manager Software Engineer, LexisNexis Risk Solutions, India

    What is the ECLS tool? The ECL Scanner tool is currently in Beta version, but it is more than just a working model. It’s about an idea with some work in progress already. This tool is developed in C# .NET which connects to ECL GIT Repos for scanning .ecl files for unwanted imports. It is eventually an .exe file and hence can be installed on any local system. The goal of this presentation is to leverage this idea and current working methodology to help the HPCC Systems ECL developer community to save on code clean up time.

Beyond HPCC Systems

Presenters share and discuss tools and techniques that extend the power of HPCC Systems.

  • HPCC Systems to PowerBI Automated Connectivity
    Lee Saunders – Consulting Data Analyst, LexisNexis Risk Solutions Group

    Are you currently manually de-spraying CSV files to keep your Power BI reports up-to-date? Once upon a time our team was doing just that spending 5 days a month updating reports, now it takes minutes. In this talk we show: 
    •    A process to set up and publish ECL queries to generate a REST URL (JSON output). 
    •    How to link your Power BI reports for direct data refresh. 
    •    Demonstrate how gateways can allow a scheduled refresh of this data, removing the need for any manual intervention in report updates. 
  • Interfacing MongoDB into ECL (Minute Marker: 19.19)
    Jack Del Vecchio – 2022 HPCC Systems Intern and Bachelor of Computer Engineering, Miami of Ohio University

    Jack Del Vecchio joined the 2022 HPCC Systems Intern Program to work on this project. The goals were to develop a plugin that uses the HPCC Systems engine to query and return results from MongoDB. In this talk, Jack shares all the features that his plugin supports. MongoDB has some pretty cool things that you can do, allowing me to demonstrate the most useful features with some example code. Jack shows how his plugin is very versatile in what it supports and explains how it may be very useful for someone writing ECL code and trying to use MongoDB. Jack also shares his experience as an intern and what it was like to start developing with very limited  experience on ECL and HPCC Systems. 

2022 Community Award Winners

In our closing plenary session, we announced the recipients of our David Kan Ambassador and Community Recognition awards. These awards are presented to people we feel have made a huge difference and contribution to the HPCC Systems Open Source Project. Every year, there are many successes achieved by our colleagues and community members. All contributions are valuable to our community and this is one way we can show our appreciation for the commitment and hard work achieved during the year.

This award recognises external community members for their innovative use of HPCC Systems in their research, solutions and open source projects and also for their contributions to the HPCC Systems open source community. 

This award recognises RELX and LexisNexis Risk Solutions Group colleagues who have significantly promoted and contributed to the growth of our community, serving as strong supporters, proud evangelists, subject matter experts and champions.

In 2016, this award was named in memory of David Kan, one of the first and very dedicated HPCC Systems Ambassadors.

Congratulations to the recipients of our 2022 Community Awards

2022 Community Recognition Award

Congratulations Tyler Menezes Executive Director CodeDay

CodeDay is a nonprofit providing welcoming and diverse opportunities for under-served students to explore a future in tech. Tyler has been instrumental in providing opportunities for young technologists to use HPCC Systems, working with the team to supply coding challenges. He also encourages participants to join the HPCC Systems Intern Program. In 2021, a quarter of the places on that program went to CodeDay students and one of those went on to become a LexisNexis Risk Solutions employee.

2022 David Kan Ambassador Award

Congratulations Ming Wang Sr Consulting SWE LexisNexis Risk Solution Group

We present this award to Ming to say thank you for how he freely gives his time, to support our academic program and community users. His investment in the learning of students via our intern program and the research projects carried out by our academic collaborators is invaluable. We also want to show our appreciation for the way he does all this. Our team and users respect his knowledge, trust his experience and value his support. We thank you Ming for all you do in supporting the HPCC Systems Open-Source Community.