HPCC Systems Marks its 10th Year Anniversary as Open Source
Ten years is a long time. A hundred generations of butterflies come and go in less than that, and events that reshaped the world happened in a fraction of that time. And yet, our beloved open source HPCC Systems big data platform and the community around it have existed for that long and thrived. Yes, it is true that HPCC Systems, the platform, was born around year 2000 and it would be almost reaching the legal age for drinking in the US at this time, but back then it was a proprietary system only used by a handful of organizations as a strategic differentiator at a time where big data was not even described as such. It wasn’t until 2011 that LexisNexis Risk Solutions and its parent company RELX decided to generously release the platform as an open source project.
During a recent podcast recording with my good friend Vijay Raghavan, who used the platform far before it was even open source, we had a stroll down memory lane remembering the early struggles when we repeatedly explained why open source could be seen as a competitive differentiator and how an open source community can introduce a significant amount of innovation and creative thinking, infusing any system with the essence needed to grow, adapt and even anticipate future needs.
Ten years later, our HPCC Systems community has validated time and time again that those core drivers were true; not only innovation became a central driver in the platform development community to introduce changes, which could be sometimes seen as radical, but it also brought a great deal of pride, which I’m sure factored into the increase in code quality, decrease in the number of bugs and improvement in the depth and breadth of new features included in new releases. And this is without even mentioning the caliber of the talent that the open source HPCC Systems platform initiative attracted, and not just to the community but also to LexisNexis Risk Solutions and RELX.
Not that it was all roses: in the early days we had our struggle to define the scope of the initiative, transform the software development process to accommodate a large group of external participants and created a significant volume of documentation and online training materials which could alleviate the potentially steep initial learning curve. Later, evangelism of the project and bringing new participants into the community took central stage, and each community member became an activist and advocate, doing everything that was in their hands to bring new members to the community. But it has never been about popularity, as Vijay very well said during our podcast. We have always known deep inside us, that we wanted to attract the right people to the community: the strength of an open source community is not counted in numbers but in talent, and this is what has made us very successful.
Now, 10 years later, the open source HPCC Systems platform looks very different from the one that we timidly set ourselves to release back in 2011. We adopted a number of standards, developed capabilities in a broad range of areas, from data profiling to machine learning and analytics, improved its security model and, now with the most recent 8.0 version release, made it fully capable of running on virtualized and containerized environments as its users drive their respective moves towards the public cloud.
As you can see, we didn’t waste any time. Ten years is a long time, but it’s dwarfed by the achievements of our HPCC Systems open source community. It has been a wild ride but a worthwhile one and I personally look forward to the next celebration, so happy anniversary HPCC Systems and I’ll see you in another ten years!
Be sure to catch all the interviews with members of the HPCC Systems community as part of the 10 year anniversary podcast series.
Meet the Participants
Vijay Raghavan is EVP and CTO for the portfolio of brands under LexisNexis® Risk Solutions Group. In this position, he is responsible for software development, R&D, information systems, security, and data center operations. Having spent almost 30 years in Information Technology, Vijay has extensive experience in the management of diverse technology functions, with oversight of the full software development lifecycle and support functions for high-availability applications, which are powered by high-performance supercomputing platforms. Previously, Vijay served as vice president, Insurance Technology, LexisNexis® Risk Solutions, where he was responsible for all aspects of application development and maintenance for the insurance products. Prior to joining the LexisNexis Risk Solutions, Vijay held senior executive positions at Paragon Solutions, where he was chief technology officer; and at McKesson, a Fortune 15 company, where he was vice president, Product Development within the Imaging Solutions Group. Vijay holds a B.E. in Electrical & Electronics Engineering from the Birla Institute of Technology and Science and completed an Advanced Executive Education Program in Management, Innovation, and Technology at MIT’s Sloan School of Management. LexisNexis Risk Solutions Group is part of RELX, a global provider of information and analytics for professional and business customers across industries.
Flavio Villanustre is CISO and VP of Technology for LexisNexis® Risk Solutions. He also leads the open source HPCC Systems platform initiative, which is focused on expanding the community gathering around the HPCC Systems Big Data platform, originally developed by LexisNexis Risk Solutions in 2001 and later released under an open source license in 2011. Flavio’s expertise covers a broad range of subjects, including hardware and systems, software engineering, and data analytics, and machine learning. He has been involved with open source software for more than two decades, founding the first Linux users’ group in Buenos Aires in 1994.
Learn more about this 10 Year Anniversary series on our wiki:
hpccsystems.com/10year-podcasts