5 Questions with Richard Chapman
In this edition of “5 Questions” we interviewed Richard Chapman, one of the founders of HPCC Systems, an original designer of ECL, and the creator of the Rapid Online XML Inquiry Engine, otherwise known as ROXIE.
With more than 25 years of experience at LexisNexis Risk Solutions, Richard now serves as the company’s VP of Research and Development and head of development for HPCC Systems.
In our interview with Richard, we discussed the early years of HPCC Systems, some of his favorite use cases, updates and features of the platform, the beginnings of Thor, and where he sees the company 10 years from now.
What was your first memory of Thor?
Thor was created to prepare data for a previous incarnation of a supercomputing platform, which was an in-memory system that needed to have its data and child tables sorted. Unfortunately, we didn’t have a way of sorting the data, because it was too large and too slow to work with on the existing hardware of the time. This is what prompted us to create Thor.
How would you compare HPCC Systems today to what it was when it was first developed? What are some of the best features and updates that have been introduced over the years?
To a large extent, the similarities are more striking than the differences. The core philosophy of using a domain specific language, which is distributed over large quantities of hardware, is the exact design philosophy we had 20 years ago. The ECL language hasn’t changed enormously, but it has been extended. One of the biggest changes over the years was when we went open source and decoupled the ECL source control system from the ECL compiler. We’ve also added connectivity to third party or external systems and languages like Java, Python and MySQL, which has enabled us to extend the roles that ECL can play.
In terms of the best features and updates, I would say ROXIE, indexes, and child queries in Thor are my top three that have been introduced over the years.
Can you recall a favorite use case for HPCC Systems?
One of my favorite use cases was the Troy Graves search. This was where we were tasked with identifying an individual from information about completely unrelated events and correlating those events to where a particular person had been at the time of specific crimes. We ended up capturing a notorious serial criminal.
What is one piece of advice you would give young technologists entering the industry?
From my point of view, you know whether technology is something you’re interested in or not. It either comes naturally or it doesn’t. If it comes naturally and you’re truly passionate about becoming a technologist, it will be blindingly obvious because you’ll be spending all of your time hacking together solutions on computers.
Where do you see HPCC Systems 10 years from now?
I think that HPCC Systems will still be recognizable to those who are familiar with the system of today. It will be a process of evolution, rather than revolution, between now and then. I think I would expect to see more progress in terms of connectivity to third parties. I would also expect to see the parallel processing be even more significant. For example, rather than just operating across multiple machines, we’ll be utilizing the thousands of cores in every machine. I also think the HPCC Systems user interface will become sleeker and easier to use, but the basic power, language and data is only going to grow.
Want to hear more from Richard’s interview with Flavio Villanustre, VP of Technology, LexisNexis Risk Solutions? Listen to the webcast where Richard continues to discuss the changes HPCC Systems has undergone in recent years as well as some other interesting use cases.