What are High Performance Computing Clusters?
A High Performance Cluster Computing platform built for high-speed data engineering.
HPCC Systems key advantage comes from its lightweight core architecture. Better performance, near real-time results and full-spectrum operational scale — no need for a massive development team, unnecessary add-ons or increased processing costs.
View the many advantages HPCC Systems brings to the maintenance of your Data Lake or Big Data environment.
Harness the benefits of Kubernetes
Learn how using our containerized, cloud native platform can improve your current cloud deployments. Today’s HPCC Systems combines the usability of our legacy bare metal platform with the automation of Kubernetes to make it easy to set up, manage and scale your implementation.
|Runs on Kubernetes||New Storage Plane Architecture supports||Elasticity||Security|
|Support for Azure Kubernetes Service|
Support for Amazon Elastic Kubernetes Service
|Object Stores: AWS Simple Storage Service (S3) and Azure Blob Storage|
Disk Stores: AWS Elastic Block Storage and Azure Files/Azure Disks
|Scaling a cluster without moving the data|
Auto wakeup to enable on demand processing by compute resources
|End to end encryption|
Service Mesh Options (Linkerd and Istio)
OAuth 2.0 support for Authentication, with built in support for Azure AD
Visit the Cloud Native Wiki page for access to Helm charts, blog content, videos and other instructional information.
A Virtual Machine containerized deployment with Docker Desktop or Minikube is an excellent resource for experimenting, evaluating and training on the HPCC Systems platform.
Documentation useful for cloud-based deployments featuring Terraform, Helm, and other deployments (large or small) as well as local testing and development deployments.
HPCC Systems key advantage comes from its lightweight core architecture. Better performance, near real-time results and full-spectrum operational scale — without a massive development team, unnecessary add-ons or increased processing costs.
HPCC Systems is an open source platform for big data implementations,
whether as a data lake or data warehouse, providing users with a
clear path from data discovery to production.
Data lakes are helping leading organizations solve the
problem of extremely large, unstructured datasets,
allowing them to increase responsiveness and
scalability while reducing costs.
A Comparative analysis of Spark and HPCC Systems including the architectures and feature support of Spark and HPCC Systems in regard to data lake capabilities and their focus on different parts of the big data pipeline.
Code Less — Accomplish More
A declarative programming language, ECL allows a programmer to express the logic of a computation without describing its flow control. Developers tell the system what they need, but leaves it up to the system to determine the best way to do it.
With the efficiency of ECL, fewer lines of code allows prototypes that can be iterated quickly, speeding both time to market and time to revenue.
Try our Enterprise Control Language (ECL), the data-oriented programming language specially designed for data processing and analytics.
From free training courses to rich community resources and a comprehensive wiki, we have all the resources for every stage from initial installation all the way to power user.
Machine Learning Library and Causality Analytics
The ML Library provides a wide range of algorithms and is designed to utilize the parallel computing capabilities of HPCC Systems. Build and test ML models and to use those models to predict qualitative or quantitative values.
A quick but potent intro to Machine Learning for those who are new to the subject. This article provides enough of the basic theory and terminology to make you dangerous.
Follow along with our trainers as they demonstrate our DBSCAN, K-Means, Logistic and Linear Regression, Generalized Neural Networks and Learning Trees bundles.
The HPCC Systems Machine Learning Library provides a wide range of Machine Learning algorithms accessible from ECL, and designed to utilize the parallel computing capabilities of HPCC Systems.
Integrate with Ease
HPCC Systems continues to develop new plugins, connectors and stand-alone applications which are free to the community to help you integrate many popular third party tools with the HPCC Systems platform.
HPCC Systems continues to develop new stand-alone applications and plug-in modules that extend the capabilities of the base HPCC Systems platform.
An ECL Bundle is a self-contained set of ECL files, designed to accomplish specific tasks. They are encapsulated for versioning, distribution and download.
Use embedded languages and external datastores with HPCC Systems to integrate your system to your data.
Use your favorite language or data source
ECL is very flexible. You can embed a number of different languages within your ECL code and process data on a HPCC Systems cluster from a variety of different sources using the various plugins and connectors we provide specifically to help you bridge the gap.
How flexible is ECL? Read about supported languages, plugins and connectors.
The full list of supported languages, plugins and connectors, including links to other information you might find useful.
Learn how ECL makes it easy to transition between declarative and procedural worlds through use of embedding.
Committed to Open Source Innovation
Freely available to the open source community for more than 10 years and licensed under Apache 2.0, we continue to push the boundaries of Big Data with a vibrant development community both online and in academic institutions.
HPCC Systems is an open source, massive parallel-processing computing platform for Big Data processing and analytics.
Receive peer to peer support on our Stack Overflow forums. Ask questions specific to your development or read and answer questions others have posted.
The HPCC Systems Team collaborates with multiple colleges, universities, high schools and institutions of higher learning around the world to help train and develop the future managers of Big Data projects.
Proven, Stable and Secure
HPCC Systems is a mature platform that has been heavily used in commercial applications for more than two decades, predating even the development of Hadoop. Created by LexisNexis Risk Solutions, an innovative pioneer in big data processing,
Detail on the many security features that make HPCC Systems a compelling solution for users that require a robust, configurable, highly secure computing platform.
Conduct curation and governance operations in an automated fashion to consistently and reliably curate huge amounts of inbound new data and ensure continuous availability.
Blog discussing some of the basic security considerations to properly secure a Big Data platform from unauthorized access or data theft.
Want to do a little more testing before you install a full cluster? If you’re ready to start building your Data Lake, you can jump straight to learning about how to install your first complete HPCC Systems cluster. Interested in learning just how powerful, flexible, and efficient ECL really is? Take a look at our ECL guide.
Containerized deployments using Docker Desktop or Minikube are easier to start up locally and provide more flexibility and stability.
Documentation & Training
Tackling big data problems? We’ve got you covered, with documentation and training to support you from initial installation all the way to power user.
Get Up and Running
Get a high level overview to help new users get started with HPCC Systems and ECL (Enterprise Control Language).
Test our code in a virtual playground using a sample dataset. Or, create your own high performance computing cluster (Thor) and/or query cluster (Roxie).
HPCC Systems: The End-to-End Data Lake Management Solution
Ready. Set. Go.
Are you ready to get started using HPCC Systems? Use the panels below to get a quick overview of the HPCC Systems platform, learn about how you can ingest, clean and deliver your mixed schema data to make it useful and relevant for both you and your customers.
A legacy of Innovation and Open Source software for more than 10 years
Freely available to the open source community for more than 10 years and licensed under Apache 2.0, we continue to push the boundaries of big data with a vibrant development community both online and in academic institutions.
Have a Question?
Check out our FAQ page. Browse the topics to discover more about HPCC Systems technology and answers to common questions about HPCC Systems, ECL and more.