ECL Programmers Guide
   Next

ECL Programmers Guide

Boca Raton Documentation Team

© 2023 HPCC Systems®. All rights reserved

Copyright © 2023 HPCC Systems®. All rights reserved

We welcome your comments and feedback about this document via email to

Please include Documentation Feedback in the subject line and reference the document name, page numbers, and current Version Number in the text of the message.

LexisNexis and the Knowledge Burst logo are registered trademarks of Reed Elsevier Properties Inc., used under license.

HPCC Systems® is a registered trademark of LexisNexis Risk Data Management Inc.

Other products, logos, and services may be trademarks or registered trademarks of their respective companies.

All names and example data used in this manual are fictitious. Any similarity to actual persons, living or dead, is purely coincidental.

ECL Programming Concepts
Attribute Creation
Similarities and Differences
"Atomic" Programming
Growing Solutions
"Ugly" ECL is Possible, Too
Easy Optimization
Creating Example Data
Getting Code Files
Generating Files
Some Constants
The RECORD Structures
Starting Point Data
Generating Parent Records
Generating Child Records
Create the Nested Child Dataset Records
Write Files to Disk
Defining the Files
Cross-Tab Reports
A Simple CrossTab
A More Complex Example
A Statistical Example
Efficient Value Type Usage
Numeric Data Type Selection
String Data Type Selection
User-Defined Data Types
Using the GROUP Function
GROUP vs. SORT
Performance Considerations
Automated ECL
Using Text Files
Job "Failure"
Non-Random RANDOM
Working with XML Data
Simple XML Data Handling
Complex XML Data Handling
Input with Complex XML Formats
Piping to Third-Party Tools
Working with BLOBs
Spraying BLOB Data
Working with BLOB Data
Despraying BLOB Data
Using ECL Keys (INDEX Files)
Simple FETCH
Full-keyed JOIN
Half-keyed JOIN
Payload INDEXes
Computed Fields in Payload Keys
Computed Fields in Search Keys
Using an INDEX like a DATASET
Working With SuperFiles
SuperFile Overview
SuperFile Existence Functions
SuperFile Inquiry Functions
SuperFile Maintenance Functions
SuperFile Transactions
Other Useful Functions
Creating and Maintaining SuperFiles
Creating Data
A Simple Example
Nesting SuperFiles
Nested SuperFile Example
Data Consolidation
Getting SuperFile Components
Indexing into SuperFiles
SuperFiles vs. SuperKeys
There is a Problem
And the Solution Is ...
Using SuperKeys
Building SuperKeys
Creating a SuperKey
Using a SuperKey
Maintaining SuperKeys
Working With Roxie
Roxie Overview
Thor
Roxie
hThor
How to Structure Roxie Queries
Testing Queries
Deploying Queries to Roxie
SOAP-enabling Queries
The ECL Key to SOAP
Putting It All Together
Complex Roxie Query Techniques
Key Selection Based on Input
Keyed Joins
Limiting Output
SOAPCALL from Thor to Roxie
One Record Input, Record Set Return
Record Set Input, Record Set Return
Performance Considerations: PARALLEL
Performance Considerations: MERGE
A Real World Example
Controlling Roxie Queries
How Graphs Execute
ALLNODES vs. THISNODE
LOCAL vs. NOLOCAL
NOROOT Indexes
Query Libraries
Library INTERFACE Definition
Library MODULE Definitions
Building an External library
Using a Query Library
Query Library Tips
Restrictions
Notes on the implementation
Suggested Structure
Smart Stepping
Overview
Trailing Field Sorts
N-ary JOINs
Getting Things Done
Cartesian Product of Two Datasets
Records Containing Any of a Set of Words
Simple Random Samples
Hex String to Decimal String
File Layout Resolution at Compile Time
Using LOOKUP on a DATASET
Using LOOKUP in a RECORDOF function
Additional Details
Embedded Languages and Data stores
Code Signing, Embedded languages, and Security
ECLCC Configuration Settings
Code Signing
Managing Source Code
The Git Repository
The Git Improvements
Using Git with HPCC
Command Line Options
Security and Authentication
   Next
   ECL Programming Concepts