ECL Language Reference

Boca Raton Documentation Team

© 2023 HPCC Systems®. All rights reserved. Except where otherwise noted, ECL Language Reference content licensed under Creative Commons public license.

We welcome your comments and feedback about this document via email to <docfeedback@hpccsystems.com> subject to the HPCC Contribution Agreement at: hpccsystems.com/contribution. Please include Documentation Feedback in the subject line and reference the document name, page numbers, and current Revision Number in the text of the message.

LexisNexis and related logos, designs, trade dress, and trademarks are owned by Reed Elsevier Properties Inc. and its affiliates, used under license and not subject to the Creative Commons license. Other trademarks owned by their respective companies and not subject to the Creative Commons license.

All names and example data used in this manual are fictitious. Any similarity to actual persons, living or dead, is purely coincidental.

This document is licensed under the Creative Commons License CC BY-ND 3.0 applicable to the jurisdiction of the principal location of the user, as available; otherwise, the CC BY-ND 3.0 Unported (https://creativecommons.org/licenses/by-nd/3.0/).


Introduction
Documentation Structure
Documentation Conventions
ECL Syntax Case
Optional Items
Example Code
ECL Basics
Overview
Definitions versus Actions
Syntax Issues
Constants
String
Numeric
Compile Time Constants
Environment Variables
HPCC_DEPLOYMENT
Definitions
Definition Name Rules
Reserved Words
Definition Naming
Basic Definition Types
Boolean Definitions
Value Definitions
Set Definitions
Set Ordering and Indexing
Record Set Definitions
Record Set Ordering and Indexing
TypeDef Definitions
Recordset Filtering
Function Definitions (Parameter Passing)
Simple Value Type Parameters
SET Parameters
Passing DATASET Parameters
Passing DICTIONARY Parameters
Passing Typeless Parameters
Passing Function Parameters
Passing NAMED Parameters
Definition Visibility
"Global"
Module
Local
Field and Definition Qualification
Imported Definitions
Fields in Datasets
Scope Resolution Operator
Actions and Definitions
Expressions as Actions
Definitions as Actions
Actions as Definitions
Debugging Uses
Expressions and Operators
Expressions and Operators
Arithmetic Operators
Bitwise Operators
Bitshift Operators
Comparison Operators
Logical Operators
Logical Expression Grouping
An XOR Operator
Record Set Operators
Prefix Append Operator
Set Operators
String Operators
IN Operator
BETWEEN Operator
Value Types
BOOLEAN
INTEGER
INTEGER Value Ranges
REAL
REAL Value Ranges
DECIMAL
STRING
QSTRING
UNICODE
UTF8
DATA
VARSTRING
VARUNICODE
SET OF
TYPEOF
RECORDOF
ENUM
Type Casting
Explicit Casting
Implicit Casting
Type Transfer
Casting Rules
Record Structures and Files
RECORD Structure
In-line Record Definitions
Field Definitions
Field Inheritance
Field Modifiers
XPATH Support
DATASET
THOR/FLAT Files
CSV Files
XML Files
JSON Files
PIPE Files
Named Output DATASETs
In-line DATASETs
Single-row DATASET Expressions
Child DATASETs
DATASET as a Parameter Type
DATASET from DICTIONARY
DATASET from TRANSFORM
DICTIONARY
DICTIONARY Definition
DICTIONARY as a Value Type
INDEX
Keyed Access INDEX
Payload INDEX
Duplicate INDEX
Scope and Logical Filenames
File Scope
Foreign Files
Landing Zone Files
Dynamic Files
Temporary SuperFiles
Implicit Dataset Relationality
Alien Data Types
TYPE Structure
TYPE Structure Special Functions
LOAD
STORE
PHYSICALLENGTH
MAXLENGTH
GETISVALID
Parsing Support
Parsing Support
PARSE Pattern Value Types
ParsePattern Definitions
NLP RECORD and TRANSFORM Functions
Pattern References
XML Parsing RECORD and TRANSFORM Functions
Reserved Keywords
ALL
EXCEPT
EXPORT
GROUP keyword
IMPORT
KEYED and WILD
LEFT and RIGHT
LIKELY and UNLIKELY
ROWS(LEFT) and ROWS(RIGHT)
SELF
SHARED
SKIP
TRUE and FALSE
Special Structures
BEGINC++ Structure
ECL to C++ Mapping
Available Options
EMBED Structure
FUNCTION Structure
FUNCTIONMACRO Structure
INTERFACE Structure
MACRO Structure
MODULE Structure
Definition Visibility Rules
MODULE Side-Effect Actions
Concrete vs. Abstract (VIRTUAL) Modules
LIBRARY Modules
TRANSFORM Structure
Transformation Attribute Definitions
TRANSFORM Functions
Inline TRANSFORMs
Shorthand Inline TRANSFORMs
Built-in Functions and Actions
ABS
ACOS
AGGREGATE
TRANSFORM Function Requirements - AGGREGATE
How AGGREGATE Works
ALLNODES
APPLY
ASCII
ASIN
ASSERT
ASSTRING
ATAN
ATAN2
AVE
BUILD
Index BUILD Options
BUILD an Access Index
BUILD a Payload Index
BUILD from an INDEX Definition
Separating INDEX from DATASET
BUILD a Query Library
CASE
CATCH
CHOOSE
CHOOSEN
CHOOSESETS
CLUSTERSIZE
COMBINE
COMBINE TRANSFORM Function Requirements
COMBINE Form 1
COMBINE Form 2
CORRELATION
COS
COSH
COUNT
COVARIANCE
CRON
DEDUP
Complex Record Set Conditions
DEFINE
DENORMALIZE
DENORMALIZE TRANSFORM Function Requirements
DISTRIBUTE
"Random" DISTRIBUTE
Expression DISTRIBUTE
Index-based DISTRIBUTE
Skew-based DISTRIBUTE
DISTRIBUTED
DISTRIBUTION
EBCDIC
ENTH
ERROR
EVALUATE
EVALUATE action
EVALUATE function
Accessing Field-level Data in a Specific Record
EVENT
EVENTNAME
EVENTEXTRA
EXISTS
EXP
FAIL
FAILCODE
FAILMESSAGE
FETCH
FETCH TRANSFORM Function Requirements
FROMJSON
FROMUNICODE
FROMXML
GETENV
GLOBAL
GRAPH
GROUP
HASH
HASH32
HASH64
HASHCRC
HASHMD5
HAVING
HTTPCALL
IF
IFF
IMPORT
INTFORMAT
ISVALID
ITERATE
TRANSFORM Function Requirements - ITERATE
JOIN
JOIN Two Datasets
Matching Logic - JOIN
Options
Keyed Joins
Join Logic
TRANSFORM Function Requirements - JOIN
Join Types: Two Datasets
JOIN Set of Datasets
Record Matching Logic
TRANSFORM Function Requirements - JOIN setofdatasets
Join Types: setofdatasets
KEYDIFF
KEYPATCH
KEYUNICODE
LENGTH
LIBRARY
LIMIT
LN
LOADXML
LOCAL
LOG
LOOP
MAP
MAX
MERGE
MERGEJOIN
Matching Logic
Join Types:
MIN
NOLOCAL
NONEMPTY
NORMALIZE
NORMALIZE Form 1
TRANSFORM Function Requirements for Form 1
NORMALIZE Form 2
TRANSFORM Function Requirements for Form 2
NOFOLD
NOTHOR
NOTIFY
ORDERED
OUTPUT
OUTPUT Field Names
OUTPUT Thor/Flat Files
OUTPUT CSV Files
OUTPUT XML Files
OUTPUT JSON Files
OUTPUT PIPE Files
Named OUTPUT
OUTPUT Scalar Values
OUTPUT Workunit Files
PARALLEL
PARSE
PARSE Text Data
PARSE XML Data
Extended PARSE Examples
PIPE
POWER
PRELOAD
PROCESS
TRANSFORM Function Requirements - PROCESS
PROJECT
TRANSFORM Function Requirements - PROJECT
PROJECT - Module
PULL
RANDOM
RANGE
RANK
RANKED
REALFORMAT
REGEXFIND
REGEXFINDSET
REGEXREPLACE
REGROUP
REJECTED
ROLLUP
TRANSFORM Function Requirements - ROLLUP
ROLLUP Form 1
ROLLUP Form 2
ROLLUP Form 3
ROUND
ROUNDUP
ROW
ROW Form 1
ROW Form 2
ROW Form 3
ROWDIFF
SAMPLE
SEQUENTIAL
SET
SIN
SINH
SIZEOF
SOAPCALL
SOAPCALL Function
SOAPCALL Action
SORT
Sorting Algorithms
Quick Sort
Insertion Sort
Heap Sort
Stable vs. Unstable
Performance Considerations
SORTED
SQRT
STEPPED
STORED
SUM
TABLE
TAN
TANH
THISNODE
TOJSON
TOPN
TOUNICODE
TOXML
TRACE
TRACE Options
TRANSFER
TRIM
TRUNCATE
UNGROUP
UNICODEORDER
UNORDERED
VARIANCE
WAIT
WHEN
WHICH
WORKUNIT
XMLDECODE
XMLENCODE
Workflow Services
Workflow Overview
CHECKPOINT
DEPRECATED
FAILURE
GLOBAL - Service
INDEPENDENT
ONCE
ONWARNING
PERSIST
PRIORITY
RECOVERY
STORED - Workflow Service
SUCCESS
WHEN
Template Language
Template Language Overview
Template Language Statements
Template Symbols
#APPEND
#CONSTANT
#DECLARE
#DEMANGLE
#ERROR
#EXPAND
#EXPORT
#EXPORTXML
#FOR
#GETDATATYPE
#IF
#IFDEFINED
#ISDEFINED
#INMODULE
#LOOP / #BREAK
#MANGLE
#ONWARNING
#OPTION
Definition of Terms
Available options
#SET
#STORED
#TEXT
#UNIQUENAME
#WARNING
#WEBSERVICE
#WORKUNIT
External Services
SERVICE Structure
CONST
External Service Implementation
.SO Initialization
Entry Points
SERVICE Structure - external
Keywords
Data Types
Passing Set Parameters to a Service
Plugin Requirements
Deployment
Constraints
An Example Service
ECL definitions
.SO code module:
Index