The following options are generally useful:
The following options are all about generating Logical graphs in a workunit.
Logical graphs are stored in the workunit and viewed in ECL Watch. They include information about which attribute/line number/column the symbols are defined in. Exported attributes are represented by <module>.<attribute> in the header of the activity. Non-exported (local) attributes are represented as <module>.<exported-attribute>::<non-exported-name>
| generateLogicalGraph | Default: false | If true, generates a Logical graph in addition to all the workunit graphs. |
| generateLogicalGraphOnly | Default: false | If true, generates only the Logical graph for the workunit. |
| logicalGraphExpandPersist | Default: true | If true, generates expands PERSISTed attributes. |
| logicalGraphExpandStored | Default: false | If true, generates expands STORED attributes. |
| logicalGraphIncludeName | Default: true | If true, generates attribute names in the header of the activity boxes. |
| logicalGraphIncludeModule | Default: true | If true, generates module.attribute names in the header of the activity boxes. |
| logicalGraphDisplayJavadoc | Default: true | If true, generates the Javadoc-style comments embedded in the ECL in place of the standard text that would be generated (see http://java.sun.com/j2se/javadoc/writingdoccomments/). Javadoc-style comments on RECORD structures or scalar attributes will not generate, as they have no graph Activity box directly associated. |
| logicalGraphDisplayJavadocParameters | Default: false | If true, generates information about parameters in any Javadoc-style comments. |
| filteredReadSpillThreshold | Default: 2 | Filtered disk reads are spilled if will be duplicated more than N times. |
| foldConstantCast | Default: true | If true, (cast)value is folded at generate time. |
| foldFilter | Default: true | If true, filters are constant folded. |
| foldAssign | Default: true | If true, TRANSFORMs are constant folded. |
| foldSQL | Default: true | If true, SQL is constant folded. |
| optimizeDiskRead | Default: true | If true, include project and filter in the transform for a disk read. |
| optimizeSQL | Default: false | If true, optimize SQL. |
| optimizeThorCounts | Default: true | If true, convert COUNT(diskfile) into optimized version. |
| peephole | Default: true | If true, peephole optimize memcpy/memsets, etc. |
| spotCSE | Default: true | If true, look for common sub-expressions in TRANSFORMs/filters. |
| noteRecordSizeInGraph | Default: true | Add estimates of record sizes to the graph |
| showActivitySizeInGraph | Default: false | Show estimates of generated c++ size in the graph |
| showMetaInGraph | Default: false | Add distribution/sort orders to the graph |
| showRecordCountInGraph | Default: true | Show estimates of record counts in the graph |
| spotTopN | Default: true | If true, convert CHOOSEN(SORT()) into a topN activity. |
| spotLocalMerge | Default: false | If true, if local JOIN and both sides are sorted, generate a light-weight merge. |
| countIndex | Default: false | If true, optimize COUNT(index) into optimized version (also requires optimizeThorCounts). |
| allowThroughSpill | Default: true | If true, allow through spills. |
| optimizeBoolReturn | Default: true | If true, improve code when returning BOOLEAN from a function. |
| optimizeSubString | Default: true | If true, don't allocate memory when doing a substring. |
| thorKeys | Default: true | If true, allow INDEX operations in Thor. |
| regexVersion | Default: 0 | If set to 1, specifies use of the previous regular expression implementation, which may be faster but also may exceed stack limits. |
| compileOptions | Default: none | Specify override compiler options (such as /Zm1000 to double the compiler heap size to workaround a heap overflow error). |
| linkOptions | Default: none | Specify override linker options. |
| optimizeProjects | Default: true | If false, disables automatic field projection/distribution optimization. |
| notifyOptimizedProjects | Default: 0 | If set to 1, reports optimizations to named attributes. If set to 2, reports all optimizations. |
| optimizeProjectsPreservePersists | Default: false | If true, disables automatic field projection/distribution optimization around reading PERSISTed files. If a PERSISTed file is read on a different size cluster than it was created on, optimizing the projected fields can mean that the distribution/sort order cannot be recreated. |
| aggressiveOptimizeProjects | Default: false | If true, enables attempted minimization of network traffic for sorts/distributes. This option doesn't usually result in significant benefits, but may do so in some specific cases. |
| percolateConstants | Default: true | If false, disables attempted aggressive constant value optimizations. |
The following options are useful for debugging:
| debugNlp | Default: false | If true, output debug information about the NLP processing to the .cpp file. |
| resourceMaxMemory | Default: 400M | Maximum amount of memory a subgraph can use. |
| resourceMaxSockets | Default: 2000 | Maximum number of sockets a subgraph can use. |
| resourceMaxActivities | Default: 200 | Maximum number of activities a subgraph can contain. |
| unlimitedResources | Default: false | If true, assume lots of resources when resourcing the graphs. |
| traceRowXML | Default: false | If true, turns on tracing in ECL Watch graphs. This should only be used with small datasets for debugging purposes. |
| _Probe | Default: false | If true, display all result rows from intermediate result sets in the graph in ECL Watch when used in conjunction with the traceRowXML option. This should only be used with small datasets for debugging purposes. |
| debugQuery | Default: false | If true, compile query using debug settings. |
| optimizeLevel | Default: 3 for roxie, else 0 | Set the C++ compiler optimization level (optimizations can cause the compiler to take a lot longer). |
| checkAsserts | Default: true | If true, enables ASSERT checking. |
| soapTraceLevel | Default: 1 | The level of detail in reporting SOAPCALL or HTTPCALL information (set to 0 for none, 1 for normal, 2 - 8 for more detail) |
| traceEnabled | Default: FALSE | Enables tracing to log files when TRACE actions are present. See TRACE. |
| traceLimit | Default: 10 | Overrides the the default KEEP setting for a TRACE statement to indicate how many TRACE statement to write to log file. See TRACE. |
The following options are for advanced code generation use:
These options should be left alone unless you REALLY know what you are doing. Typically they are used internally by our developers to enable/disable features that are still in development. Occasionally the technical support staff will suggest that you change one of these settings to work around a problem that you encounter, but otherwise the default settings are recommended in all cases.
| filteredReadSpillThreshold | Default: 2 | Filtered disk reads are spilled if will be duplicated more than N times. |
| foldConstantCast | Default: true | If true, (cast)value is folded at generate time. |
| foldFilter | Default: true | If true, filters are constant folded. |
| foldAssign | Default: true | If true, TRANSFORMs are constant folded. |
| foldSQL | Default: true | If true, SQL is constant folded. |
| optimizeDiskRead | Default: true | If true, include project and filter in the transform for a disk read. |
| optimizeSQL | Default: false | If true, optimize SQL. |
| optimizeThorCounts | Default: true | If true, convert COUNT(diskfile) into optimized version. |
| peephole | Default: true | If true, peephole optimize memcpy/memsets, etc. |
| spotCSE | Default: true | If true, look for common sub-expressions in TRANSFORMs/filters. |
| spotTopN | Default: true | If true, convert CHOOSEN(SORT()) into a topN activity. |
| spotLocalMerge | Default: false | If true, if local JOIN and both sides are sorted, generate a light-weight merge. |
| countIndex | Default: false | If true, optimize COUNT(index) into optimized version (also requires optimizeThorCounts). |
| allowThroughSpill | Default: true | If true, allow through spills. |
| optimizeBoolReturn | Default: true | If true, improve code when returning BOOLEAN from a function. |
| optimizeSubString | Default: true | If true, don't allocate memory when doing a substring. |
| thorKeys | Default: true | If true, allow INDEX operations in thor. |
| regexVersion | Default: 0 | If set to 1, specifies use of the previous regular expression implementation, which may be faster but also may exceed stack limits. |
| compileOptions | Default: none | Specify override compiler options (such as /Zm1000 to double the compiler heap size to workaround a heap overflow error). |
| linkOptions | Default: none | Specify override linker options. |
| optimizeProjects | Default: true | If false, disables automatic field projection/distribution optimization. |
| notifyOptimizedProjects | Default: 0 | If set to 1, reports optimizations to named attributes. If set to 2, reports all optimizations. |
| optimizeProjectsPreservePersists | Default: false | If true, disables automatic field projection/distribution optimization around reading PERSISTed files. If a PERSISTed file is read on a different size cluster than it was created on, optimizing the projected fields can mean that the distribution/sort order cannot be recreated. |
| aggressiveOptimizeProjects | Default: false | If true, enables attempted minimization of network traffic for sorts/distributes. This option doesn't usually result in significant benefits, but may do so in some specific cases. |
| percolateConstants | Default: true | If false, disables attempted aggressive constant value optimizations. |
| exportDependencies | Default: false | Generate information about inter-definition dependencies |
| maxCompileThreads | Default 4 for eclccserver and 1 for eclcc | Number of compiler instances to compile the c++ |
| reportCppWarnings | Default: false | Report warnings from c++ compilation |
| saveCppTempFiles | Default: false | Retain the generated c++ files |
| spanMultipleCpp | Default: true | Generate a work unit in multiple c++ files |
| activitiesPerCpp | Default 500 for Linux or 800 for Windows | Number of activities in each c++ file (requires spanMultipleCpp) |
| obfuscateOutput | Default false | If true, details are removed from the generated workunit, including ECL code, estimates of record size, and number of records. |
The following options are for the workunit analyzer:
| analyzeWorkunit | Default: true | If set to FALSE, disables analysis of the workunit |
| analyzer_minInterestingTime | Default: 1000 | Analyze activities that exceed this minimum time to execute (milliseconds) |
| analyzer_minInterestingCost | Default: 30000 | Report issues where the time penalty exceeds this value (milliseconds) |
| analyzer_skewThreshold | Default: 20 | Report skew related issues that exceed this threshold |
| analyzer_minRowsPerNode | Default: 1000 | Ignore activities that have this average number of rows per node |
Example:
#OPTION('traceRowXml', TRUE);
#OPTION('_Probe', TRUE);
my_rec := RECORD
STRING20 lname;
STRING20 fname;
STRING2 age;
END;
d := DATASET([{ 'PORTLY', 'STUART' , '39'},
{ 'PORTLY', 'STACIE' , '36'},
{ 'PORTLY', 'DARA' , ' 1'},
{ 'PORTLY', 'GARRETT', ' 4'}], my_rec);
OUTPUT(d(d.age > ' 1'), {lname, fname, age} );
//************************************
//This example demonstrates Logical Graphs and
// Javadoc-style comment blocks
#OPTION('generateLogicalGraphOnly',TRUE);
#OPTION('logicalGraphDisplayJavadocParameters',TRUE);
/**
* Defines a record that contains information about a person
*/
namesRecord :=
RECORD
string20 surname;
string10 forename;
integer2 age := 25;
END;
/**
Defines a table that can be used to read the information from the file
and then do something with it.
*/
namesTable := DATASET('x',namesRecord,FLAT);
/**
Allows the name table to be filtered.
@param ages The ages that are allowed to be processed.
badForename Forname to avoid.
@return the filtered dataset.
*/
namesTable filtered(SET OF INTEGER2 ages, STRING badForename) :=
namesTable(age in ages, forename != badForename);
OUTPUT(filtered([10,20,33], ''));