Fri Feb 22, 2019 5:20 am
Login Register Lost Password? Contact Us

Updating a DATASET as an action tied to a RETURN

Comments and questions related to the Enterprise Control Language

Wed Jan 16, 2019 5:30 pm Change Time Zone

Simple issue - I hope,

I want to append to a DATASET a record on returning from a FUNCTION.
I have:
Code: Select all
NamesUsed := DATASET([],{STRING namelist});

   Exist := COUNT(NamesUsed(namelist = nm));
   RETURN WHEN(nm+IF(Exist > 0,(STRING)Exist,''),EVALUATE(NamesUsed := NamesUsed & ROW({nm},{STRING namelist})));

But this is not right.
Any ideas?
Posts: 342
Joined: Sat Oct 01, 2011 7:26 pm

Wed Jan 16, 2019 6:49 pm Change Time Zone


I would just do it like this:
Code: Select all
NamesUsed := DATASET([{'Fred'},{'Joe'}],{STRING namelist});

NamesToTry := DATASET([{'Fred'},{'Sam'}],{STRING namelist});

MergedNames := NamesUsed & NamesToTry(nameList NOT IN SET(NamesUsed,namelist));


Or, if you have tons of records in each, then you could do it like this:
Code: Select all
AllNames := NamesUsed & NamesToTry;
UniqueNames := DEDUP(SORT(AllNames,namelist),namelist);


Community Advisory Board Member
Community Advisory Board Member
Posts: 1409
Joined: Wed Oct 26, 2011 7:40 pm

Thu Jan 17, 2019 9:34 am Change Time Zone


No, does not work, doing:
Code: Select all
NamesUsed := DATASET([],{STRING namelist});
   Exist := COUNT(NamesUsed(namelist = nm));
   NamesUsed := NamesUsed & ROW({nm},{STRING namelist});
   RETURN nm+IF(Exist > 0,(STRING)Exist,'');

gives error:
Code: Select all
Error:    syntax error near ":=" : expected ';' (385, 18), 3002, Reports.DataGetterTransformer

Code: Select all
xx := NamesUsed & ROW({nm},{STRING namelist});

I don't want any dedup as I using the COUNT of the number of times the 'name' has been found to help generate a unique name in a record structure I'm creating in a FUNCTIONMACRO.
I need to record how many times a field name has been encountered so I can generate a 'fieldname<n>' where 'n' is a COUNT of the number of times 'fieldname' has already been processed. Consequently I need a side-effect action to update my list of fieldnames every time I process a fieldname.
I can't just use #UNIQUENAME or the COUNTER in the ITERATE as I want to retain the original fieldnames as closely as possible. So if 'fieldname' is only encountered once the fieldname will be unaltered from the input fieldlist to the FUNCTIONMACRO.

Posts: 342
Joined: Sat Oct 01, 2011 7:26 pm

Mon Jan 21, 2019 8:42 am Change Time Zone

Actually for my specific requirement, the built in PROCESS works fine.
Posts: 342
Joined: Sat Oct 01, 2011 7:26 pm

Tue Jan 22, 2019 2:09 pm Change Time Zone


Glad that PROCESS worked out for you. Can you post example code for what you're doing (so all the world can see)?


Community Advisory Board Member
Community Advisory Board Member
Posts: 1409
Joined: Wed Oct 26, 2011 7:40 pm

Wed Jan 23, 2019 9:49 am Change Time Zone

A section of ECL, that takes the XML of a record structure and flattens it into another record structure. Details in the ECL comments. Note the code only flattens the 1st record of any child datasets, which is ok for our functionality, but limits this codes use in the general case.

Code: Select all
EXPORT RunScoreStats(TYPEOF(LogLayouts.content_type) pExternalVendor
                    ,STRING xmlOfRecordStructure
                    ,STRING rootFldToScores
                    ,STRING ECLForStat) := FUNCTION
       Flatten vendor scores into a format that is usable by stat generating attributes,
       and actually daisy-chain off a WU to generate the stat.

       pExternalVendor       Filter for the specific vendor being analysed.
       xmlOfRecordStructure  The XML of the record structure of the data holding the scores to be processed.
       rootFldToScores       The scores can be nested quite deep inside the record structure, so a 'root' to that
                             structure needs to be supplied.
                                Vendor1 Scores are under:
                                      <Row><XXX>scores are all here</XX></Row>       // (Some tags removed for clarity of explanation)
                                So if the entire structure of the XML score structure is supplied in xmlOfRecordStructure
                                then the 'root' to the structure to flatten is 'XXX'.
                                If the <XXX> structure alone is supplied in xmlOfRecordStructure then this
                                rootFldToScores must be ''. 
       ECLForStat            Text of ECL to generate the actual stat. Make any reference to the flattened input in the ECL as 'input'.

       An EXPORTed routine named 'pExternalVendor' must exist in this MODULE.
       It must return a DATASET in the format as described by 'xmlOfRecordStructure'.

       1.     stat WU initiated
       2.     WUid of said WU.

  ExternalVendor := TRIM(pExternalVendor,LEFT,RIGHT);
  VendorList     := [LOGCONST.Vendor1Response,LOGCONST.Vendor2Response,LOGCONST.Vendor3Response,LOGCONST.Vendor4Response];

  // Check inputs

  ValidExternalVendorID := ASSERT(ExternalVendor IN VendorList,'Unregognised External Vendor',FAIL);

  Checks := ORDERED(ValidDates,ValidExternalVendorID);

  // Construct ECL to convert this hierarchical structure into a vertical slice record structure.
  // This will flatten the entire response structure into <fldName>,<value> pairs.


GetRecStructure := FUNCTIONMACRO

    /* Cope with duplicate field names.
       As the flattened structure must give a unique name to each of its fields, the input hierarchical structure may well have the same
field name in different parts of its hierarchy.
       To Solve this, retain a list of field names and generate a field name from the input field name but append the COUNT of that
field name already encountered in the input.

LEFT constructs the string to be the right hand side of each flattened assignment, e.g.
       string dateraised8 := __datasetName__11979__.Results.addresslinks[1].notices[1].dateraised;

       RIGHT side retains the current state of referencing down to the fields currently having assignments generated for them. e.g.

      This is an improvement on just using COUNTER to the ITERATE to generate unique fieldnames as the original field names
are retained as far as possible.

    RRec := RECORD
        STRING ecltype    := XMLTEXT('@ecltype');
        BOOLEAN isRecord  := (BOOLEAN) XMLTEXT('@isRecord');
        STRING label      := XMLTEXT('@label');
        STRING name       := XMLTEXT('@name');
        STRING position   := XMLTEXT('@position');
        STRING rawtype    := XMLTEXT('@rawtype');
        STRING size       := XMLTEXT('@size');
        STRING ttype      := XMLTEXT('@type');
        BOOLEAN isEnd     := (BOOLEAN) XMLTEXT('@isEnd');
        STRING FldEntry   := '';

    OnRec := {STRING FldEntry};
    InputDSToFlattener := PARSE(DATASET([{xmlOfRecordStructure}],OnRec),FldEntry,RRec,XML('Data/Field'));

    rootToFldNormalised := REGEXREPLACE('(^\\.{1,}|\\.{1,}$)',rootFldToScores,'');      // Normalise path to root of scores, i.e.  .fred.charlie.  => fred.charlie
    StartStack := DATASET([%'datasetName'%+'.'+rootToFldNormalised+IF(rootToFldNormalised != '','.','')],OnRec);

    RDS  := {DATASET(OnRec) namelist;
             DATASET(OnRec) DirEntry};

    RRec RecordFlattener(RRec L,RDS R) := TRANSFORM

      // Note this only works if all input records root field type is 'string'. (i.e. still ok if field type is STRING12)
      // If not the test L.ttype = 'string' will have to be enhanced to truly detect a 'terminal' field type.
      STRING GetName := + IF(COUNT(R.namelist(FldEntry = > 0,(STRING) COUNT(R.namelist(FldEntry =,'');
SELF.FldEntry  := IF(L.ttype = 'string','string '+GetName+' := '+ConcatenateStringFields(R.DirEntry,FldEntry,'')';','');
SELF           := L;

    RDS RetainHierarchicalState(RRec L,RDS R) := TRANSFORM

IsDataSet     := REGEXFIND('table of',L.ttype);
SELF.Namelist := R.namelist(FldEntry != '') & ROW({},OnRec);
SELF.DirEntry := MAP(IsDataSet        => R.DirEntry & ROW({'[1].'},OnRec)
          ,L.isRecord       => R.DirEntry & ROW({'.'},OnRec)
          ,L.isEnd          => R.DirEntry[1..COUNT(R.DirEntry)-1]
          ,                    R.DirEntry);

    irStruct := PROCESS(InputDSToFlattener
                       ,STABLE)(fldEntry != '');

    RETURN ConcatenateStringFields(irStruct,fldEntry,'\n');


  eclRun :=   '#WORKUNIT(\'name\',\''+ExternalVendor+' Score Stats '+FromDate+' to '+ToDate+'\');\n'
            + %'datasetName'%+' := Reports.DataGetterTransformer('+FromDate+','+ToDate+').'+ExternalVendor+';\n'
            + %'recStructName'% +' := RECORD ' + GetRecStructure() + 'END;\ninput := TABLE('+%'datasetName'%+','+%'recStructName'%+',LOCAL);\n'
            + ECLForStat;

  Cluster := NOTHOR(STD.System.Workunit.WorkunitList(WORKUNIT,WORKUNIT)[1].cluster) : INDEPENDENT;

  RETURN WHEN(WorkUnitManagement.fSubmitNewWorkunit(eclRun,Cluster, Cluster+'.thor'),Checks);


And to use:
Code: Select all
IMPORT Vendor1;
IMPORT * FROM Reports;


                 ,'OUTPUT(Profile_Everything(input, \'STAT\'),NAMED(\'Profile\'));');

Actually the function name 'RunScoreStats' is inappropriate, as the ECL using the flattened dataset, could be anything, nothing to do with 'Scores'. I requested a change of name but got overruled.
Posts: 342
Joined: Sat Oct 01, 2011 7:26 pm

Return to ECL

Who is online

Users browsing this forum: No registered users and 1 guest