Sun May 19, 2019 9:27 pm
Login Register Lost Password? Contact Us


Adjusting # of Iterations for Template Language #LOOP

Questions around writing code and queries

Fri May 10, 2019 7:30 pm Change Time Zone

I am trying to create basic bootstrap code in ECL, that can be configurable on run-time for a certain # of iterations (whatever is required to hit our target # of records).

Originally I tried using a count of the created bootstrapped dataset to generate a cutoff, but the code required a Constant Expression. I changed to a number of iterations method, which works...so long as I pass a hard-coded integer.

If I do a calculation to create the # of iterations (as in the code below), it gives me the same 'Constant Expression Expected' error. I have tried utilizing multiple castings, #STORED, #DECLARE, both inside and out of the FunctionMacro, but ECL seems deadset on not letting me have this run-to-run variable input.

Is there any way to 'force' the Template Language to accept my simple math, or is there another way to adjust the # of iterations?

Thanks,

Matt Rumsey

#LOOP Code in FunctionMacro
Code: Select all
IMPORT   zz_mrumsey;

EXPORT FMAC_BootStrap_Basic(d_Input, FinCnt) := FUNCTIONMACRO
   LOADXML('<xml/>');

   LOCAL   ds_0         :=   DATASET([], RECORDOF(d_Input));
   LOCAL   cnt_Input   :=   COUNT(d_Input);
   LOCAL   NumIters   :=   ROUNDUP(FinCnt / (cnt_Input * .25)); //Calculates ~# of Samples Needed.  Samples Taken At 25% of Input Dataset
      
   #DECLARE(ds_xxx);
   #DECLARE(ds_xxxOld);
   #DECLARE(i);
   #DECLARE(j);
   
   #SET(i, 1);
   #SET(j, 0);   

   #LOOP
      #SET(ds_xxx, 'ds_')
      #SET(ds_xxxOld, 'ds_')
      
         #IF(%i% > NumIters)
            #APPEND(ds_xxxOld, %'j'%);
            RETURN   %ds_xxxOld%;
            #BREAK
         #ELSE         
            #APPEND   (ds_xxx, %'i'%)
            #APPEND (ds_xxxOld, %'j'%);

            #UNIQUENAME(ds_Temp)
            %ds_Temp%   :=   zz_mrumsey.FMAC_RandomSample(d_Input, .25);
            %ds_xxx% := %ds_xxxOld% + %ds_Temp%;         

            #SET (j, %i%);
            #SET (i, %i% + 1);
         #END
   #END
ENDMACRO;
mrumsey
 
Posts: 25
Joined: Wed Apr 09, 2014 8:21 pm

Mon May 13, 2019 4:06 pm Change Time Zone

UPDATE:

I realize that code I had looked at previously had math (for # of iterations) based on constants supplied prior to compile-time, thereby satisfying the 'Constant Expression' requirement as it could be computed prior to code generation.

I am hoping to get around this by using a BWR file but I need a way to generate or discover a record structure.

Does anyone know of a way to generate a full record-structure that is unknown at compile time? I know there is the 'LOOKUP' option for a DATASET call, but it drops fields that are unknown, which makes the code less flexible and less viable for projects other than my current one.

I feel this should be possible, as I can get a record structure from HPCC (via ECL Watch) as well as the system telling me when my layout is incorrect.

Thanks again,

Matt Rumsey
mrumsey
 
Posts: 25
Joined: Wed Apr 09, 2014 8:21 pm

Tue May 14, 2019 8:28 pm Change Time Zone

Matt, can you share your MACRO solution with the community please?
Or just an abstract if the code is proproetary.

Thanks!

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 983
Joined: Wed Jun 29, 2011 7:13 pm

Wed May 15, 2019 2:55 pm Change Time Zone

hi Matthew,
for particular to this requirement, where you would want to call your randomsample for the noofiterations, I think template langauage is not required or it could be little more burden to achieve what we need. This requirement can be easily achieved with Dataset form as below and rollup all the records together to get the summed results out.

I have shared the code below, for your reference. we believe this problem can be achived in many ways, this one is one of the simple solutions I could think of.

please let us know alternative solutions which is better in performance if any.

processedDs := DATASET(
NbrIters,
TRANSFORM({typeof(ds_useme) x},
self.x := zz_mrumsey.FMAC_RandomSample(ds_useme, .25);
)
);

rolledupDs := ROLLUP(processedDs,
TRUE,
TRANSFORM(RECORDOF(processedDs),
SELF.x := IF(EXISTS(left.x), left.x + right.x, RIGHT.X);
)
);

OUTPUT(rolledupDs[1].x);

Regards,
Raja
raja
 
Posts: 4
Joined: Thu Feb 18, 2016 7:49 pm


Return to Programming

Who is online

Users browsing this forum: Bing [Bot] and 1 guest

cron