Mon Aug 20, 2018 8:15 pm
Login Register Lost Password? Contact Us


Appending datasets with other datasets while adding a field.

Comments and questions related to the Enterprise Control Language

Thu Oct 19, 2017 8:40 pm Change Time Zone

So I had a situation today in which I had 16 different datasets. They all have the same output record structure so I figured it would look nicer if they were output in a single table instead of separate ones.

The issue was I needed to differentiate them by the macros they came from, so I figured I'd add a new column for this. I got it to work but it's a very brute force method and I feel like there's probably some way to do it using code generation functions in ECL.

Here's how I have it now:
Code: Select all
newLayout := {
   STRING ResultType,
   doxie.layout_best
};

newLayout AddResultType(doxie.layout_best L, STRING ResultType) := TRANSFORM
   SELF.ResultType := ResultType;
   SELF := L;
END;

//These macros assign a dataset in doxie.layout_best structure to the first param.
doxie.MAC_fetch_watchdog(results_glb,did_stream,LexID,Watchdog.Key_Watchdog_glb);
doxie.MAC_fetch_watchdog(results_glb_nonutil,did_stream,LexID,Watchdog.Key_Watchdog_glb_nonutil);
doxie.MAC_fetch_watchdog(results_glb_nonutil_nonblank,did_stream,LexID,Watchdog.Key_Watchdog_glb_nonutil_nonblank);
//etc...

//Project the results to our new structure and add the correct resulttype string.
//And append them all to each other using the + operator.
   FormattedResults :=
   PROJECT(results_glb, AddResultType(LEFT, 'glb')) +
   PROJECT(results_glb_nonutil, AddResultType(LEFT, 'glb_nonutil')) +
   PROJECT(results_glb_nonutil_nonblank, AddResultType(LEFT, 'glb_nonutil_nonblank')) +
//etc...


So that works and it wasn't too hard to generate the code using regular expressions in another tool but I'm wondering if it would be better to contain the code generation inside ECL and am not sure where to start.

My gut says it's somewhere with a for loop iterating over a set of variable names and substituting them where appropriate but I could use a little nudge in the right direction.
BGehalo
 
Posts: 12
Joined: Thu Sep 28, 2017 8:06 pm

Fri Oct 20, 2017 1:58 pm Change Time Zone

BGehalo,
I'm wondering if it would be better to contain the code generation inside ECL and am not sure where to start.
ECL's Template Language would be my suggestion.

So a good place to start would be our online training course: Applied ECL - ECL Code Generation Tools (available here: https://hpccsystems.com/training).

This course covers all the ECL code generation tools: MACROs, FUNCTIONMACROs, and the Template Language.

HTH,

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1370
Joined: Wed Oct 26, 2011 7:40 pm

Fri Oct 20, 2017 3:38 pm Change Time Zone

Thanks, I went and revisited some of the videos but I'm still having a bit of trouble getting it to work.

I've been playing around on a local vm trying to emulate what I need, I think something strange is happening where it's using double quotes when it's calling a macro from within a function macro.

Code: Select all
SHARED TestMac(testStr, idx) := MACRO
   testStr := idx;
ENDMACRO;

CodeGeneration(Vars) := FUNCTIONMACRO
   len := COUNT(vars);
   #DECLARE(idx);
   #DECLARE(code);
   #SET(idx, 1);
   #SET(code, '');
   #LOOP
      #IF (%idx% > len) #BREAK
      #ELSE
         #APPEND(code,'TestMac('+vars[%idx%]+',\''+%idx%+'\');');
         #SET(idx, %idx%+1);
       #END
   #END
   Return %code%;
ENDMACRO;

CodeGeneration(['a','b', 'c']);
a; b; c;


This gives me a syntax error which references the test macro above.
Error: syntax error near " (1, 7), 3002,

It must be putting double quotes in there or something strange.

If I wrap the return %code% in a #TEXT declaration I can see my results as string:
Code: Select all
TestMac( a, '1') ; TestMac( b, '2') ; TestMac( c, '3') ;


Strange that it's adding spaces in there. If I run the above generated code it works, I can then output a;b;c; and it's as expected.
BGehalo
 
Posts: 12
Joined: Thu Sep 28, 2017 8:06 pm

Fri Oct 20, 2017 6:27 pm Change Time Zone

Okay I got it to work by changing it to a macro. If I understand how macros work it executes the string saved to %code% as ECL code when I call it at the bottom of the macro.

Code: Select all
SHARED TestMac(outVar, idx) := MACRO
   outVar := idx;
ENDMACRO;

CodeGeneration(Vars) := MACRO
   len := COUNT(vars);
   #DECLARE(idx);
   #DECLARE(code);
   #SET(idx, 1);
   #SET(code, '');
   #LOOP
      #IF (%idx% > len) #BREAK
      #ELSE
         #APPEND(code,'TestMac('+vars[%idx%]+',\''+%idx%+'\');');
         #SET(idx, %idx%+1);
       #END
   #END
   %code%
ENDMACRO;

CodeGeneration(['a','b','c']);
a; b; c;
BGehalo
 
Posts: 12
Joined: Thu Sep 28, 2017 8:06 pm

Fri Oct 20, 2017 6:31 pm Change Time Zone

BGehalo,

Your first issue is, since your TestMac MACRO wants a string (and you're generating in just the characters), you need to get single quotes into your generated MACRO call surrounding the set value to pass, like this:
Code: Select all
#APPEND(code,'TestMac(\'' + vars[%idx%] + '\');\n');
And I always prefer generating code that could be readable, so I also added the \n to the end.

Next, you only need to do this to see the generated code:
Code: Select all
RETURN %'code'%;
Note the addition of the single quotes inside the percent signs. That changes it from generating the code itself, to a STRING showing the generated code.

And finally, here's all my test code. I generated the STRING of code then ran them (commented out) to make sure the TestMac MACRO works as expected. It does.
Code: Select all
TestMac(testStr) := MACRO
  OUTPUT(testStr);
ENDMACRO;

CodeGeneration(Vars) := FUNCTIONMACRO
   len := COUNT(vars);
   #DECLARE(idx);
   #DECLARE(code);
   #SET(idx, 1);
   #SET(code, '');
   #LOOP
      #IF (%idx% > len) #BREAK
      #ELSE
         #APPEND(code,'TestMac(\'' + vars[%idx%] + '\');\n');
         #SET(idx, %idx%+1);
       #END
   #END
   // RETURN %'code'%;
   RETURN %code%;
ENDMACRO;

CodeGeneration(['a','b','c']);
// TestMac('a');
// TestMac('b');
// TestMac('c');
But using the FUNCTIONMACRO to generate the MACRO calls fails, making this a prime candidate for a JIRA ticket. Please submit that so the developers can have a look at this anomaly.

HTH,

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1370
Joined: Wed Oct 26, 2011 7:40 pm

Fri Oct 20, 2017 7:11 pm Change Time Zone

BGehalo
 
Posts: 12
Joined: Thu Sep 28, 2017 8:06 pm


Return to ECL

Who is online

Users browsing this forum: No registered users and 1 guest