ECL Snippets

MLUsageExamples contains over 200 short example programs that illustrate how to use functions of the Machine Learning (ML) library.

You can find these example programs on github: MLUsageExamples.

MLUsageExamples contains small example ECL programs that illustrate how to use the functions and macros in the Machine Learning (ML) library. Each of these examples are stand-alone except for the need of ML. To execute any of them, you only need the ML library in your repository.

There are more than 200 example programs. Plus, the number of example programs will increase over time. When I add more, I post again, to let everyone know.

There is a README.txt file in MLUsageExamples that says everything I have in this post.

ORGANIZATION

MLUsageExamples is organized like the ML library’s folder structure. For example an example of how to use ML.Mat.Add will be found in the Mat folder because ML.Mat.Add.ecl is in the ML/Mat folder of the ML library. Furthermore, modules of the ML library will have examples in a folder named after the module. For example, examples of how to use the functions of the ML.Mat.Each.ecl module will be found in the folder MLUsageExamples/Mat/Each.

This project started with a conversation overheard in the lunch room. Simple code shows basics of data transformation using NORMALIZE and PROJECT.

An ECL programmer on our team was frustrated because he did not know of a reference file that returned a city name from a zip code entered. While studying our local repository I found a built-in function that returned a comma delimited list of cities for every 5-digit zip code. My project was to take this format and transform it into the single zipcode, city record that my friend was looking for.

For example- Change this input: 0,’61111′,’IL’,’3,LOVES PARK,MACHESNEY PARK,MACHESNEY PK’ To this output: 13,61111,IL,LOVES PARK 14,61111,IL,MACHESNEY PARK 15,61111,IL,MACHESNEY PK

Here is the complete code:

//Spec: Create a file of zipcodes and related cities IMPORT Std.Str; 

//Import String library used in transform 

//record definition of raw input data 

//since Rtn field is variable length, we can also use this same record 

//definition in our output file 




Rec := RECORD UNSIGNED3 CSZID; 

STRING5 Zip; 

STRING2 State; 

STRING Rtn{MAXLENGTH(1024)}; 

END; 




//this data set simulates the data returned from the internal function(s) 

FuncDS := DATASET([

	{0,'00000','',0}, 

	{0,'14513','NY','2,NEWARK,EAST PALMYRA'}, 

	{0,'29710','SC','3,CLOVER,LAKE WYLIE,RIVER HILLS'}, 

	{0,'33334','FL','4,FORT LAUDERDALE,FT LAUDERDALE,OAKLAND PARK,WILTON MANORS'}, 

	{0,'33424','FL','1,BOYNTON BEACH'}, {0,'55555','MN','1,YOUNG AMERICA'}, 

	{0,'60933','IL','1,ELLIOTT'}, 

	{0,'61111','IL','3,LOVES PARK,MACHESNEY PARK,MACHESNEY PK'}, 

	{0,'66604','KS','1,TOPEKA'}, 

	{0,'68836','NE','2,ELBA,COTESFIELD'}, 

	{0,'74652','OK','2,SHIDLER,FORAKER'}, 

	{0,'81252','CO','2,WESTCLIFFE,SILVER CLIFF'}, 

	{0,'99999','',0}

	], Rec); 




//Moves data from FuncDS DATASET to ZipsIn recordset 

Rec Xf1(Rec L,INTEGER C) := TRANSFORM SELF.CSZID := 0; 

SELF.Zip := IF(INTFORMAT(c,5,1)= L.ZIP, L.ZIP, ''); 

SELF := L; 

END; 




//removes blank records - cleans input data //this function was originally used because we had an internal library 

//that returned zip code information for all zip codes. That is why we 

//used 100000 as the number of iterations. The use of INTFORMAT changed 

//counter values to the zero padded zip format 

ZipsIN := NORMALIZE(FuncDS,100000,XF1(LEFT,COUNTER))(Rtn <> '0', Zip <> ''); 




//Here is the original code, and why we used NORMALIZE: 


/* 

	Rec Xf1(Rec L,INTEGER C) := TRANSFORM SELF.CSZID := 0; 

	SELF.Zip := INTFORMAT(c,5,1); 

	SELF.State := ZipLib.ZipToState2(SELF.Zip); 

	SELF.Rtn := ZipLib.ZipToCities(SELF.Zip); 

	END; 

	ZipsIN := NORMALIZE(BlankDS,100000,XF1(LEFT,COUNTER))(Rtn <> '0'); 

*/ 




//This was the input data that we needed to process 

OUTPUT(ZipsIN,NAMED('ZipsIn')); 

// This TRANSFORM has three local ECL definitions (InstanceComma, StartPos 

// and EndPos) 

// used to find the commas and parse the city information between them. 

Rec XF2(Rec L,INTEGER C) := TRANSFORM InstanceComma := Str.Find(L.Rtn,',',C+1); 

EndPos := IF(InstanceComma=0,LENGTH(TRIM(L.Rtn)),InstanceComma-1);

StartPos := Str.Find(L.Rtn,',',C) + 1; 

SELF.Zip := L.Zip; 

SELF.Rtn := L.Rtn[StartPos .. EndPos]; 

SELF := L; 

END; 




//DISTRIBUTE allows for more efficient processing 

//String index of 1 for RTN determines how many times the transform 

//gets called 

ZipsOut := NORMALIZE(

	DISTRIBUTE(ZipsIn,HASH32(Zip)),

	(INTEGER)LEFT.Rtn[1],XF2(LEFT,COUNTER)

); 

Rec XF3(Rec L,INTEGER C) := TRANSFORM SELF.CSZID := C; 

SELF := L; 

END; 




//Finally, after we NORMALIZE each record we assign a unique ID to each 

//one using a simple PROJECT 

UIDzips := PROJECT(ZipsOut,XF3(LEFT,COUNTER)); 

OUTPUT(UIDZips,,'~TEST::OUT::CityStateZip',OVERWRITE);

In our training class, we like to demonstrate examples of the ECL language in action without the need to reference an external file. Using an inline DATASET, here is a simple example that shows the many variations of JOIN.

Joining two datasets is a fundamental cornerstone of any database system. This code example shows the ECL JOIN statement and its many variations of JOIN types. This example uses an inline dataset, so all you need to do is download it, modify the TXT extension to ECL, copy it to any of your ECL folders, and run. Enjoy!

Associated File

JOIN_Ex.txt

It’s a FUNCTION, no it’s a MACRO. Wait, it’s both! Hot off the presses, here is a new and handy structure to make your ECL programming tasks more flexible and powerful.

The FUNCTIONMACRO structure is a code generation tool, like the MACRO structure, coupled with code encapsulation benefits of the FUNCTION structure. This means that #UNIQUENAME is not necessary to prevent definition name clashes — the definitions in the code are local within the FUNCTIONMACRO. One additional advantage the FUNCTIONMACRO has over the MACRO structure is that it may be called in an expression context, just like a FUNCTION would be.

Associated File

FUNCTIONMACRO_Example.ecl

I am attempting to pass in an integer, and get back the binary representation of that integer in a string format. Example: I pass in integer 14 and the result would be ‘1110’ I pass in integer 322 and the result would be ‘101000010’ How can I do this in ECL?

This code snippet shows a handy function for accomplishing the above. Note the use of POWER to determine the binary weighting of a particular bit position. This example was written by Richard Taylor, our Chief ECL Trainer who has been working with ECL since its inception over 13 years ago. Here it is:

STRING32 BinStr(UNSIGNED4 val) := FUNCTION

  BOOLEAN IsBitOn(UNSIGNED4 Bitmap,UNSIGNED1 Bit2Chk) := FUNCTION

        UNSIGNED4 Mask := (UNSIGNED4)POWER(2,Bit2Chk-1);

        RETURN BitMap & Mask = Mask;

   END;

   STRING1 Bin2Str(UNSIGNED4 Bin, UNSIGNED1 Bit) := IF(IsBitOn(Bin,Bit),'1','0');

   RETURN Bin2Str(val,32) + Bin2Str(val,31) + Bin2Str(val,30) + Bin2Str(val,29) +  Bin2Str(val,28) + Bin2Str(val,27) + Bin2Str(val,26) + Bin2Str(val,25) +  Bin2Str(val,24) + Bin2Str(val,23) + Bin2Str(val,22) + Bin2Str(val,21) +  Bin2Str(val,20) + Bin2Str(val,19) + Bin2Str(val,18) + Bin2Str(val,17) +  Bin2Str(val,16) + Bin2Str(val,15) + Bin2Str(val,14) + Bin2Str(val,13) +  Bin2Str(val,12) + Bin2Str(val,11) + Bin2Str(val,10) + Bin2Str(val,9) +  Bin2Str(val,8) + Bin2Str(val,7) + Bin2Str(val,6) + Bin2Str(val,5) +  Bin2Str(val,4) + Bin2Str(val,3) + Bin2Str(val,2) + Bin2Str(val,1);

END;

OUTPUT(BinStr(532860));

//Example of usage

Comments

Faster Way

Permalink Submitted by richardkchapman on Tue, 08/20/2013 – 12:21pm
While the above will work, it will be painfully slow I fear!

This is an example of where ECL is not really the best language to use – but it’s simple enough to drop into C++:

For example:

string32 str2bin(unsigned4 a) := BEGINC++

    unsigned mask = 0x80000000;

    for (int i = 0; i < 32; i++)

    {

        if (a & mask)

          __result[i] = '1';

      else

          __result[i] = '0';

          mask >>= 1;

    }

ENDC++;

str2bin(5);