Machine Learning (ML) Usage Examples

[tlhumphrey2]

I have placed Machine Learning (ML) example programs on github: https://github.com/hpcc-systems/ecl-samples. They are in the folder MLUsageExamples.

These are small example ECL programs that illustrate how to use the functions and macros in the Machine Learning (ML) library. Each of these examples are stand-alone except for the need of ML. To execute any of them, you only need the ML library in your repository.

There are more than 200 example programs. Furthermore, the number of example programs will increase over time. I will post again when I add additional programs.

In MLUsageExamples, there is a README.txt file that tells a little about the organization of these example programs.

ORGANIZATION

MLUsageExamples is organized like the ML library's folder structure. For example an example of how to use ML.Mat.Add will be found in the Mat folder because ML.Mat.Add.ecl is in the ML/Mat folder of the ML library. Furthermore, modules of the ML library will have examples in a folder named after the module. For example, examples of how to use the functions of the ML.Mat.Each.ecl module will be found in the folder MLUsageExamples/Mat/Each.

[tlhumphrey2]

If anyone has questions about the ML usage examples I've placed on github, please post here. I now have a feed to the Machine Learning forum and should get instant notification when someone posts.

[tlhumphrey2]

24 new machine learning library usage examples have been added to those already in https://github.com/hpcc-systems/ecl-sam ... geExamples. Plus, all of those examples already there that had syntax errors have been corrected.

[tdelbecque]

Hello,

I am trying to run your example:

Use_ML.ML.Mat.Has.Stats.ecl

but I keep getting this error:

Error: Unknown identifier "Has" (22, 11 - ml.mat.mul)
...(I don't copy the whole bunch of lines)

which I find weird. I have installed the latest ML package, and I am able to run ML algorithms such as Kmeans for example, so I don't think this is an installation problem.

I have spent some amount of time to understand what was going on, but I did not find the problem. Did you ever run into this problem, or do you have some kind of idea about the problem ?

Thanks in advance,

Bests, Thierry.

[tlhumphrey2]

tdelbecque,

Your error message, "Error: Unknown identifier "Has" (22, 11 - ml.mat.mul)", indicates that the error occurred on line 22 of ml.mat.mul.

I looked at the code of "Use_ML.ML.Mat.Has.Stats.ecl" and there isn't a reference to ml.mat.mul in it. So, you must have other code than just that of "Use_ML.ML.Mat.Has.Stats.ecl". But, with that said, try placing the following line at the top of the code and see if the error goes away:

IMPORT * FROM ML.Mat;

tlhumphrey2

[tdelbecque]

Hello,

There is really nothing else than the content of Use_ML.ML.Mat.Has.Stats.ecl, that I have cut and pasted in an empty builder. I know that there is no reference to ml.mat.mul there, this is why I find this rather strange. Adding the IMPORT do not solve it.

Thanks.

[tlhumphrey2]

tdelbecque,

You getting this error doesn't make sense to me, either. But, would you provide all the error lines you got (in you 1st post you mentioned that you didn't include all the lines.).

[tdelbecque]

Sure, following is the whole content.

I have tried to find the problem by disabling some part of some modules in ML, and it is as if the compiler get confused beetwen each.mul and ml.mul ...

As you do not have this problem, I guess this is due to my own configuration, though merely put the latest ML package in the repository. Bizarre bizarre ...

Thanks.


eclserver ml.mat.mul (22,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (26,18) : 3002: syntax error near "T" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (28,1) : 3002: syntax error near "END" : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (39,5) : 3002: syntax error near ":=" : expected ';'
eclserver ml.mat.mul (48,4) : 3002: syntax error near ":=" : expected ';'
eclserver ml.mat.mul (52,17) : 3002: syntax error near "T" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (61,19) : 3002: syntax error near "mT" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (63,1) : 3002: syntax error near "END" : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (66,13) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (67,13) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (68,34) : 3002: syntax error near "." : expected ')'
eclserver ml.mat.mul (70,34) : 2073: Expected boolean expression
eclserver ml.mat.mul (72,16) : 2073: Expected boolean expression
eclserver ml.mat.mul (72,56) : 2167: Unknown identifier "Mul_Default"
eclserver ml.mat.mul (74,5) : 3002: syntax error : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.add (5,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.add (6,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.add (7,32) : 3002: syntax error near "." : expected ')'
eclserver ml.mat.add (15,32) : 2073: Expected boolean expression
eclserver ml.mat.add (17,14) : 2073: Expected boolean expression
eclserver ml.mat.add (5,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.add (6,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.add (7,32) : 3002: syntax error near "." : expected ')'
eclserver ml.mat.add (15,32) : 2073: Expected boolean expression
eclserver ml.mat.add (17,14) : 2073: Expected boolean expression
eclserver ml.mat.mul (22,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (26,18) : 3002: syntax error near "T" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (28,1) : 3002: syntax error near "END" : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (39,5) : 3002: syntax error near ":=" : expected ';'
eclserver ml.mat.mul (48,4) : 3002: syntax error near ":=" : expected ';'
eclserver ml.mat.mul (52,17) : 3002: syntax error near "T" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (61,19) : 3002: syntax error near "mT" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (63,1) : 3002: syntax error near "END" : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (66,13) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (67,13) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (68,34) : 3002: syntax error near "." : expected ')'
eclserver ml.mat.mul (70,34) : 2073: Expected boolean expression
eclserver ml.mat.mul (72,16) : 2073: Expected boolean expression
eclserver ml.mat.mul (72,56) : 2167: Unknown identifier "Mul_Default"
eclserver ml.mat.mul (74,5) : 3002: syntax error : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.add (5,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.add (6,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.add (7,32) : 3002: syntax error near "." : expected ')'
eclserver ml.mat.add (15,32) : 2073: Expected boolean expression
eclserver ml.mat.add (17,14) : 2073: Expected boolean expression
eclserver ml.mat.add (5,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.add (6,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.add (7,32) : 3002: syntax error near "." : expected ')'
eclserver ml.mat.add (15,32) : 2073: Expected boolean expression
eclserver ml.mat.add (17,14) : 2073: Expected boolean expression
eclserver ml.mat.has (19,25) : 2167: Unknown identifier "Each"
eclserver ml.mat.mul (22,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (26,18) : 3002: syntax error near "T" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (28,1) : 3002: syntax error near "END" : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (39,5) : 3002: syntax error near ":=" : expected ';'
eclserver ml.mat.mul (48,4) : 3002: syntax error near ":=" : expected ';'
eclserver ml.mat.mul (52,17) : 3002: syntax error near "T" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (61,19) : 3002: syntax error near "mT" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (63,1) : 3002: syntax error near "END" : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (66,13) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (67,13) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (68,34) : 3002: syntax error near "." : expected ')'
eclserver ml.mat.mul (70,34) : 2073: Expected boolean expression
eclserver ml.mat.mul (72,16) : 2073: Expected boolean expression
eclserver ml.mat.mul (72,56) : 2167: Unknown identifier "Mul_Default"
eclserver ml.mat.mul (74,5) : 3002: syntax error : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (22,11) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (26,18) : 3002: syntax error near "T" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (28,1) : 3002: syntax error near "END" : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (39,5) : 3002: syntax error near ":=" : expected ';'
eclserver ml.mat.mul (48,4) : 3002: syntax error near ":=" : expected ';'
eclserver ml.mat.mul (52,17) : 3002: syntax error near "T" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (61,19) : 3002: syntax error near "mT" : expected DICTIONARY, RANGE, ROWSET, SELF, SUCCESS, datarow, dataset, dictionary, module-name, identifier, identifier, function-name, identifier, macro-name, '^', '(', '['
eclserver ml.mat.mul (63,1) : 3002: syntax error near "END" : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver ml.mat.mul (66,13) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (67,13) : 2167: Unknown identifier "Has"
eclserver ml.mat.mul (68,34) : 3002: syntax error near "." : expected ')'
eclserver ml.mat.mul (70,34) : 2073: Expected boolean expression
eclserver ml.mat.mul (72,16) : 2073: Expected boolean expression
eclserver ml.mat.mul (72,56) : 2167: Unknown identifier "Mul_Default"
eclserver ml.mat.mul (74,5) : 3002: syntax error : expected APPLY, _ARRAY_, BIG_ENDIAN, BUILD, DICTIONARY, DISTRIBUTION, EMBEDDED, expression, EXPORT, FEATURE, IMPORT, KEYDIFF, KEYPATCH, _LINKCOUNTED_, LITTLE_ENDIAN, LOADXML, NOTIFY, OUTPUT, PACKED, PARALLEL, PATTERN, RANGE, RECORD, RETURN, ROWSET, RULE, SET, SHARED, <typename>, STREAMED, SUCCESS, TOKEN, TRANSFORM, TYPEOF, UNSIGNED, UPDATE, WAIT, WILD, <, datarow, identifier, action, constant, #CONSTANT, #OPTION, #WORKUNIT, #STORED, #LINK, #ONWARNING, '^', ';', '$'
eclserver (19,23) : 2171: Object 'mat' does not have a member named 'has'
eclserver (19,23) : 2167: Unknown identifier "Has"

[tlhumphrey2]

You shouldn't be getting all those errors unless something when wrong with you installation of the new ML package.

You mentioned that you tried kmeans and had no trouble which is not in ML.Mat. So, I'm thinking something happened to ML.Mat during the installation.

Tell me about your HPCC environment. Is this a virtual hpcc cluster that I are running on? Or, exactly what is your HPCC environment?

[tlhumphrey2]

tdelbecque,

Please post and let everyone know how you fixed the problem. I just ran ML_use.Mat.Has.Stats.ecl on my hpcc and it ran fine.