Error messages meaning ?

[rtaylor]

JM,

Then, is it possible to do a real loop in ecl (not just simulated with iterate) ?

Doing a "real loop" is an inherently procedural task, and since ECL is a declarative, non-procedural language, the answer to that would be no.

PROJECT, ITERATE, and all the rest of the functions that use TRANSFORMs are designed to handle all the "looping" type of situations. The LOOP and GRAPH functions aren't even real loops either, but more a form of recursion.

HTH,

Richard

[ideal]

Richard,

Let me rephrase my question to clarify it : why is there an entry about LOOP function in documentation when it is not available in the software, as stated in previous post ?

Thanks,
JM.

[ideal]

If LOOP is not available, how is it possible to replace it by a potentially infinite loop with break test ?

[rtaylor]

JM,

why is there an entry about LOOP function in documentation when it is not available in the software

LOOP is a valid function in the ECL Language that performs recursive tasks. It was originally designed to operate only on hThor (a testbed platform) and Roxie clusters (a production environment) and not on Thor clusters (primarily dedicated to ETL work). It was documented that way when it was first introduced several years ago, and it still works correctly on those platforms.

When we Open Sourced HPCC, both LOOP and GRAPH were meant to be updated to also operate on Thor. You are already aware of the GRAPH bug in 3.6 that is fixed in 3.8. If your LOOP code does not work on Thor, then that should also be reported as a bug. Your workaround is to run that job on hThor.

If LOOP is not available, how is it possible to replace it by a potentially infinite loop with break test ?

Rather than answer this question, let me ask this one:

What are you trying to accomplish?

I ask this because it is quite possible that there is another more "ECL-ish" way to accomplish what you want to do. ECL is not like other languages, and frequently requires a different mindset to approaching problems than that used in procedural languages -- take a look at the "Attribute Creation" Programmer's Guide article for a discussion of the "bottom-up" approach to problem solving that works best with ECL.

HTH,

Richard

[ideal]

If your LOOP code does not work on Thor, then that should also be reported as a bug

As I said before, error message tells us it is not a bug :

Code: Select all

Error:    System error: 0: Graph[1], Thor currently, does not support a dataset loop condition, activity id: 7 (0, 0), 0,

That is the reason why I am asking you :

why is there an entry about LOOP function in documentation when it is not available in the software

.
I choose hpcc by reading the documentation, finding good things inside, some are essential to me but it seems it does not work as written. In that sense, it is a bug. My partial workaround until now is an iterate function but it is not perfectly accurate.

Roughly speaking, What I need to do is starting with a set of records and add or remove records by applying transformations or tests implying all records together. This process must run until there is no more new records to add. Then, this is typically

a potentially infinite loop with break test

.

When you say it is available on hthor, it does not suit my requirements because what I need the most is scalability.

If you can solve this, it would be nice.

JM

[ideal]

If your LOOP code does not work on Thor, [..] Your workaround is to run that job on hThor

In a distributed parallel computing environment, I don't think hthor can be considered as a "workaround".

JM.

[ghalliday]

Unfortunately the current version of Thor doesn't support that particular variety of LOOP. It required some changes in the code generation - which are included in 3.8 - but didn't make it in time for Thor to be updated.

The plan is to include it in the next release - either 3.10 or possibly (depending on the scope of the change) 3.8.2.

[ideal]

OK, it is not convenient to me, but at least, it is clear. I am looking for an ITERATE-like (GRAPH I think) function instead. It allows transforming input dataset "recursively" but it force to determine in advance the number of iterations.

This close the subject.

Thanks,
JM.