Mon Jun 01, 2020 8:24 pm

## generate unique random numbers

Comments and questions related to the Enterprise Control Language
Hi everyone,

I am wondering is there a way to generate unique random numbers in ECL?
I tried to use RANDOM() but the result contains same random numbers.

Thanks,
Lily
lily

Posts: 13
Joined: Fri Nov 04, 2016 1:02 pm

Lily,

In my experience, it can be difficult to make RANDOM() generate the same numbers when you want it to, so I'm wondering how you used RANDOM()? In what context? Can you show me the code that produces the "same" non-unique numbers, please?

HTH,

Richard
rtaylor

Posts: 1535
Joined: Wed Oct 26, 2011 7:40 pm

Hi Taylor,

Thanks for help!

The code that generated the repeated random numbers is as shown below:

out := DATASET(5, TRANSFORM({INTEGER r}, SELF.r:= RANDOM()%10));
OUTPUT(out);

The output result is as shown below:

## r
1 8
2 0
3 8
4 7
5 1

Thanks,
Lily

rtaylor wrote:Lily,

In my experience, it can be difficult to make RANDOM() generate the same numbers when you want it to, so I'm wondering how you used RANDOM()? In what context? Can you show me the code that produces the "same" non-unique numbers, please?

HTH,

Richard
lily

Posts: 13
Joined: Fri Nov 04, 2016 1:02 pm

Lily,

The appearance of duplication is due to your use of the modulus operator limiting your result to only 10 possibilities (the remainders of dividing the actual RANDOM() number by 10).

This example demonstrates that the RANDOM() function itself returns a very different value for each use:
Code: Select all
`out := DATASET(5, TRANSFORM({INTEGER r,INTEGER r1,INTEGER r2},                             SELF.r  := RANDOM(),                            SELF.r1 := SELF.r % 10,                            SELF.r2 := SELF.r % 100));OUTPUT(out);`
I just ran this code and got this result:
Code: Select all
`1911583916   0   563647224352   8   21636695419   8   513689116099   3   61012783233   9   79`
You will note in the first record that the first RANDOM() result is 1911583916, the modulus 10 result is 0, and the modulus 100 result is 56. This doesn't make sense if the first random value (1911583916) is used for the two modulus calculations. But it does, because the RANDOM() function is actually called again each time an expression is calculated using it, producing this (correct) result.

So, as you see, RANDOM() actually DOES return unique values each time. If you really did want the modulus values to use the first RANDOM() in each record you would have to do it this way:
Code: Select all
`ds := DATASET(5, TRANSFORM({INTEGER r,INTEGER r1,INTEGER r2},                             SELF.r  := RANDOM(),                            SELF.r1 := 0,                            SELF.r2 := 0));                                          out := PROJECT(ds, TRANSFORM({INTEGER r,INTEGER r1,INTEGER r2},                             SELF.r  := LEFT.r,                            SELF.r1 := LEFT.r % 10,                            SELF.r2 := LEFT.r % 100));OUTPUT(out);`
to produce this result:
Code: Select all
`3040707218   8   181727978997   7   973116981210   0   102703444385   5   854157152711   1   11`
And now the modulus results definitely come from the generated RANDOM() values.
HTH,

Richard
rtaylor

Posts: 1535
Joined: Wed Oct 26, 2011 7:40 pm

Thank you very much Taylor! It really helps!

Lily

rtaylor wrote:Lily,

The appearance of duplication is due to your use of the modulus operator limiting your result to only 10 possibilities (the remainders of dividing the actual RANDOM() number by 10).

This example demonstrates that the RANDOM() function itself returns a very different value for each use:
Code: Select all
`out := DATASET(5, TRANSFORM({INTEGER r,INTEGER r1,INTEGER r2},                             SELF.r  := RANDOM(),                            SELF.r1 := SELF.r % 10,                            SELF.r2 := SELF.r % 100));OUTPUT(out);`
I just ran this code and got this result:
Code: Select all
`1911583916   0   563647224352   8   21636695419   8   513689116099   3   61012783233   9   79`
You will note in the first record that the first RANDOM() result is 1911583916, the modulus 10 result is 0, and the modulus 100 result is 56. This doesn't make sense if the first random value (1911583916) is used for the two modulus calculations. But it does, because the RANDOM() function is actually called again each time an expression is calculated using it, producing this (correct) result.

So, as you see, RANDOM() actually DOES return unique values each time. If you really did want the modulus values to use the first RANDOM() in each record you would have to do it this way:
Code: Select all
`ds := DATASET(5, TRANSFORM({INTEGER r,INTEGER r1,INTEGER r2},                             SELF.r  := RANDOM(),                            SELF.r1 := 0,                            SELF.r2 := 0));                                          out := PROJECT(ds, TRANSFORM({INTEGER r,INTEGER r1,INTEGER r2},                             SELF.r  := LEFT.r,                            SELF.r1 := LEFT.r % 10,                            SELF.r2 := LEFT.r % 100));OUTPUT(out);`
to produce this result:
Code: Select all
`3040707218   8   181727978997   7   973116981210   0   102703444385   5   854157152711   1   11`
And now the modulus results definitely come from the generated RANDOM() values.
HTH,

Richard
lily

Posts: 13
Joined: Fri Nov 04, 2016 1:02 pm

Hi,

I tried the proposed solution to generate unique random numbers. However, I am still getting duplicates. Is there a way to get unique random numbers with 100% certainty using ECL?

Best regards,
Vannel,
vzeufack

Posts: 28
Joined: Tue Sep 25, 2018 3:52 pm

Vannel,

Please post your code that produced duplicate values and the result showing the duplicates so I can try to recreate the problem.

HTH,

Richard
rtaylor

Posts: 1535
Joined: Wed Oct 26, 2011 7:40 pm

This is the code:
Code: Select all
`ds := DATASET(5, TRANSFORM({INTEGER r,INTEGER r1},                            SELF.r  := RANDOM(),                            SELF.r1 := 0));                                          out := PROJECT(ds, TRANSFORM({INTEGER r,INTEGER r1},                            SELF.r  := LEFT.r,                            SELF.r1 := LEFT.r % 10));OUTPUT(out);`

This is the output:
Code: Select all
`1   819659058    82   1733070309   93   535821437    74   3518949408   85   905533075    5`
vzeufack

Posts: 28
Joined: Tue Sep 25, 2018 3:52 pm

Vannel,

Here's an example of how to generate any number of guaranteed unique random numbers:
Code: Select all
`GenerateUniqueRandoms(UNSIGNED4 U) := FUNCTION  //generate 10% extra  ds := DATASET(U*1.1,                    TRANSFORM({UNSIGNED4 r},                           SELF.r  := RANDOM()));  //then dedup the result                                            out := DEDUP(SORT(ds,r),r);   //and limit to the desired number  RETURN out[1..U];END;COUNT(GenerateUniqueRandoms(1000000));  //I want a million uniquesCOUNT(GenerateUniqueRandoms(1000));     //now I want a thousand`

HTH,

Richard
rtaylor

Posts: 1535
Joined: Wed Oct 26, 2011 7:40 pm

Vannel,
This is the code:
CODE: SELECT ALL
ds := DATASET(5, TRANSFORM({INTEGER r,INTEGER r1},
SELF.r := RANDOM(),
SELF.r1 := 0));

out := PROJECT(ds, TRANSFORM({INTEGER r,INTEGER r1},
SELF.r := LEFT.r,
SELF.r1 := LEFT.r % 10));
OUTPUT(out);

This is the output:
CODE: SELECT ALL
1 819659058 8
2 1733070309 9
3 535821437 7
4 3518949408 8
5 905533075 5
Not to put too fine a point on it, but none of those random numbers are duplicated. The modulus 10 numbers do have duplicates, but those are hardly "random" -- they are simply the remainders after division by 10 of the actual random numbers.

HTH,

Richard
rtaylor