Fri Dec 03, 2021 12:45 am
Login Register Lost Password? Contact Us


Converting numpy array to ecl dataset (embed python)

Comments and questions related to the Enterprise Control Language

Fri Sep 22, 2017 8:15 pm Change Time Zone

I have code that uses embed python to convert a numeric ecl dataset to a python numpy array. Then it does computation with the numpy array and returns a STRING that represents the result of the computation. Below is that code.

But, I want the embed python to return an ecl dataset of the result of the computation. Any ideas?

Code: Select all
import python;

rec := RECORD
  set of REAL row;
END;

STRING matrix(DATASET(rec) s) := embed(Python)
    import numpy as np
    import re

    #return s

    def ECLDataset2NPArray(s):
      zarray=np.empty([3,2],dtype=float)
      p=''
      i=0
      for row in s:
        z='%s' % (' '.join(map(str,row)))
        p+=z;
        z = re.sub('[\[\]]', '', z)
        zlist=z.split(',')
        zarray[i]=np.asarray(zlist)
        i+=1
      return zarray
   
    zarray=ECLDataset2NPArray(s)
    zarray+=1000;
    str_zarray=np.array_str(zarray)
    str_zarray=np.array_str(zarray)

    return str_zarray

endembed;

s := DATASET([{[1.1,2.2]},{[3.3,4.4]},{[5.5,6.6]}],rec);

matrix(s);
tlhumphrey2
 
Posts: 260
Joined: Mon May 07, 2012 6:23 pm

Fri Sep 22, 2017 8:21 pm Change Time Zone

Removed a couple of useless lines of code from that originally posted. Here is new code.

Code: Select all
import python;

rec := RECORD
  set of REAL row;
END;

STRING matrix(DATASET(rec) s) := embed(Python)
    import numpy as np
    import re

    def ECLDataset2NPArray(s):
      zarray=np.empty([3,2],dtype=float)
      i=0
      for row in s:
        z='%s' % (' '.join(map(str,row)))
        z = re.sub('[\[\]]', '', z)
        zlist=z.split(',')
        zarray[i]=np.asarray(zlist)
        i+=1
      return zarray

    zarray=ECLDataset2NPArray(s)
    zarray+=1000;
    str_zarray=np.array_str(zarray)

    return str_zarray

endembed;

s := DATASET([{[1.1,2.2]},{[3.3,4.4]},{[5.5,6.6]}],rec);

matrix(s);

tlhumphrey2
 
Posts: 260
Joined: Mon May 07, 2012 6:23 pm

Fri Sep 22, 2017 8:53 pm Change Time Zone

Here's an example from the regression suite:

Code: Select all
IMPORT Python;

childrec := RECORD
   string name => unsigned value;
END;

namesRecord := RECORD
    STRING name1;
    STRING10 name2;
    LINKCOUNTED DATASET(childrec) childnames;
    LINKCOUNTED DICTIONARY(childrec) childdict{linkcounted};
    childrec r;
    unsigned1 val1;
    integer1   val2;
    UTF8 u1;
    UNICODE u2;
    UNICODE8 u3;
    BIG_ENDIAN unsigned4 val3;
    DATA d;
    BOOLEAN b;
    SET OF STRING ss1;
END;

dataset(namesRecord) blockedNames(string prefix) := EMBED(Python)
  return ["Gavin","John","Bart"]
ENDEMBED;

_linkcounted_ dataset(namesRecord) linkedNames(string prefix) := EMBED(Python)
  return ["Gavin","John","Bart"]
ENDEMBED;

dataset(namesRecord) streamedNames(data d, utf8 u) := EMBED(Python)
  return [  \
     ("Gavin", "Halliday", [("a", 1),("b", 2),("c", 3)], [("aa", 11)], ("aaa", 111), 250, -1,  U'là',  U'là',  U'là', 1, d, False, ["1","2"]), \
     ("John", "Smith", [], [], ("c", 3), 250, -1,  U'là',  U'là',  u, 2, d, True, set(["3"])) \
     ]
ENDEMBED;

// Test use of Python generator object for lazy evaluation...

dataset(childrec) testGenerator(unsigned lim) := EMBED(Python:time)
  num = 0
  while num < lim:
    yield ("Generate:", num)
    num += 1
ENDEMBED;

output(streamedNames(d'AA', u'là'));
output (testGenerator(10));

// Test what happens when two threads pull from a generator
c := testGenerator(1000);
count(c(value < 500));
count(c(value > 500));

// Test Python code returning named tuples
childrec tnamed(string s) := EMBED(Python)
  import collections;
  childrec = collections.namedtuple("childrec", "value,name")
  return childrec(1,s)
ENDEMBED;

output(tnamed('Yo').name);

// Test passing records into Python

dataset(namesRecord) streamInOut(dataset(namesRecord) recs) := EMBED(Python)
  for rec in recs:
    if rec.name1 == 'Gavin':
       yield rec
ENDEMBED;

output(streamInOut(streamedNames(d'AA', u'là')));
richardkchapman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 110
Joined: Fri Jun 17, 2011 8:59 am

Sat Sep 23, 2017 9:55 am Change Time Zone

richardkchapman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 110
Joined: Fri Jun 17, 2011 8:59 am


Return to ECL

Who is online

Users browsing this forum: No registered users and 1 guest