Fri Aug 17, 2018 3:19 am
Login Register Lost Password? Contact Us


Dataset too large to output to workunit Exception

Comments and questions related to the Enterprise Control Language

Thu Apr 26, 2012 7:49 pm Change Time Zone

I want to return Dataset in Roxie which contains only 2 Rows but the Datasize is more than 10 Mb, So when i Compile my Roxie and try to query it From ESP(Port 8002),It gives me an Exception
Reported by: eclagent
Message: System error: 0: Dataset too large to output to workunit (limit 10) megabytes, in result (sequence=0).
Roxie Query is as Follows:

Code: Select all
IMPORT std;
EXPORT NgramRoxie := FUNCTION

STRING text_value := '': STORED('search'); ;

text_value_regexPattern := '\\b'+ text_value +'\\b';

ngramRec:=Record
UNSIGNED id;
String200 word;
End;

TokenizeDataset := DATASET('~poc::naveen::result::ngramsdata',{ngramRec,UNSIGNED8 RecPos{virtual(fileposition)}},THOR);

     
myIndex := INDEX(TokenizeDataset,{id,word,RecPos},'~IDX::Email::NewIndex');

//mySet := FETCH(myText, myIndex(REGEXFIND(text_value_regexPattern,text)), RIGHT.RecPos);

mySet := FETCH(TokenizeDataset, myIndex(REGEXFIND(text_value_regexPattern,word,NOCASE)), RIGHT.RecPos); //,TRUE

outout := Dedup(mySet,id);

readDataRec := RECORD
STRING filename;
DATA inputfile;
String10000000 text; //{maxlength(10000000)};
END;


readDataRec1 := RECORD
integer id;
STRING filename;
String10000000 text; //{maxlength(10000000)};
END;

filedata := dataset('~poc::naveen::result::wedidit3',readDataRec,thor);
filedatawithid:=Project(filedata,Transform(readDataRec1,Self.id:=Counter,Self:=Left));


JoinedDS:=join(outout,filedatawithid,Left.id=Right.id);
// ret:=OUTPUT(JoinedDS,{text});

RETURN JoinedDS;

END;
Bhagwant
 
Posts: 10
Joined: Tue Feb 14, 2012 3:17 pm

Fri Apr 27, 2012 11:58 am Change Time Zone

Try adding the #OPTION(outputlimit,<new value>) to your query. The default is 10 MB.

Regards,

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 975
Joined: Wed Jun 29, 2011 7:13 pm

Tue Dec 12, 2017 3:30 pm Change Time Zone

First, the first parameter is a string, and needs to be quoted.
Second, the number is the number of megabytes to use.
Third, the strings that you give are case sensitive.
Fourth, there are two settings: outputLimit and outputLimitMb. These are synonyms. Both take the size in megabytes.

Working and tested example:
Code: Select all
IMPORT std;
#OPTION('outputLimitMb', 100);
Std.File.LogicalFileList('*', 1, 1, FALSE);


Let's just say we have a lot of files :D
kovacsbv
 
Posts: 33
Joined: Fri Aug 05, 2011 12:53 pm


Return to ECL

Who is online

Users browsing this forum: No registered users and 1 guest

cron