Wed Aug 15, 2018 3:05 pm
Login Register Lost Password? Contact Us


Memory pool exhausted error

Topics related to the set of Machine Learning libraries and Matrix processing algorithms

Thu Jun 25, 2015 5:18 pm Change Time Zone

Dear Maryam,

Thanks, I guess that might be the problem, I was spraying that train-images.idx3-ubyte file directly.

Actually, I tried similar thing yesterday, I copied my result of 100 samples from input_data_tmp into a csv and then sprayed, I got the result(may be wrong as input is corrupted - not sure) but the conf value (confidence) was almost same in all the output records.

Please correct me if I am wrong - I included NNClassify also in the code to get the classified result. What I need to check here? Since there are 10 output neurons, I have to check one with highest probability or conf as the classified output right? Is this understanding correct or output represents something else?

Regards,
Pooja.
chennapooja
 
Posts: 61
Joined: Wed Oct 08, 2014 11:49 pm

Mon Jun 29, 2015 2:43 pm Change Time Zone

Dear Maryam,

As thor was down, could not test with "csv" sprayed file as input from 2 days, today when I tested getting below error:

System error: -1: Graph[326], localresultwrite[332]: SLAVE #1 [10.0.1.1:20100]: CMemoryBufferSerialStream::get read past end of stream (15680000,7421865),

and these kind of errors are not understandable at all. It will be of great help if any suggestions given to overcome above error.

Thanks and Regards,
Pooja.
chennapooja
 
Posts: 61
Joined: Wed Oct 08, 2014 11:49 pm

Tue Jun 30, 2015 1:37 pm Change Time Zone

Hi,

Did you try to spray just a very simple cvs file?
for example:
1,2,3
4,5,6

and then upload and spray the data.
use a simple program as below to make sure you can read the data:

value_record := RECORD
real f1 ;
real f2 ;
real f3 ;
END;

input_data_tmp := DATASET('~::SimpleUploadedFile', value_record, CSV);

OUTPUT(input_data_tmp);



Look at this manual to see how to upload and spary the daya :
http://cdn.hpccsystems.com/install/docs ... ndling.pdf

you first upload the data, then you use "spray delimited" to spray the data.

I hope these help!

-Maryam
maryamregister
 
Posts: 7
Joined: Tue Jun 23, 2015 4:51 pm

Tue Jun 30, 2015 1:42 pm Change Time Zone

Dear Maryam,

Its working perfectly fine with a simple dataset. I sprayed it and tested. I just extracted few records from MNIST dataset also, to be specific 5 records and put them into separate csv, then sprayed. It is also working good but the output probabilities are not that good, almost all the outputs have same probability. I understand its possible because training and no of iterations is not sufficient. But when I test with whole dataset, facing these kind of memory and thor slave related errors which I am not able to understand.

Thanks,
Pooja.
chennapooja
 
Posts: 61
Joined: Wed Oct 08, 2014 11:49 pm

Tue Jun 30, 2015 1:54 pm Change Time Zone

Dear Maryam,

I am still doubtful with data I am using though I have csv format file from byte file format now. So is it possible to attach the dataset input training and test files with which I can check once.

Thanks,
Pooja.
chennapooja
 
Posts: 61
Joined: Wed Oct 08, 2014 11:49 pm

Previous

Return to Machine Learning

Who is online

Users browsing this forum: No registered users and 1 guest