Wed Aug 15, 2018 4:37 pm
Login Register Lost Password? Contact Us


Processing a Zipped File

Questions around writing code and queries

Fri Aug 04, 2017 2:32 pm Change Time Zone

I have a file that is sent to me zipped (datafile.zip) the contents are records with a total length of 1742 bytes.

I have sprayed the file as a BLOB (since the data is compressed). I then attempt to read the file and process the records. I am using the PIPE option of the DATASET and pass it 'gunzip' (which exists on the server and when tested on the server, it decompresses the file).

Code: Select all
layout_zipfile := RECORD
   string1742 fullrec;
END;

ds_zipfile := dataset(  '~spray::target::20170803::p701650.compressed_data'
                      , layout_zipfile
                      , PIPE('gunzip', THOR)
                     );

ds_zipfile;

But I keep getting the message: System error: -1: CFileSerialStream::get read past end of stream

I'm not sure if I'm spraying the data correctly (should I spray as a CSV even though uncompressed, the data is fixed?). Any insight would be greatly appreciated.
P.S. I tried using an EMBED for python, but it wouldn't accept the embedded code (a simple "HELLO WORLD") and we have no code in production I could reference.

Thank You
John Meier
 
Posts: 14
Joined: Wed Jun 29, 2016 7:45 pm

Fri Aug 04, 2017 7:23 pm Change Time Zone

Hi John,

Try spraying the file as a CSV (Delimited) and then remove the THOR reference from your PIPE command.

Regards,

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 975
Joined: Wed Jun 29, 2011 7:13 pm

Tue Apr 10, 2018 11:29 pm Change Time Zone

John, if you're watching...
Did you ever get this working?
jwilt
 
Posts: 50
Joined: Wed Feb 27, 2013 7:46 pm


Return to Programming

Who is online

Users browsing this forum: No registered users and 1 guest

cron