An error while reading a key file on Thor.

[akhileshbadhri]

Hello Everyone,

I am reading a key file on THOR and distributing it. I get the following error while I do that -

Remote streaming failure, failing over to direct read for: <Logical file path>
ERROR: cmd=RFCStreamRead, error=Internal Error (0, Failed to read key header: file too small, could not read 272 bytes)

Am I doing something wrong here ? Request your help.

Thanks and regards,
Akhilesh Badhri

[rtaylor]

Akhilesh,

Am I doing something wrong here ?

Without more details, it's impossible to say.

Remote streaming failure, failing over to direct read for: <Logical file path>
ERROR: cmd=RFCStreamRead, error=Internal Error (0, Failed to read key header: file too small, could not read 272 bytes)

This error message tells me:

• "streaming failure, failing over to direct read" -- how are you trying to access this file?
• "Failed to read key header" -- maybe there's something about the physical file that may be corrupt?
• Or possibly your INDEX definition is specifying the wrong filename?
• Or ... ???

HTH,

Richard

[akhileshbadhri]

Thank you Richard. Following are the details about what I am doing -

I am getting the error while the key is distributed. The key is distributed for a join with another big dataset on Thor. Following is the distribute I tried and both of them give error.

distribute(Key, hash64(field1+field2));
distribute(pull(Key), hash64(field1+field2));


If I simply read the key like "Key;" it works fine.
I am running the code on the same Thor environment where the logical file resides. I am not doing foreign.

Hope this helps.

Thanks and regards,
Akhilesh Badhri.

[rtaylor]

Akhilesh,

I am getting the error while the key is distributed. The key is distributed for a join with another big dataset on Thor.

OK, so I'm assuming by "key" you mean INDEX. For a JOIN of two INDEXes, you simply need to use the INDEXes as the join files and use the KEYED keyword in the JOIN condition to ensure you get the performance advantage of INDEXes.

Using DISTRIBUTE treats the INDEX as a DATASET, eliminating the performance advantages of using the binary tree to get to the specific records you want. Not recommended.

I'd start by just eliminating the DISTRIBUTEs and see how well the JOIN works.

HTH,

Richard

[akhileshbadhri]

Fine Richard.
Thanks a lot for the input.

I will try this approach.

Thanks and regards,
Akhilesh Badhri.