Reducing Skew

[sban]

Hello,

I'm facing an issue wherein my data file is extremely skewed (+2900%, -100%)across slaves.

This is in spite of doing a hash32 distribute on two fields (one of which admittedly has lots of 0s, while the other is a mostly unique 38 digit integer).

Is there anything that I can do to reduce this skew? I could:
1. Exclude the field with 0s from the hash key
2. Use some other hashing function

Will be happy to provide whatever other information you need.

[rtaylor]

sban,

I would think your #1 would be your best/simplest/easiest option.

HTH,

Richard