Can we spray binary data on multiple nodes
Hi..
As if we spray CSV, XML or FLAT files, then distribution of the file across all the nodes of the target cluster is such that each single record is always whole and complete on a single node and the records are "evenly" distributed across the nodes.
But in binary data my one record can be on 50 rows and other can be on 2 rows. Also each row can differ in columns then how spraying works...?????
Is there any option to define delimiters so that it know different records and spray it on different nodes...
Plzz help..
As if we spray CSV, XML or FLAT files, then distribution of the file across all the nodes of the target cluster is such that each single record is always whole and complete on a single node and the records are "evenly" distributed across the nodes.
But in binary data my one record can be on 50 rows and other can be on 2 rows. Also each row can differ in columns then how spraying works...?????
Is there any option to define delimiters so that it know different records and spray it on different nodes...
Plzz help..
- Ankita Singla
- Posts: 21
- Joined: Tue Jul 24, 2012 7:02 am
Ankita,
The closest thing that I can think of to what I think you're describing would be a file wherein the data (either binary or textual) is comprised of multiple rows of different record types which are grouped together to describe a single logical entity in your data. These multiple physical records would be grouped either by a grouping identifier in the data (the logical entity identifier), or simply the physical positioning of the records in the file -- such as:
Your real problem with this kind of data is in defining the RECORD structure for the file so that you can easily work with it. Take a look at the IFBLOCK discussion in the RECORD structure documentation for one way to accomplish this.
However, if I've misunderstood and this is not the type of data you're working with, then can you please expand on your description so we can better help you?
HTH,
Richard
We work with binary data files all the time, with both fixed-length and variable-length records. Typically, each record has the same structure and all the data is contained in that one record with a delimiter indicating the end of the record. Spraying this type of data simply requires knowing what the record delimiter is.But in binary data my one record can be on 50 rows and other can be on 2 rows. Also each row can differ in columns then how spraying works...?????
The closest thing that I can think of to what I think you're describing would be a file wherein the data (either binary or textual) is comprised of multiple rows of different record types which are grouped together to describe a single logical entity in your data. These multiple physical records would be grouped either by a grouping identifier in the data (the logical entity identifier), or simply the physical positioning of the records in the file -- such as:
- record type 10 is always first,
- always followed by a 20,
- then optionally some 30s and/or 40s,
- and the next grouping begins with the next instance of a 10
Your real problem with this kind of data is in defining the RECORD structure for the file so that you can easily work with it. Take a look at the IFBLOCK discussion in the RECORD structure documentation for one way to accomplish this.
However, if I've misunderstood and this is not the type of data you're working with, then can you please expand on your description so we can better help you?
HTH,
Richard
- rtaylor
- Community Advisory Board Member
- Posts: 1409
- Joined: Wed Oct 26, 2011 7:40 pm
Thanks for the reply.....
May be it can help me in some way...
May be it can help me in some way...

- Ankita Singla
- Posts: 21
- Joined: Tue Jul 24, 2012 7:02 am
3 posts
• Page 1 of 1
Who is online
Users browsing this forum: Bing [Bot] and 1 guest