Tue Oct 19, 2021 9:19 am
Login Register Lost Password? Contact Us


Reading in Sequence files using H2H Connector

Topics related to the Hadoop Connector or migrating data from Hadoop

Tue Dec 16, 2014 2:30 pm Change Time Zone

Anyone have an example on reading a sequence file from HDFS into HPCC?

We tried using the pipe connector, but the options are only FLAT or CSV and the HDFS file is default codec compression and the Key Value is Text:Text

The file is generated by the wordcount benchmark. You can download the source at https://github.com/intel-hadoop/HiBench

Thanks,
Lee
Lee_Meadows
 
Posts: 16
Joined: Mon Jul 21, 2014 1:43 pm

Fri Dec 19, 2014 6:50 pm Change Time Zone

Lee, the H2H commercial only supports FLAT/CSV files as you pointed out. I'm not familiar with the particular file type you're working with, can it be treated as a fixed length record file? If it's a variable length record file, is there a delimiter which denotes the end of the record?
rodrigo.pastrana
 
Posts: 26
Joined: Tue Jun 10, 2014 2:19 pm


Return to From Hadoop

Who is online

Users browsing this forum: No registered users and 1 guest

cron