Kasi Subrahmanyam
2015-10-27 21:31:37 UTC
Hi,
I have a requirement where i have to send one line to one mapper in a file
but doing it using hive.
How can we implement the functionality of NLineInptFormat in Hive.
I couldnt find this and i tried the following configuration in hive
set hive.merge.mapfiles=false;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
set mapreduce.input.fileinputformat.split.maxsize=100;
I have a file with onlt 10000 rows but i need each row to go different mapper
But if i used the above configuration in hive some of the rows are missing
based on the max split size i provide
Thanks,
I have a requirement where i have to send one line to one mapper in a file
but doing it using hive.
How can we implement the functionality of NLineInptFormat in Hive.
I couldnt find this and i tried the following configuration in hive
set hive.merge.mapfiles=false;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
set mapreduce.input.fileinputformat.split.maxsize=100;
I have a file with onlt 10000 rows but i need each row to go different mapper
But if i used the above configuration in hive some of the rows are missing
based on the max split size i provide
Thanks,