Discussion:
Using NlineInputFormat in HIve
Kasi Subrahmanyam
2015-10-27 21:31:37 UTC
Permalink
Hi,
I have a requirement where i have to send one line to one mapper in a file
but doing it using hive.
How can we implement the functionality of NLineInptFormat in Hive.
I couldnt find this and i tried the following configuration in hive

set hive.merge.mapfiles=false;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;

set mapreduce.input.fileinputformat.split.maxsize=100;

I have a file with onlt 10000 rows but i need each row to go different mapper

But if i used the above configuration in hive some of the rows are missing

based on the max split size i provide


Thanks,

Loading...