Discussion:
where does the temporary task directory get created in mapred?
Koert Kuipers
2014-09-27 18:28:51 UTC
Permalink
i was looking at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter's setupTask
method, but it does nothing and states:

// FileOutputCommitter's setupTask doesn't do anything. Because the
// temporary task directory is created on demand when the
// task is writing.

ok i got that. but then where is the temporary task then created "on demand
when the task is writing"? i looked all over the OutputCommitters,
RecordWriters, OutputFormats, but i cannot find it. i must have overlooked
it...

can someone please point me to the relevant class/method? thank you so much!

best, koert

Loading...