Logo Search packages:      
Sourcecode: hadoop version File versions  Download package

static void org::apache::hadoop::tools::DistCp::setMapCount ( long  totalBytes,
JobConf  job 
) throws IOException [inline, static, private]

Calculate how many maps to run. Number of maps is bounded by a minimum of the cumulative size of the copy / (distcp.bytes.per.map, default BYTES_PER_MAP or -m on the command line) and at most (distcp.max.map.tasks, default MAX_MAPS_PER_NODE * nodes in the cluster).

totalBytes Count of total bytes for job
job The job to configure
Count of maps to run.

Definition at line 925 of file DistCp.java.

Referenced by setup().

    int numMaps =
      (int)(totalBytes / job.getLong(BYTES_PER_MAP_LABEL, BYTES_PER_MAP));
    numMaps = Math.min(numMaps, 
        job.getInt(MAX_MAPS_LABEL, MAX_MAPS_PER_NODE *
          new JobClient(job).getClusterStatus().getTaskTrackers()));
    job.setNumMapTasks(Math.max(numMaps, 1));

Generated by  Doxygen 1.6.0   Back to index