Examples of org.apache.mahout.df.mapreduce.Builder

org.apache.mahout.df.mapreduce.Builder
Base class for Mapred DecisionForest builders. Takes care of storing the parameters common to the mapred implementations.
The child classes must implement at least :
- void configureJob(Job) : to further configure the job before its launch; and
- DecisionForest parseOutput(Job, PredictionCallback) : in order to convert the job outputs into a DecisionForest and its corresponding oob predictions

    Dataset dataset = Dataset.load(getConf(), datasetPath);
    
    ForestPredictions callback = isOob ? new ForestPredictions(dataset.nbInstances(), dataset.nblabels())
        : null;
    
    Builder forestBuilder;
    
    if (isPartial) {
      log.info("Partial Mapred implementation");
      forestBuilder = new PartialBuilder(treeBuilder, dataPath, datasetPath, seed, getConf());
    } else {
      log.info("InMem Mapred implementation");
      forestBuilder = new InMemBuilder(treeBuilder, dataPath, datasetPath, seed, getConf());
    }
    
    log.info("Building the forest...");
    long time = System.currentTimeMillis();
    
    DecisionForest forest = forestBuilder.build(nbTrees, callback);
    
    time = System.currentTimeMillis() - time;
    log.info("Build Time: {}", DFUtils.elapsedTime(time));
    
    if (isOob) {

View Full Code Here

    
    // disable the second step because we can test without it
    // and we won't be able to serialize the MockNode
    PartialBuilder.setStep2(conf, false);
    long seed = 1L;
    Builder builder = new PartialSequentialBuilder(treeBuilder, dataPath, dataset, seed, conf);


    // remove the output path (its only used for testing)
    Path outputPath = builder.getOutputPath(conf);
    HadoopUtil.overwriteOutput(outputPath);


    builder.build(NUM_TREES, new MockCallback(data));
  }

View Full Code Here

    
    // disable the second step because we can test without it
    // and we won't be able to serialize the MockNode
    PartialBuilder.setStep2(conf, false);
    long seed = 1L;
    Builder builder = new PartialSequentialBuilder(treeBuilder, dataPath, dataset, seed, conf);


    // remove the output path (its only used for testing)
    Path outputPath = builder.getOutputPath(conf);
    HadoopUtil.delete(conf, outputPath);


    builder.build(NUM_TREES, new MockCallback(data));
  }

View Full Code Here

TOP

Related Classes of org.apache.mahout.df.mapreduce.Builder

org.apache.hadoop.conf.Configuration

org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.Path

org.apache.hadoop.mapreduce.Job

org.apache.mahout.df.DecisionForest

org.apache.mahout.df.mapred.BuildForest

org.apache.mahout.df.mapreduce.partial.PartitionBugTest

java.io.IOException

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.