Examples of GenericOptionsParser

org.apache.hadoop.util.GenericOptionsParser

GenericOptionsParser is a utility to parse command line arguments generic to the Hadoop framework. GenericOptionsParser recognizes several standarad command line arguments, enabling applications to easily specify a namenode, a jobtracker, additional configuration resources etc.

Generic Options

The supported generic options are:

 -conf <configuration file>     specify a configuration file -D <property=value>            use value for given property -fs <local|namenode:port>      specify a namenode -jt <local|jobtracker:port>    specify a job tracker -files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster -libjars <comma separated list of jars>   specify comma separated jar files to include in the classpath. -archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.

The general command line syntax is:

 bin/hadoop command [genericOptions] [commandOptions]

Generic command line arguments might modify Configuration objects, given to constructors.

The functionality is implemented using Commons CLI.

Examples:

 $ bin/hadoop dfs -fs darwin:8020 -ls /data list /data directory in dfs with namenode darwin:8020 $ bin/hadoop dfs -D fs.default.name=darwin:8020 -ls /data list /data directory in dfs with namenode darwin:8020 $ bin/hadoop dfs -conf hadoop-site.xml -ls /data list /data directory in dfs with conf specified in hadoop-site.xml $ bin/hadoop job -D mapred.job.tracker=darwin:50020 -submit job.xml submit a job to job tracker darwin:50020 $ bin/hadoop job -jt darwin:50020 -submit job.xml submit a job to job tracker darwin:50020 $ bin/hadoop job -jt local -submit job.xml submit a job to local runner $ bin/hadoop jar -libjars testlib.jar  -archives test.tgz -files file.txt inputjar args job submission with libjars, files and archives

@see Tool @see ToolRunner

Examples of org.apache.hadoop.util.GenericOptionsParser

  }




  public int run(String[] args) throws Exception {
    Configuration conf = getConf();
    args = new GenericOptionsParser(conf, args).getRemainingArgs();


    String serverUri = args[0];
    String inputDir = args[1];
    String tableName = args[2];
    String outputDir = args[3];

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

    .create("extrafiles");
    
    opts.addOption(logFiles);
    opts.addOption(extraFiles);
    
    GenericOptionsParser parser =
      new GenericOptionsParser(this.getConf(), opts, args);
    
    String[] remainingArgs = parser.getRemainingArgs();
    if (remainingArgs.length != 1) {
      usage();
      return -1;
    }
    if (remainingArgs[0].compareTo("check") == 0) {
      this.readOnly = true;
    } else if (remainingArgs[0].compareTo("upgrade") != 0) {
      usage();
      return -1;
    }


    if (readOnly) {
      this.logFiles = ACTION.IGNORE;
      this.otherFiles = ACTION.IGNORE;


    } else {
      CommandLine commandLine = parser.getCommandLine();


      ACTION action = null;
      if (commandLine.hasOption("logfiles")) {
        action = options.get(commandLine.getOptionValue("logfiles"));
        if (action == null) {

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

    if (conf == null)
      conf = new HdfsConfiguration();
    
    if (args != null) {
      // parse generic hadoop options
      GenericOptionsParser hParser = new GenericOptionsParser(conf, args);
      args = hParser.getRemainingArgs();
    }
    
    if (!parseArguments(args, conf)) {
      printUsage(System.err);
      return null;

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

        "Name of the delegation token renewer");
    fetcherOptions.addOption(CANCEL, false, "cancel the token");
    fetcherOptions.addOption(RENEW, false, "renew the token");
    fetcherOptions.addOption(PRINT, false, "print the token");
    fetcherOptions.addOption(HELP_SHORT, HELP, false, "print out help information");
    GenericOptionsParser parser = new GenericOptionsParser(conf,
        fetcherOptions, args);
    CommandLine cmd = parser.getCommandLine();
    
    // get options
    final String webUrl = cmd.hasOption(WEBSERVICE) ? cmd
        .getOptionValue(WEBSERVICE) : null;
    final String renewer = cmd.hasOption(RENEWER) ? 
        cmd.getOptionValue(RENEWER) : null;
    final boolean cancel = cmd.hasOption(CANCEL);
    final boolean renew = cmd.hasOption(RENEW);
    final boolean print = cmd.hasOption(PRINT);
    final boolean help = cmd.hasOption(HELP);
    String[] remaining = parser.getRemainingArgs();


    // check option validity
    if (help) {
      printUsage(System.out);
      System.exit(0);

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

  public static JobConf createValueAggregatorJob(String args[])
    throws IOException {


    Configuration conf = new Configuration();
    
    GenericOptionsParser genericParser 
      = new GenericOptionsParser(conf, args);
    args = genericParser.getRemainingArgs();
    
    if (args.length < 2) {
      System.out.println("usage: inputDirs outDir "
          + "[numOfReducer [textinputformat|seq [specfile [jobName]]]]");
      GenericOptionsParser.printGenericCommandUsage(System.out);

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

    cli.addOption("program", false, "URI to application executable", "class");
    Parser parser = cli.createParser();
    try {


      // check generic arguments -conf
      GenericOptionsParser genericParser = new GenericOptionsParser(getConf(),
          args);
      // get other arguments
      CommandLine results = parser.parse(cli.options,
          genericParser.getRemainingArgs());


      BSPJob job = new BSPJob(getConf());


      if (results.hasOption("input")) {
        FileInputFormat.setInputPaths(job, results.getOptionValue("input"));

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

  public static JobConf createValueAggregatorJob(String args[])
    throws IOException {


    Configuration conf = new Configuration();
    
    GenericOptionsParser genericParser 
      = new GenericOptionsParser(conf, args);
    args = genericParser.getRemainingArgs();
    
    if (args.length < 2) {
      System.out.println("usage: inputDirs outDir "
          + "[numOfReducer [textinputformat|seq [specfile [jobName]]]]");
      GenericOptionsParser.printGenericCommandUsage(System.out);

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

    .create("extrafiles");
    
    opts.addOption(logFiles);
    opts.addOption(extraFiles);
    
    GenericOptionsParser parser =
      new GenericOptionsParser(this.getConf(), opts, args);
    
    String[] remainingArgs = parser.getRemainingArgs();
    if (remainingArgs.length != 1) {
      usage();
      return -1;
    }
    if (remainingArgs[0].compareTo("check") == 0) {
      this.readOnly = true;
    } else if (remainingArgs[0].compareTo("upgrade") != 0) {
      usage();
      return -1;
    }


    if (readOnly) {
      this.logFiles = ACTION.IGNORE;
      this.otherFiles = ACTION.IGNORE;


    } else {
      CommandLine commandLine = parser.getCommandLine();


      ACTION action = null;
      if (commandLine.hasOption("logfiles")) {
        action = options.get(commandLine.getOptionValue("logfiles"));
        if (action == null) {

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

  }


  @SuppressWarnings("static-access")
  private int parseArgs(String[] args) {
    Options opts = new Options();
    GenericOptionsParser parser =
      new GenericOptionsParser(this.getConf(), opts, args);
    String[] remainingArgs = parser.getRemainingArgs();
    if (remainingArgs.length != 1) {
      usage();
      return -1;
    }
    if (remainingArgs[0].compareTo("check") == 0) {

View Full Code Here

Examples of org.apache.hadoop.util.GenericOptionsParser

   * @param region HRegion to add to <code>meta</code>
   *
   * @throws IOException
   */
  private int parseArgs(String[] args) {
    GenericOptionsParser parser =
      new GenericOptionsParser(this.getConf(), args);
    
    String[] remainingArgs = parser.getRemainingArgs();
    if (remainingArgs.length != 3) {
      usage();
      return -1;
    }
    tableName = Bytes.toBytes(remainingArgs[0]);

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.