List of copyToLocalFile() Examples

Examples of copyToLocalFile()

org.apache.hadoop.fs.FileSystem.copyToLocalFile()
The src file is under FS, and the dst is on the local disk. Copy it from FS control to the local dst name.
org.apache.hadoop.fs.HarFileSystem.copyToLocalFile()
copies the file in the har filesystem to a local file.
org.apache.hadoop.fs.LocalFileSystem.copyToLocalFile()

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

                                                      +"/"+jobid + ".xml");
    this.localJarFile = default_job_conf.getLocalPath(JobTracker.SUBDIR
                                                      +"/"+ jobid + ".jar");
    FileSystem fs = FileSystem.get(default_conf);
    Path jobFile = new Path(default_conf.getSystemDir(), jobid + "/job.xml");
    fs.copyToLocalFile(jobFile, localJobFile);
    conf = new JobConf(localJobFile);
    this.priority = conf.getJobPriority();
    this.profile = new JobProfile(conf.getUser(), jobid, 
                                  jobFile.toString(), url, conf.getJobName());
    String jarFile = conf.getJar();

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

    this.priority = conf.getJobPriority();
    this.profile = new JobProfile(conf.getUser(), jobid, 
                                  jobFile.toString(), url, conf.getJobName());
    String jarFile = conf.getJar();
    if (jarFile != null) {
      fs.copyToLocalFile(new Path(jarFile), localJarFile);
      conf.setJar(localJarFile.toString());
    }


    this.numMapTasks = conf.getNumMapTasks();
    this.numReduceTasks = conf.getNumReduceTasks();

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

    this.localJarFile = default_job_conf.getLocalPath(JobTracker.SUBDIR
                                                      +"/"+ jobid + ".jar");
    Path jobDir = jobtracker.getSystemDirectoryForJob(jobId);
    FileSystem fs = jobDir.getFileSystem(default_conf);
    jobFile = new Path(jobDir, "job.xml");
    fs.copyToLocalFile(jobFile, localJobFile);
    conf = new JobConf(localJobFile);
    this.priority = conf.getJobPriority();
    this.status.setJobPriority(this.priority);
    this.profile = new JobProfile(conf.getUser(), jobid, 
                                  jobFile.toString(), url, conf.getJobName(),

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

    this.profile = new JobProfile(conf.getUser(), jobid, 
                                  jobFile.toString(), url, conf.getJobName(),
                                  conf.getQueueName());
    String jarFile = conf.getJar();
    if (jarFile != null) {
      fs.copyToLocalFile(new Path(jarFile), localJarFile);
      conf.setJar(localJarFile.toString());
    }


    this.numMapTasks = conf.getNumMapTasks();
    this.numReduceTasks = conf.getNumReduceTasks();

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

                pathSrc.getName());
            // LOG.info("user.dir: "+System.getProperty("user.dir"));
            // LOG.info("WorkingDirectory: "+local.getWorkingDirectory());
            // LOG.info("pathDst: " + pathDst);
            LOG.debug("copyToLocalFile: " + pathDst);
            hdfs.copyToLocalFile(pathSrc, pathDst);
            local.deleteOnExit(pathDst);
            files.append(pathDst.toUri().getPath());
          }
          first = false;
        }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

            // File dst = File.createTempFile(pathSrc.getName() + "-", ".jar");
            Path pathDst = new Path(local.getWorkingDirectory(),
                pathSrc.getName());


            LOG.debug("copyToLocalFile: " + pathDst);
            hdfs.copyToLocalFile(pathSrc, pathDst);
            local.deleteOnExit(pathDst);


            libjars[i] = new URL(local.makeQualified(pathDst).toString());
          }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

                                                      +"/"+jobid + ".xml");
    this.localJarFile = default_job_conf.getLocalPath(JobTracker.SUBDIR
                                                      +"/"+ jobid + ".jar");
    FileSystem fs = FileSystem.get(default_conf);
    Path jobFile = new Path(default_conf.getSystemDir(), jobid + "/job.xml");
    fs.copyToLocalFile(jobFile, localJobFile);
    conf = new JobConf(localJobFile);
    this.priority = conf.getJobPriority();
    this.profile = new JobProfile(conf.getUser(), jobid, 
                                  jobFile.toString(), url, conf.getJobName());
    String jarFile = conf.getJar();

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

    this.priority = conf.getJobPriority();
    this.profile = new JobProfile(conf.getUser(), jobid, 
                                  jobFile.toString(), url, conf.getJobName());
    String jarFile = conf.getJar();
    if (jarFile != null) {
      fs.copyToLocalFile(new Path(jarFile), localJarFile);
      conf.setJar(localJarFile.toString());
    }


    this.numMapTasks = conf.getNumMapTasks();
    this.numReduceTasks = conf.getNumReduceTasks();

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

          if (!b)
            throw new IOException("Not able to create job directory "
                                  + jobDir.toString());
        }
        FileSystem fs =FileSystem.getNamed(jobClient.getFilesystemName(),fConf);
        fs.copyToLocalFile(new Path(jobFile), localJobFile);
        JobConf localJobConf = new JobConf(localJobFile);
        
        // create the 'work' directory
        File workDir = new File(new File(localJobFile.toString()).getParent(),
                                "work");

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.copyToLocalFile()

        
        // unjar the job.jar files in workdir
        String jarFile = localJobConf.getJar();
        if (jarFile != null) {
          localJarFile = new Path(jobDir,"job.jar");
          fs.copyToLocalFile(new Path(jarFile), localJarFile);
          localJobConf.setJar(localJarFile.toString());
          OutputStream out = localFs.create(localJobFile);
          try {
            localJobConf.write(out);
          } finally {

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.