List of listLocatedStatus() Examples

Examples of listLocatedStatus()

org.apache.hadoop.fs.FileSystem.listLocatedStatus()
List the statuses of the files/directories in the given path if the path is a directory. Return the file's status and block locations If the path is a file. If a returned status is a file, it contains the file's block locations. @param f is the path @return an iterator that traverses statuses of the files/directories in the given path @throws FileNotFoundException If f does not exist @throws IOException If an I/O error occurred
org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus()

Examples of org.apache.hadoop.fs.FileSystem.listLocatedStatus()

      FileStatus file = files.get(i);
      if (file.isDir()) {     // it's a MapFile
        Path p = file.getPath();
        FileSystem fs = p.getFileSystem(job.getConfiguration());
        // use the data file
        files.set(i, fs.listLocatedStatus(
            new Path(p, MapFile.DATA_FILE_NAME)).next());
      }
    }
    return files;
  }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.listLocatedStatus()

      } else if (matches.length == 0) {
        errors.add(new IOException("Input Pattern " + p + " matches 0 files"));
      } else {
        for (FileStatus globStat: matches) {
          for(RemoteIterator<LocatedFileStatus> itor = 
            fs.listLocatedStatus(globStat.getPath(),
              inputFilter); itor.hasNext();) {
            result.add(itor.next());
          }          
        }
      }

View Full Code Here

Examples of org.apache.hadoop.fs.FileSystem.listLocatedStatus()

          LOG.info(dirToClean + " doesn't exist");
          return;
        }
  
        RemoteIterator<LocatedFileStatus> itor;
        for( itor = fs.listLocatedStatus(dirToClean); itor.hasNext();) {
          LocatedFileStatus dirStat = itor.next();
          // Check if this is a directory matching the pattern
          if (!dirStat.isDir()) {
            continue;
          }

View Full Code Here

Examples of org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus()

     * Look at the output directory to see how many files belong to 
     * the current datanode
     */
    public int getNumberOfFiles() throws IOException {
      DistributedFileSystem dfs = (DistributedFileSystem)fs;
      RemoteIterator<LocatedFileStatus> iter = dfs.listLocatedStatus(outputPath);
      int fn = 0;
      while (iter.hasNext()) {
        LocatedFileStatus lfs = iter.next();
        if (lfs.isDir()) 
          continue;

View Full Code Here

0 1

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.