Examples of eu.stratosphere.core.fs.FileSystem.open()

Class eu.stratosphere.core.fs.FileSystem

Examples of eu.stratosphere.core.fs.FileSystem.open()

eu.stratosphere.core.fs.FileSystem.open()
Opens an FSDataInputStream at the indicated Path. @param f the file name to open @param bufferSize the size of the buffer to be used.

      // Write out the length of the file
      final FileStatus file = fs.getFileStatus(jar);
      out.writeLong(file.getLen());


      // Now write the jar file
      final FSDataInputStream inStream = fs.open(this.userJars.get(i));
      final byte[] buf = new byte[BUFFERSIZE];
      int read = inStream.read(buf, 0, buf.length);
      while (read > 0) {
        out.write(buf, 0, read);
        read = inStream.read(buf, 0, buf.length);

View Full Code Here

      try {
        if (!lfs.exists(tmp)) {
          FSDataOutputStream lfsOutput = lfs.create(tmp, false);
          Path distributedPath = new Path(filePath);
          FileSystem fs = distributedPath.getFileSystem();
          FSDataInputStream fsInput = fs.open(distributedPath);
          IOUtils.copyBytes(fsInput, lfsOutput);
        }
      } catch (IOException e1) {
        throw new RuntimeException("Error copying a file from hdfs to the local fs", e1);
      }

View Full Code Here

      long start = split.getStart();
      long length = split.getLength();


      final FileSystem fs = FileSystem.get(split.getPath().toUri());


      final FSDataInputStream fdis = fs.open(split.getPath());


      final LineReader lineReader = new LineReader(fdis, start, length, (1024 * 1024));


      byte[] line = lineReader.readLine();

View Full Code Here

      final long start = split.getStart();
      final long length = split.getLength();


      final FileSystem fs = FileSystem.get(split.getPath().toUri());


      final FSDataInputStream fdis = fs.open(split.getPath());


      final LineReader lineReader = new LineReader(fdis, start, length, (1024 * 1024));


      byte[] line = lineReader.readLine();

View Full Code Here

    final FSDataOutputStream outputStream = fs.create(objectPath, false);
    generateTestData(outputStream, fileSize);
    outputStream.close();


    // Now read the same file back from S3
    final FSDataInputStream inputStream = fs.open(objectPath);
    testReceivedData(inputStream, fileSize);
    inputStream.close();


    // Delete test bucket
    fs.delete(bucketPath, true);

View Full Code Here


    @Override
    public void run() {
      try {
        final FileSystem fs = FileSystem.get(this.split.getPath().toUri());
        this.fdis = fs.open(this.split.getPath());
        
        // check for canceling and close the stream in that case, because no one will obtain it
        if (this.aborted) {
          final FSDataInputStream f = this.fdis;
          this.fdis = null;

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.