Examples of co.cask.cdap.data.stream.TimePartitionedStreamFileWriter

co.cask.cdap.data.stream.TimePartitionedStreamFileWriter
Stream file path format:

Each file has path pattern
```
 [streamName]/[partitionName]/[bucketName].[dat|idx] 
```
Where {@code .dat} is the event data file, {@code .idx} is the accompany index file.

The {@code partitionName} is formatted as
```
 [partitionStartTime].[duration] 
```
with both {@code partitionStartTime} and {@code duration} in seconds.

The {@code bucketName} is formatted as
```
 "bucket".[bucketId].[seqNo] 
```
where the {@code bucketId} is an integer. The {@code seqNo} is a strictly increasing integer for the same{@code bucketId}.

      Preconditions.checkNotNull(config.getLocation(), "Location for stream {} is unknown.", config.getName());


      Location baseLocation = StreamUtils.createGenerationLocation(config.getLocation(), generation);
      Locations.mkdirsIfNotExists(baseLocation);


      return new TimePartitionedStreamFileWriter(baseLocation, config.getPartitionDuration(),
                                                 filePrefix, config.getIndexInterval());


    } catch (Exception e) {
      Throwables.propagateIfPossible(e, IOException.class);
      throw new IOException(e);

View Full Code Here

TOP

Related Classes of co.cask.cdap.data.stream.TimePartitionedStreamFileWriter

co.cask.cdap.data.runtime.LocationStreamFileWriterFactory

co.cask.cdap.data.stream.TimePartitionedStreamFileWriter.TimePartition

org.apache.twill.filesystem.Location

java.io.IOException

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.