Examples of com.linkedin.databus.client.pub.FileSystemCheckpointPersistenceProvider

com.linkedin.databus.client.pub.FileSystemCheckpointPersistenceProvider
Stores checkpoints to disk as simple files. The format is that the checkpoint for each stream is stored in a different file whose name can be uniquely derived from the stream. For debugging purposes, the provider can also support rolling files, where the last few (configurable) checkpoint files are also stored. The format of the files is <StreamId>.xxx where xxx is monotonically increasing number. The latest checkpoint is <StreamId>.current . The format of the files is Line 1: json representation of the checkpoint Line 2: crc of json @author cbotev


    @Override
    public CheckpointPersistenceProvider createCheckpointPersistenceProvider(DbusPartitionInfo partition)
        throws InvalidConfigException
    {
      return new FileSystemCheckpointPersistenceProvider();
    }

View Full Code Here

      if (cp3Conf.getType() != CheckpointPersistenceStaticConfig.ProviderType.FILE_SYSTEM)
      {
        throw new RuntimeException("don't know what to do with cp3 type:" + cp3Conf.getType());
      }


      cp3 = new FileSystemCheckpointPersistenceProvider(cp3Conf.getFileSystem(), 2);
    }
    else if (null != _clientProps)
    {
      DatabusHttpClientImpl.Config clientConfBuilder =
          new DatabusHttpClientImpl.Config();
      ConfigLoader<DatabusHttpClientImpl.StaticConfig> configLoader =
          new ConfigLoader<DatabusHttpClientImpl.StaticConfig>(_propPrefix, clientConfBuilder);
      configLoader.loadConfig(_clientProps);


      DatabusHttpClientImpl.StaticConfig clientConf = clientConfBuilder.build();
      if (clientConf.getCheckpointPersistence().getType() !=
          CheckpointPersistenceStaticConfig.ProviderType.FILE_SYSTEM)
      {
        throw new RuntimeException("don't know what to do with cp3 type:" +
                                   clientConf.getCheckpointPersistence().getType());
      }


      cp3 = new FileSystemCheckpointPersistenceProvider(
          clientConf.getCheckpointPersistence().getFileSystem(), 2);
    }


    List<String> sourceList = Arrays.asList(_sources);
    Checkpoint cpOld = null != cp3 ? cp3.loadCheckpoint(sourceList) : new Checkpoint();

View Full Code Here

      switch (getType())
      {
        case FILE_SYSTEM:
        LOG.info("Creating file-system checkpoint persistence provider");
        cpPersistenceProvider =
            new FileSystemCheckpointPersistenceProvider(getFileSystem(), _protocolVersion);
        break;
        case SHARED:
        LOG.info("Creating shared ZooKeeper-based checkpoint persistence provider");
        cpPersistenceProvider = new SharedCheckpointPersistenceProvider(groupMember, getSharedState());
        break;

View Full Code Here

TOP

Related Classes of com.linkedin.databus.client.pub.FileSystemCheckpointPersistenceProvider

com.linkedin.databus.client.DatabusHttpClientImpl$CheckpointPersistenceStaticConfig

com.linkedin.databus.client.registration.TestDatabusV2ClusterRegistrationImpl$TestableDatabusV2ClusterRegistrationImpl

com.linkedin.databus.core.Checkpoint

com.linkedin.databus2.client.util.CheckpointSerializerMain

java.io.FileReader

java.io.BufferedReader

java.util.Formatter

java.io.IOException

java.io.FilenameFilter

java.io.PrintWriter

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.