Examples of SnapshotDeletionPolicy

org.apache.lucene.index.SnapshotDeletionPolicy
A {@link IndexDeletionPolicy} that wraps around any other{@link IndexDeletionPolicy} and adds the ability to hold andlater release a single "snapshot" of an index. While the snapshot is held, the {@link IndexWriter} will notremove any files associated with it even if the index is otherwise being actively, arbitrarily changed. Because we wrap another arbitrary {@link IndexDeletionPolicy}, this gives you the freedom to continue using whatever {@link IndexDeletionPolicy} you would normally want to use with yourindex. Note that you can re-use a single instance of SnapshotDeletionPolicy across multiple writers as long as they are against the same index Directory. Any snapshot held when a writer is closed will "survive" when the next writer is opened.
WARNING: This API is a new and experimental and may suddenly change.
org.elasticsearch.index.deletionpolicy.SnapshotDeletionPolicy
Snapshot deletion policy allows to get snapshots of an index state (last commit or all commits) and if the deletion policy is used with all open index writers (JVM level) then the snapshot state will not be deleted until it will be released. @author kimchy (shay.banon)

Examples of org.apache.lucene.index.SnapshotDeletionPolicy

    for (int i = 0; i < 2048; i++) {
      builder.append(chars[random().nextInt(chars.length)]);
    }
    String content = builder.toString();


    final SnapshotDeletionPolicy sdp = new SnapshotDeletionPolicy(new KeepOnlyLastCommitDeletionPolicy());
    final Directory dir = new NRTCachingDirectory(newFSDirectory(createTempDir("nrt")), 5, 128);
    IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_4_6,
                                                     new MockAnalyzer(random()));
    config.setIndexDeletionPolicy(sdp);
    config.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
    final IndexWriter iw = new IndexWriter(dir, config);
    SearcherManager sm = new SearcherManager(iw, true, new SearcherFactory());
    final TrackingIndexWriter tiw = new TrackingIndexWriter(iw);
    ControlledRealTimeReopenThread<IndexSearcher> controlledRealTimeReopenThread =
      new ControlledRealTimeReopenThread<>(tiw, sm, maxStaleSecs, 0);


    controlledRealTimeReopenThread.setDaemon(true);
    controlledRealTimeReopenThread.start();


    List<Thread> commitThreads = new ArrayList<>();


    for (int i = 0; i < 500; i++) {
      if (i > 0 && i % 50 == 0) {
        Thread commitThread =  new Thread(new Runnable() {
            @Override
            public void run() {
              try {
                iw.commit();
                IndexCommit ic = sdp.snapshot();
                for (String name : ic.getFileNames()) {
                  //distribute, and backup
                  //System.out.println(names);
                  assertTrue(slowFileExists(dir, name));
                }

View Full Code Here

Examples of org.apache.lucene.index.SnapshotDeletionPolicy

    }
    
    @Override
    protected IndexWriterConfig createIndexWriterConfig(OpenMode openMode) {
      IndexWriterConfig conf = super.createIndexWriterConfig(openMode);
      conf.setIndexDeletionPolicy(new SnapshotDeletionPolicy(conf.getIndexDeletionPolicy()));
      return conf;
    }

View Full Code Here

Examples of org.apache.lucene.index.SnapshotDeletionPolicy


  private void runTest(Directory dir) throws IOException {
    // Run for ~7 seconds
    final long stopTime = System.currentTimeMillis() + 7000;


    SnapshotDeletionPolicy dp = new SnapshotDeletionPolicy(new KeepOnlyLastCommitDeletionPolicy());
    final IndexWriter writer = new IndexWriter(dir, true, new StandardAnalyzer(), dp);


    // Force frequent commits
    writer.setMaxBufferedDocs(2);

View Full Code Here

Examples of org.apache.lucene.index.SnapshotDeletionPolicy

    for (int i = 0; i < 2048; i++) {
      builder.append(chars[random().nextInt(chars.length)]);
    }
    String content = builder.toString();


    final SnapshotDeletionPolicy sdp = new SnapshotDeletionPolicy(new KeepOnlyLastCommitDeletionPolicy());
    final Directory dir = new NRTCachingDirectory(newFSDirectory(createTempDir("nrt")), 5, 128);
    IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_46,
                                                     new MockAnalyzer(random()));
    config.setIndexDeletionPolicy(sdp);
    config.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
    final IndexWriter iw = new IndexWriter(dir, config);
    SearcherManager sm = new SearcherManager(iw, true, new SearcherFactory());
    final TrackingIndexWriter tiw = new TrackingIndexWriter(iw);
    ControlledRealTimeReopenThread<IndexSearcher> controlledRealTimeReopenThread =
      new ControlledRealTimeReopenThread<>(tiw, sm, maxStaleSecs, 0);


    controlledRealTimeReopenThread.setDaemon(true);
    controlledRealTimeReopenThread.start();


    List<Thread> commitThreads = new ArrayList<>();


    for (int i = 0; i < 500; i++) {
      if (i > 0 && i % 50 == 0) {
        Thread commitThread =  new Thread(new Runnable() {
            @Override
            public void run() {
              try {
                iw.commit();
                IndexCommit ic = sdp.snapshot();
                for (String name : ic.getFileNames()) {
                  //distribute, and backup
                  //System.out.println(names);
                  assertTrue(slowFileExists(dir, name));
                }

View Full Code Here

Examples of org.apache.lucene.index.SnapshotDeletionPolicy

    serverIndexDir = newDirectory();
    serverReplicator = new LocalReplicator();
    startServer();
    
    IndexWriterConfig conf = newIndexWriterConfig(TEST_VERSION_CURRENT, null);
    conf.setIndexDeletionPolicy(new SnapshotDeletionPolicy(conf.getIndexDeletionPolicy()));
    writer = new IndexWriter(serverIndexDir, conf);
    reader = DirectoryReader.open(writer, false);
  }

View Full Code Here

Examples of org.apache.lucene.index.SnapshotDeletionPolicy

  
  @Test
  public void testNoCommit() throws Exception {
    Directory indexDir = newDirectory();
    IndexWriterConfig conf = new IndexWriterConfig(TEST_VERSION_CURRENT, null);
    conf.setIndexDeletionPolicy(new SnapshotDeletionPolicy(conf.getIndexDeletionPolicy()));
    IndexWriter indexWriter = new IndexWriter(indexDir, conf);
    
    Directory taxoDir = newDirectory();
    SnapshotDirectoryTaxonomyWriter taxoWriter = new SnapshotDirectoryTaxonomyWriter(taxoDir);
    try {

View Full Code Here

Examples of org.apache.lucene.index.SnapshotDeletionPolicy

  
  @Test
  public void testRevisionRelease() throws Exception {
    Directory indexDir = newDirectory();
    IndexWriterConfig conf = new IndexWriterConfig(TEST_VERSION_CURRENT, null);
    conf.setIndexDeletionPolicy(new SnapshotDeletionPolicy(conf.getIndexDeletionPolicy()));
    IndexWriter indexWriter = new IndexWriter(indexDir, conf);
    
    Directory taxoDir = newDirectory();
    SnapshotDirectoryTaxonomyWriter taxoWriter = new SnapshotDirectoryTaxonomyWriter(taxoDir);
    try {

View Full Code Here

Examples of org.apache.lucene.index.SnapshotDeletionPolicy

  
  @Test
  public void testSegmentsFileLast() throws Exception {
    Directory indexDir = newDirectory();
    IndexWriterConfig conf = new IndexWriterConfig(TEST_VERSION_CURRENT, null);
    conf.setIndexDeletionPolicy(new SnapshotDeletionPolicy(conf.getIndexDeletionPolicy()));
    IndexWriter indexWriter = new IndexWriter(indexDir, conf);
    
    Directory taxoDir = newDirectory();
    SnapshotDirectoryTaxonomyWriter taxoWriter = new SnapshotDirectoryTaxonomyWriter(taxoDir);
    try {

View Full Code Here

Examples of org.apache.lucene.index.SnapshotDeletionPolicy

  
  @Test
  public void testOpen() throws Exception {
    Directory indexDir = newDirectory();
    IndexWriterConfig conf = new IndexWriterConfig(TEST_VERSION_CURRENT, null);
    conf.setIndexDeletionPolicy(new SnapshotDeletionPolicy(conf.getIndexDeletionPolicy()));
    IndexWriter indexWriter = new IndexWriter(indexDir, conf);
    
    Directory taxoDir = newDirectory();
    SnapshotDirectoryTaxonomyWriter taxoWriter = new SnapshotDirectoryTaxonomyWriter(taxoDir);
    try {

View Full Code Here

Examples of org.elasticsearch.index.deletionpolicy.SnapshotDeletionPolicy

//        Store store = new NioFsStore(shardId, settings);


        store.deleteContent();


        ThreadPool threadPool = new ThreadPool();
        SnapshotDeletionPolicy deletionPolicy = new SnapshotDeletionPolicy(new KeepOnlyLastDeletionPolicy(shardId, settings));
        Engine engine = new RobinEngine(shardId, settings, new ThreadPool(), new IndexSettingsService(shardId.index(), settings), store, deletionPolicy, new FsTranslog(shardId, EMPTY_SETTINGS, new File("work/fs-translog")), new LogByteSizeMergePolicyProvider(store, new IndexSettingsService(shardId.index(), EMPTY_SETTINGS)),
                new ConcurrentMergeSchedulerProvider(shardId, settings), new AnalysisService(shardId.index()), new SimilarityService(shardId.index()), new NoneBloomCache(shardId.index()));
        engine.start();


        SimpleEngineBenchmark benchmark = new SimpleEngineBenchmark(store, engine)

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.