List of forceMerge() Examples

Examples of forceMerge()

org.apache.lucene.index.BlurIndexWriter.forceMerge()
org.apache.lucene.index.IndexWriter.forceMerge()
Forces merge policy to merge segments until there's <= maxNumSegments. The actual merges to be executed are determined by the {@link MergePolicy}.
This is a horribly costly operation, especially when you pass a small {@code maxNumSegments}; usually you should only call this if the index is static (will no longer be changed).

Note that this requires up to 2X the index size free space in your Directory (3X if you're using compound file format). For example, if your index size is 10 MB then you need up to 20 MB free for this to complete (30 MB if you're using compound file format). Also, it's best to call {@link #commit()} afterwards,to allow IndexWriter to free up disk space.

If some but not all readers re-open while merging is underway, this will cause > 2X temporary space to be consumed as those new readers will then hold open the temporary segments at that time. It is best not to re-open readers while merging is running.

The actual temporary usage could be much less than these figures (it depends on many factors).

In general, once the this completes, the total size of the index will be less than the size of the starting index. It could be quite a bit smaller (if there were many pending deletes) or just slightly smaller.

If an Exception is hit, for example due to disk full, the index will not be corrupt and no documents will have been lost. However, it may have been partially merged (some segments were merged but not all), and it's possible that one of the segments in the index will be in non-compound format even when using compound file format. This will occur when the Exception is hit during conversion of the segment into compound format.

This call will merge those segments present in the index when the call started. If other threads are still adding documents and flushing segments, those newly created segments will not be merged unless you call forceMerge again.

NOTE: if this method hits an OutOfMemoryError you should immediately close the writer. See above for details.

NOTE: if you call {@link #close(boolean)}with false, which aborts all running merges, then any thread still running this method might hit a {@link MergePolicy.MergeAbortedException}. @throws CorruptIndexException if the index is corrupt @throws IOException if there is a low-level IO error @see MergePolicy#findMerges @param maxNumSegments maximum number of segments leftin the index after merging finishes
org.apache.lucene.index.RandomIndexWriter.forceMerge()
Forces a forceMerge.
NOTE: this should be avoided in tests unless absolutely necessary, as it will result in less test coverage. @see IndexWriter#forceMerge(int)

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

    * byte-shuffling exercise to test.
    */
   public static void optimizeIndex(Directory dir) throws IOException {
      IndexWriterConfig indexWriterConfig = new IndexWriterConfig(LuceneSettings.LUCENE_VERSION, LuceneSettings.analyzer);
      IndexWriter iw = new IndexWriter(dir, indexWriterConfig);
      iw.forceMerge(1, true);
      iw.close();
   }


}

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

    * byte-shuffling exercise to test.
    */
   public static void optimizeIndex(Directory dir) throws IOException {
      IndexWriterConfig indexWriterConfig = new IndexWriterConfig(LuceneSettings.LUCENE_VERSION, LuceneSettings.analyzer);
      IndexWriter iw = new IndexWriter(dir, indexWriterConfig);
      iw.forceMerge(1, true);
      iw.close();
   }
   
   /**
    * Useful tool to debug the Lucene invocations into the directory;

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

        }
      } finally {
        releaseSearcher(indexSearcher);
      }
      if (fullMerge) {
        writer.forceMerge(1);
      }
      // close writer
      writer.close();
      // TODO: this isn't that great, maybe in the future SpellChecker should take
      // IWC in its ctor / keep its writer open?

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

    assertEquals(1, reader.getParallelTaxonomyArrays().parents().length);


    // add category and call forceMerge -- this should flush IW and merge segments down to 1
    // in ParentArray.initFromReader, this used to fail assuming there are no parents.
    writer.addCategory(new FacetLabel("1"));
    iw.forceMerge(1);
    
    // now calling openIfChanged should trip on the bug
    TaxonomyReader newtr = TaxonomyReader.openIfChanged(reader);
    assertNotNull(newtr);
    reader.close();

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

    TaxonomyReader reader = new DirectoryTaxonomyReader(writer);
    assertEquals(2, reader.getSize());
    assertEquals(2, reader.getParallelTaxonomyArrays().parents().length);


    // merge all the segments so that NRT reader thinks there's a change 
    iw.forceMerge(1);
    
    // now calling openIfChanged should trip on the wrong assert in ParetArray's ctor
    TaxonomyReader newtr = TaxonomyReader.openIfChanged(reader);
    assertNotNull(newtr);
    reader.close();

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

    int size = 100;
    IndexReader[] readers = new IndexReader[size];
    for (int i = 0; i < size; i++) {
      writer.addDocument(getDoc());
      readers[i] = DirectoryReader.open(writer, true);
      writer.forceMerge(1);
    }


    try {
      for (int i = 0; i < size; i++) {
        checkReader(readers[i], i);

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

    IndexWriter writer = new IndexWriter(counter, conf);
    int size = 100;
    IndexReader[] readers = new IndexReader[size];
    for (int i = 0; i < size; i++) {
      writer.addDocument(getDoc());
      writer.forceMerge(1);
      readers[i] = DirectoryReader.open(writer, true);
    }


    for (int i = 0; i < size; i++) {
      assertEquals(i + 1, readers[i].numDocs());

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

        }


        for (GenericValue gv : contentList) {
            indexContent(dispatcher, delegator, context, gv, writer);
        }
        writer.forceMerge(1);
        writer.close();
    }


    public static void deleteContentDocument(GenericValue content, String path) throws Exception {
        Directory directory = FSDirectory.open(new File(getIndexPath(path)));

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

            }
            if (Debug.infoOn()) Debug.logInfo("Created new directory:" + directory.toString(), module);
        }


        indexContent(dispatcher, delegator, context, content, writer);
        writer.forceMerge(1);
        writer.close();
    }


    public static void indexContent(LocalDispatcher dispatcher, Delegator delegator, Map<String, Object> context, GenericValue content, IndexWriter writer) throws Exception {
        Document doc = ContentDocument.Document(content, context, dispatcher);

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.forceMerge()

            writer  = new IndexWriter(directory, conf);
        } finally {
            IndexWriterConfig.setDefaultWriteLockTimeout(savedWriteLockTimeout);
        }
        indexDataResource(delegator, context, id, writer);
        writer.forceMerge(1);
        writer.close();
    }


    public static void indexDataResource(Delegator delegator, Map<String, Object> context, String id, IndexWriter writer) throws Exception {
        Document doc = DataResourceDocument.Document(id, delegator, context);

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.