List of addIndexes() Examples

Examples of addIndexes()

com.basho.riak.pbc.RPB.RpbContent.Builder.addIndexes()
com.basho.riak.protobuf.RiakKvPB.RpbContent.Builder.addIndexes()
org.apache.cocoon.components.search.components.IndexManager.addIndexes()
Adds indexes from the given configuration file to the index manager. @param confSource @throws ConfigurationException
org.apache.lucene.index.IndexWriter.addIndexes()
Adds all segments from an array of indexes into this index.
This may be used to parallelize batch indexing. A large document collection can be broken into sub-collections. Each sub-collection can be indexed in parallel, on a different thread, process or machine. The complete index can then be created by merging sub-collection indexes with this method.
NOTE: the index in each {@link Directory} must not bechanged (opened by a writer) while this method is running. This method does not acquire a write lock in each input Directory, so it is up to the caller to enforce this.
This method is transactional in how Exceptions are handled: it does not commit a new segments_N file until all indexes are added. This means if an Exception occurs (for example disk full), then either no indexes will have been added or they all will have been.
Note that this requires temporary free space in the {@link Directory} up to 2X the sum of all input indexes(including the starting index). If readers/searchers are open against the starting index, then temporary free space required will be higher by the size of the starting index (see {@link #forceMerge(int)} for details).
NOTE: this method only copies the segments of the incomning indexes and does not merge them. Therefore deleted documents are not removed and the new segments are not merged with the existing ones. Also, if the merge policy allows compound files, then any segment that is not compound is converted to such. However, if the segment is compound, it is copied as-is even if the merge policy does not allow compound files.

This requires this index not be among those to be added.
NOTE: if this method hits an OutOfMemoryError you should immediately close the writer. See above for details. @throws CorruptIndexException if the index is corrupt @throws IOException if there is a low-level IO error
org.apache.lucene.index.RandomIndexWriter.addIndexes()
org.hibernate.cfg.annotations.EntityBinder.addIndexes()
org.springmodules.lucene.index.factory.LuceneIndexWriter.addIndexes()
@see IndexWriter#addIndexes(Directory[]) @throws IOException

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

    writer.setMaxMergeDocs(conf.getInt("indexer.maxMergeDocs", IndexWriter.DEFAULT_MAX_MERGE_DOCS));
    writer.setTermIndexInterval(conf.getInt("indexer.termIndexInterval", IndexWriter.DEFAULT_TERM_INDEX_INTERVAL));
    writer.setInfoStream(LogUtil.getDebugStream(LOG));
    writer.setUseCompoundFile(false);
    writer.setSimilarity(new NutchSimilarity());
    writer.addIndexes(dirs);
    writer.close();


    //
    // Put target back
    //

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

            iw.setMaxBufferedDocs(1000);
      IndexReader readers[] = new IndexReader[args.length -1];
      for (int i = 0; i < args.length - 1; i++) {
        readers[i] = IndexReader.open(args[i+1]);
      }
      iw.addIndexes(readers);
      iw.optimize();
      iw.close();
    }
    catch (IOException e) {
      System.err.println(e);

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

  private void copyAndOptimizeInFlightDir() throws IOException {
    CopyRateDirectory copyRateDirectory = new CopyRateDirectory(_finalDir, _copyRateCounter);
    copyRateDirectory.setLockFactory(NoLockFactory.getNoLockFactory());
    DirectoryReader reader = DirectoryReader.open(_localDir);
    IndexWriter writer = new IndexWriter(copyRateDirectory, _conf.clone());
    writer.addIndexes(reader);
    writer.close();
    rm(_localPath);
  }


  private void copyDir() throws IOException {

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

    writer1.addDocument(getDoc());
    writer1.close();


    HdfsDirectory dir2 = new HdfsDirectory(_configuration, new Path(_base, "dir2"));
    IndexWriter writer2 = new IndexWriter(dir2, conf.clone());
    writer2.addIndexes(dir1);
    writer2.close();


    DirectoryReader reader1 = DirectoryReader.open(dir1);
    DirectoryReader reader2 = DirectoryReader.open(dir2);

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

      if (masters.size() > 1) {
        LOG.info(" - merging subindexes...");
        stage = SegmentMergeStatus.STAGE_MERGEIDX;
        IndexReader[] ireaders = new IndexReader[masters.size() - 1];
        for (int i = 0; i < masters.size() - 1; i++) ireaders[i] = IndexReader.open((File)masters.get(i));
        iw.addIndexes(ireaders);
        for (int i = 0; i < masters.size() - 1; i++) {
          ireaders[i].close();
          FileUtil.fullyDelete((File)masters.get(i));
        }
      }

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

  public void optimize() throws IOException {
    IndexReader reader = IndexReader.open(new File(directory, "index"));
    OptimizingReader optimizer = new OptimizingReader(reader);
    IndexWriter writer = new IndexWriter(new File(directory, "index-opt"),
                                         null, true);
    writer.addIndexes(new IndexReader[] { optimizer });
  }


  /** */
  public static void main(String[] args) throws Exception {
    File directory;

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

    writer.maxMergeDocs = MAX_MERGE_DOCS;
    writer.setTermIndexInterval(TERM_INDEX_INTERVAL);
    writer.infoStream = LogFormatter.getLogStream(LOG, Level.FINE);
    writer.setUseCompoundFile(false);
    writer.setSimilarity(new NutchSimilarity());
    writer.addIndexes(dirs);
    writer.close();


    //
    // Put target back
    //

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

        Path pathToMerge = paths[i + p * newShardCount];
        System.out.println("Merge [" + pathToMerge + "]");
        dirs[p - 1] = new HdfsDirectory(getConf(), pathToMerge);
        pathsToDelete[p - 1] = pathToMerge;
      }
      indexWriter.addIndexes(dirs);
      // Causes rewrite of of index and the symlinked files are
      // merged/rewritten.
      indexWriter.forceMerge(1);
      indexWriter.close();
      FileSystem fileSystem = path.getFileSystem(getConf());

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

  private void copyAndOptimizeInFlightDir() throws IOException {
    CopyRateDirectory copyRateDirectory = new CopyRateDirectory(_finalDir, _copyRateCounter);
    copyRateDirectory.setLockFactory(NoLockFactory.getNoLockFactory());
    DirectoryReader reader = DirectoryReader.open(_localDir);
    IndexWriter writer = new IndexWriter(copyRateDirectory, _conf.clone());
    writer.addIndexes(reader);
    writer.close();
    rm(_localPath);
  }


  private void copyDir() throws IOException {

View Full Code Here

Examples of org.apache.lucene.index.IndexWriter.addIndexes()

    IndexWriter writer = new IndexWriter(localOutput, null, true);
    writer.mergeFactor = 50;
    writer.infoStream = LogFormatter.getLogStream(LOG, Level.INFO);
    writer.setUseCompoundFile(false);
    writer.setSimilarity(new NutchSimilarity());
    writer.addIndexes(dirs);
    writer.close();


    //
    // Put target back
    //

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.