Examples of TaxonomyWriter

org.apache.lucene.facet.taxonomy.TaxonomyWriter
TaxonomyWriter is the interface which the faceted-search library uses to dynamically build the taxonomy at indexing time.
Notes about concurrent access to the taxonomy:
An implementation must allow multiple readers and a single writer to be active concurrently. Readers follow so-called "point in time" semantics, i.e., a reader object will only see taxonomy entries which were available at the time it was created. What the writer writes is only available to (new) readers after the writer's commit() is called.
Faceted search keeps two indices - namely Lucene's main index, and this taxonomy index. When one or more readers are active concurrently with the writer, care must be taken to avoid an inconsistency between the state of these two indices: When writing to the indices, the taxonomy must always be committed to disk *before* the main index, because the main index refers to categories listed in the taxonomy. Such control can best be achieved by turning off the main index's "autocommit" feature, and explicitly calling commit() for both indices (first for the taxonomy, then for the main index). In old versions of Lucene (2.2 or earlier), when autocommit could not be turned off, a more complicated solution needs to be used. E.g., use some sort of (possibly inter-process) locking to ensure that a reader is being opened only right after both indices have been flushed (and before anything else is written to them). @lucene.experimental

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

  @Test
  public void testRollupValues() throws Exception {
    Directory indexDir = newDirectory();
    Directory taxoDir = newDirectory();


    TaxonomyWriter taxonomyWriter = new DirectoryTaxonomyWriter(taxoDir);
    IndexWriter iw = new IndexWriter(indexDir, newIndexWriterConfig(TEST_VERSION_CURRENT, new MockAnalyzer(random())));
    FacetIndexingParams fip = new FacetIndexingParams(new CategoryListParams() {
      @Override
      public OrdinalPolicy getOrdinalPolicy(String dimension) {
        return OrdinalPolicy.NO_PARENTS;
      }
    });
    FacetFields facetFields = new FacetFields(taxonomyWriter, fip);
    for (int i = 0; i < 4; i++) {
      Document doc = new Document();
      doc.add(new NumericDocValuesField("price", (i+1)));
      facetFields.addFields(doc, Collections.singletonList(new CategoryPath("a", Integer.toString(i % 2), "1")));
      iw.addDocument(doc);
    }
    
    taxonomyWriter.close();
    iw.close();
    
    DirectoryReader r = DirectoryReader.open(indexDir);
    DirectoryTaxonomyReader taxo = new DirectoryTaxonomyReader(taxoDir);

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

    doTestReadRecreatedTaxonomy(random(), false);
  }
  
  private void doTestReadRecreatedTaxonomy(Random random, boolean closeReader) throws Exception {
    Directory dir = null;
    TaxonomyWriter tw = null;
    TaxonomyReader tr = null;
    
    // prepare a few categories
    int  n = 10;
    CategoryPath[] cp = new CategoryPath[n];
    for (int i=0; i<n; i++) {
      cp[i] = new CategoryPath("a", Integer.toString(i));
    }
    
    try {
      dir = newDirectory();
      
      tw = new DirectoryTaxonomyWriter(dir);
      tw.addCategory(new CategoryPath("a"));
      tw.close();
      
      tr = new DirectoryTaxonomyReader(dir);
      int baseNumCategories = tr.getSize();
      
      for (int i=0; i<n; i++) {
        int k = random.nextInt(n);
        tw = new DirectoryTaxonomyWriter(dir, OpenMode.CREATE);
        for (int j = 0; j <= k; j++) {
          tw.addCategory(cp[j]);
        }
        tw.close();
        if (closeReader) {
          tr.close();
          tr = new DirectoryTaxonomyReader(dir);
        } else {
          TaxonomyReader newtr = TaxonomyReader.openIfChanged(tr);

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

      };


      RandomIndexWriter iw = new RandomIndexWriter(random(), iDir,
          newIndexWriterConfig(TEST_VERSION_CURRENT,
              new MockAnalyzer(random())).setOpenMode(OpenMode.CREATE));
      TaxonomyWriter tw = new DirectoryTaxonomyWriter(tDir);
      prvt_add(iParams, iw, tw, "a", "b");
      prvt_add(iParams, iw, tw, "a", "b", "1");
      prvt_add(iParams, iw, tw, "a", "b", "1");
      prvt_add(iParams, iw, tw, "a", "b", "2");
      prvt_add(iParams, iw, tw, "a", "b", "2");
      prvt_add(iParams, iw, tw, "a", "b", "2");
      prvt_add(iParams, iw, tw, "a", "b", "3");
      prvt_add(iParams, iw, tw, "a", "b", "4");
      prvt_add(iParams, iw, tw, "a", "c");
      prvt_add(iParams, iw, tw, "a", "c");
      prvt_add(iParams, iw, tw, "a", "c");
      prvt_add(iParams, iw, tw, "a", "c");
      prvt_add(iParams, iw, tw, "a", "c");
      prvt_add(iParams, iw, tw, "a", "c", "1");
      prvt_add(iParams, iw, tw, "a", "d");
      prvt_add(iParams, iw, tw, "a", "e");


      IndexReader ir = iw.getReader();
      iw.close();
      tw.commit();
      tw.close();


      IndexSearcher is = newSearcher(ir);
      DirectoryTaxonomyReader tr = new DirectoryTaxonomyReader(tDir);


      // Get all of the documents and run the query, then do different

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

    // 4. Segment w/ categories, but only some results
    
    IndexWriterConfig conf = newIndexWriterConfig(TEST_VERSION_CURRENT, new MockAnalyzer(random()));
    conf.setMergePolicy(NoMergePolicy.COMPOUND_FILES); // prevent merges, so we can control the index segments
    IndexWriter indexWriter = new IndexWriter(indexDir, conf);
    TaxonomyWriter taxoWriter = new DirectoryTaxonomyWriter(taxoDir);


    allExpectedCounts = newCounts();
    termExpectedCounts = newCounts();
    
    // segment w/ no categories

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

    doTestReadRecreatedTaxonomy(random(), false);
  }
  
  private void doTestReadRecreatedTaxonomy(Random random, boolean closeReader) throws Exception {
    Directory dir = null;
    TaxonomyWriter tw = null;
    TaxonomyReader tr = null;
    
    // prepare a few categories
    int  n = 10;
    FacetLabel[] cp = new FacetLabel[n];
    for (int i=0; i<n; i++) {
      cp[i] = new FacetLabel("a", Integer.toString(i));
    }
    
    try {
      dir = newDirectory();
      
      tw = new DirectoryTaxonomyWriter(dir);
      tw.addCategory(new FacetLabel("a"));
      tw.close();
      
      tr = new DirectoryTaxonomyReader(dir);
      int baseNumCategories = tr.getSize();
      
      for (int i=0; i<n; i++) {
        int k = random.nextInt(n);
        tw = new DirectoryTaxonomyWriter(dir, OpenMode.CREATE);
        for (int j = 0; j <= k; j++) {
          tw.addCategory(cp[j]);
        }
        tw.close();
        if (closeReader) {
          tr.close();
          tr = new DirectoryTaxonomyReader(dir);
        } else {
          TaxonomyReader newtr = TaxonomyReader.openIfChanged(tr);

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

    
    // create and open an index writer
    RandomIndexWriter iw = new RandomIndexWriter(random(), indexDir, newIndexWriterConfig(
        TEST_VERSION_CURRENT, new MockAnalyzer(random(), MockTokenizer.WHITESPACE, false)));
    // create and open a taxonomy writer
    TaxonomyWriter tw = new DirectoryTaxonomyWriter(taxoDir, OpenMode.CREATE);
    FacetsConfig config = getConfig();


    seedIndex(tw, iw, config);


    IndexReader ir = iw.getReader();
    tw.commit();


    // prepare index reader and taxonomy.
    TaxonomyReader tr = new DirectoryTaxonomyReader(taxoDir);


    // prepare searcher to search against

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

    
    // create and open an index writer
    RandomIndexWriter iw = new RandomIndexWriter(random(), indexDir, newIndexWriterConfig(
        TEST_VERSION_CURRENT, new MockAnalyzer(random(), MockTokenizer.WHITESPACE, false)));
    // create and open a taxonomy writer
    TaxonomyWriter tw = new DirectoryTaxonomyWriter(taxoDir, OpenMode.CREATE);


    FacetsConfig config = getConfig();
    config.setIndexFieldName("Author", "$author");
    seedIndex(tw, iw, config);


    IndexReader ir = iw.getReader();
    tw.commit();


    // prepare index reader and taxonomy.
    TaxonomyReader tr = new DirectoryTaxonomyReader(taxoDir);


    // prepare searcher to search against

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

    
    // create and open an index writer
    RandomIndexWriter iw = new RandomIndexWriter(random(), indexDir, newIndexWriterConfig(
        TEST_VERSION_CURRENT, new MockAnalyzer(random(), MockTokenizer.WHITESPACE, false)));
    // create and open a taxonomy writer
    TaxonomyWriter tw = new DirectoryTaxonomyWriter(taxoDir, OpenMode.CREATE);


    FacetsConfig config = getConfig();
    config.setIndexFieldName("Band", "$music");
    config.setIndexFieldName("Composer", "$music");
    seedIndex(tw, iw, config);


    IndexReader ir = iw.getReader();
    tw.commit();


    // prepare index reader and taxonomy.
    TaxonomyReader tr = new DirectoryTaxonomyReader(taxoDir);


    // prepare searcher to search against

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter


    // create and open an index writer
    RandomIndexWriter iw = new RandomIndexWriter(random(), indexDir, newIndexWriterConfig(
        TEST_VERSION_CURRENT, new MockAnalyzer(random(), MockTokenizer.WHITESPACE, false)));
    // create and open a taxonomy writer
    TaxonomyWriter tw = new DirectoryTaxonomyWriter(taxoDir, OpenMode.CREATE);


    FacetsConfig config = getConfig();
    config.setIndexFieldName("Band", "$bands");
    config.setIndexFieldName("Composer", "$composers");
    seedIndex(tw, iw, config);


    IndexReader ir = iw.getReader();
    tw.commit();


    // prepare index reader and taxonomy.
    TaxonomyReader tr = new DirectoryTaxonomyReader(taxoDir);


    // prepare searcher to search against

View Full Code Here

Examples of org.apache.lucene.facet.taxonomy.TaxonomyWriter

    
    // create and open an index writer
    RandomIndexWriter iw = new RandomIndexWriter(random(), indexDir, newIndexWriterConfig(
        TEST_VERSION_CURRENT, new MockAnalyzer(random(), MockTokenizer.WHITESPACE, false)));
    // create and open a taxonomy writer
    TaxonomyWriter tw = new DirectoryTaxonomyWriter(taxoDir, OpenMode.CREATE);


    FacetsConfig config = getConfig();
    config.setIndexFieldName("Band", "$music");
    config.setIndexFieldName("Composer", "$music");
    config.setIndexFieldName("Author", "$literature");
    seedIndex(tw, iw, config);


    IndexReader ir = iw.getReader();
    tw.commit();


    // prepare index reader and taxonomy.
    TaxonomyReader tr = new DirectoryTaxonomyReader(taxoDir);


    // prepare searcher to search against

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.