Examples of org.apache.lucene.benchmark.byTask.feeds.ContentSource

org.apache.lucene.benchmark.byTask.feeds.ContentSource

Represents content from a specified source, such as TREC, Reuters etc. A {@link ContentSource} is responsible for creating {@link DocData} objects forits documents to be consumed by {@link DocMaker}. It also keeps track of various statistics, such as how many documents were generated, size in bytes etc.

Supports the following configuration parameters:

content.source.forever - specifies whether to generate documents forever (default=true).
content.source.verbose - specifies whether messages should be output by the content source (default=false).
content.source.encoding - specifies which encoding to use when reading the files of that content source. Certain implementations may define a default value if this parameter is not specified. (default=null).
content.source.log.step - specifies for how many documents a message should be logged. If set to 0 it means no logging should occur. NOTE: if verbose is set to false, logging should not occur even if logStep is not 0 (default=0).

    properties.setProperty("docs.file", wikipedia.getAbsolutePath());
    properties.setProperty("content.source.forever", "false");
    properties.setProperty("keep.image.only.docs", String.valueOf(keepImageOnlyDocs));
    Config config = new Config(properties);


    ContentSource source = new EnwikiContentSource();
    source.setConfig(config);
    
    DocMaker docMaker = new DocMaker();
    docMaker.setConfig(config, source);
    docMaker.resetInputs();
    if (wikipedia.exists()) {

    properties.setProperty("docs.file", wikipedia.getAbsolutePath());
    properties.setProperty("content.source.forever", "false");
    properties.setProperty("keep.image.only.docs", String.valueOf(keepImageOnlyDocs));
    Config config = new Config(properties);


    ContentSource source = new EnwikiContentSource();
    source.setConfig(config);
    
    DocMaker docMaker = new DocMaker();
    docMaker.setConfig(config, source);
    docMaker.resetInputs();
    if (wikipedia.exists()) {

Examples of org.apache.lucene.benchmark.byTask.feeds.ContentSource

Related Classes of org.apache.lucene.benchmark.byTask.feeds.ContentSource