List of readCollectionName() Examples

Examples of readCollectionName()

ivory.core.RetrievalEnvironment.readCollectionName()

Examples of ivory.core.RetrievalEnvironment.readCollectionName()


    FileSystem fs = FileSystem.get(getConf());


    RetrievalEnvironment re = new RetrievalEnvironment(indexPath, fs);


    String collectionName = re.readCollectionName();
    int numDocs = re.readCollectionDocumentCount();
    Path docLengthPath = re.getDoclengthsData();
    String scoringModel = getConf().get("Ivory.ScoringModel");


    sLogger.info("Characteristics of the collection:");

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()

    String indexPath = conf.get(Constants.IndexPath);
    RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);


    int reduceTasks = 10;


    String collectionName = env.readCollectionName();
    String termDocVectorsPath = env.getTermDocVectorsDirectory();
    String termDfCfPath = env.getTermDfCfDirectory();


    if (!fs.exists(new Path(indexPath))) {
      LOG.info("index path doesn't existing: skipping!");

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()

    Configuration conf = getConf();
    FileSystem fs = FileSystem.get(conf);


    String indexPath = conf.get(Constants.IndexPath);
    RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);
    String collectionName = env.readCollectionName();


    LOG.info("Tool: " + BuildTermDocVectorsForwardIndex.class.getCanonicalName());
    LOG.info(String.format(" - %s: %s", Constants.CollectionName, collectionName));
    LOG.info(String.format(" - %s: %s", Constants.IndexPath, indexPath));

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()

    Configuration conf = getConf();
    FileSystem fs = FileSystem.get(conf);


    String indexPath = conf.get(Constants.IndexPath);
    RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);
    String collectionName = env.readCollectionName();


    LOG.info("PowerTool: " + BuildIntDocVectors.class.getCanonicalName());
    LOG.info(String.format(" - %s: %s", Constants.CollectionName, collectionName));
    LOG.info(String.format(" - %s: %s", Constants.IndexPath, indexPath));

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()

    Configuration conf = getConf();
    FileSystem fs = FileSystem.get(conf);


    String indexPath = conf.get(Constants.IndexPath);
    RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);
    String collectionName = env.readCollectionName();


    LOG.info("Tool: " + BuildIntDocVectorsForwardIndex.class.getCanonicalName());
    LOG.info(String.format(" - %s: %s", Constants.CollectionName, collectionName));
    LOG.info(String.format(" - %s: %s", Constants.IndexPath, indexPath));

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()


    int minSplitSize = conf.getInt(Constants.MinSplitSize, 0);
    String indexPath = conf.get(Constants.IndexPath);


    RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);
    String collectionName = env.readCollectionName();


    LOG.info("Tool: " + BuildIntPostingsForwardIndex.class.getCanonicalName());
    LOG.info(String.format(" - %s: %s", Constants.IndexPath, indexPath));
    LOG.info(String.format(" - %s: %s", Constants.CollectionName, collectionName));

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()

    FileSystem fs = FileSystem.get(conf);


    String indexPath = conf.get(Constants.IndexPath);
    RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);


    String collectionName = env.readCollectionName();


    int reduceTasks = conf.getInt(Constants.NumReduceTasks, 0);
    int minSplitSize = conf.getInt(Constants.MinSplitSize, 0);
    int collectionDocCnt = env.readCollectionDocumentCount();

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()


    String indexPath = conf.get("Ivory.IndexPath");
    String statsPath = conf.get("Ivory.GlobalStatsPath");


    RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);
    String collectionName = env.readCollectionName();


    sLogger.info(" - CollectionName: " + collectionName);
    sLogger.info(" - IndexPath: " + indexPath);
    sLogger.info(" - NumMapTasks: " + mapTasks);
    sLogger.info(" - NumReduceTasks: " + reduceTasks);

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()

    String indexPath = conf.get(Constants.IndexPath);
    RetrievalEnvironment env = new RetrievalEnvironment(indexPath, fs);


    int reduceTasks = 10;


    String collectionName = env.readCollectionName();
    String termDocVectorsPath = env.getTermDocVectorsDirectory();
    String termDfCfPath = env.getTermDfCfDirectory();


    if (!fs.exists(new Path(indexPath))) {
      LOG.info("index path doesn't existing: skipping!");

View Full Code Here

Examples of ivory.core.RetrievalEnvironment.readCollectionName()

    }
    JobConf job = new JobConf(conf, ComputeSignaturesRandom.class);
    FileSystem fs = FileSystem.get(job);


    RetrievalEnvironment re = new RetrievalEnvironment(dir, fs);
    job.setJobName("ComputeSignatures_random_D="+D+":"+re.readCollectionName());


    String inputPath = PwsimEnvironment.getFileNameWithPars(dir, "IntDocs");
    String outputPath = PwsimEnvironment.getFileNameWithPars(dir, "SignaturesRandom");
    String randomVectorFile = PwsimEnvironment.getFileNameWithPars(dir, "RandomVectors")+"/part-00000";
    DistributedCache.addCacheFile(new URI(randomVectorFile), job);

View Full Code Here

0 1 2 3 4

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.