Examples of Deduper | massapi.com

  private ChunkIndex deduplicateAndCreateChunkIndex(final List<File> inputFiles, FrameworkCombination combination) throws IOException {
    logger.log(Level.INFO, "- Deduplicate and create chunk index ...");


    final ChunkIndex chunkIndex = new ChunkIndex();
    
    Deduper deduper = new Deduper(combination.chunker, combination.multiChunker, combination.transformer);
    deduper.deduplicate(inputFiles, new DeduperListener() {      
      @Override
      public void onMultiChunkWrite(MultiChunk multiChunk, Chunk chunk) {
        logger.log(Level.INFO, "    - Adding chunk "+StringUtil.toHex(chunk.getChecksum())+" to multichunk "+multiChunk.getId()+" ...");
        chunkIndex.chunkIDToMultiChunkID.put(new ChunkChecksum(chunk.getChecksum()), multiChunk.getId());        
      }

Examples of org.syncany.chunk.Deduper


    // New vector clock
    VectorClock newVectorClock = findNewVectorClock(lastVectorClock);


    // Index
    Deduper deduper = new Deduper(config.getChunker(), config.getMultiChunker(), config.getTransformer());
    Indexer indexer = new Indexer(config, deduper);


    DatabaseVersion newDatabaseVersion = indexer.index(localFiles);


    newDatabaseVersion.setVectorClock(newVectorClock);

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.