Examples of DynamicBloomFilter

org.apache.accumulo.core.bloomfilter.DynamicBloomFilter
ne-lab.org">European Commission One-Lab Project 034819. @see Filter The general behavior of a filter @see BloomFilter A Bloom filter @see Theory and Network Applications of Dynamic Bloom Filters
org.apache.hadoop.util.bloom.DynamicBloomFilter
ne-lab.org">European Commission One-Lab Project 034819. @see Filter The general behavior of a filter @see BloomFilter A Bloom filter @see Theory and Network Applications of Dynamic Bloom Filters

Examples of org.apache.accumulo.core.bloomfilter.DynamicBloomFilter

      // <code>n</code> is the number of keys and <code>c</code> is the desired
      // max. error rate.
      // Our desired error rate is by default 0.005, i.e. 0.5%
      double errorRate = acuconf.getFraction(Property.TABLE_BLOOM_ERRORRATE);
      vectorSize = (int) Math.ceil(-HASH_COUNT * numKeys / Math.log(1.0 - Math.pow(errorRate, 1.0 / HASH_COUNT)));
      bloomFilter = new DynamicBloomFilter(vectorSize, HASH_COUNT, Hash.parseHashType(acuconf.get(Property.TABLE_BLOOM_HASHTYPE)), numKeys);
      
      /**
       * load KeyFunctor
       */
      try {

View Full Code Here

Examples of org.apache.accumulo.core.bloomfilter.DynamicBloomFilter

          String ClassName = null;
          DataInputStream in = null;
          
          try {
            in = reader.getMetaStore(BLOOM_FILE_NAME);
            DynamicBloomFilter tmpBloomFilter = new DynamicBloomFilter();
            
            // check for closed again after open but before reading the bloom filter in
            if (closed)
              return;
            
            /**
             * Load classname for keyFunctor
             */
            ClassName = in.readUTF();
            
            Class<? extends KeyFunctor> clazz = AccumuloVFSClassLoader.loadClass(ClassName, KeyFunctor.class);
            transformer = clazz.newInstance();
            
            /**
             * read in bloom filter
             */
            
            tmpBloomFilter.readFields(in);
            // only set the bloom filter after it is fully constructed
            bloomFilter = tmpBloomFilter;
          } catch (NoSuchMetaStoreException nsme) {
            // file does not have a bloom filter, ignore it
          } catch (IOException ioe) {

View Full Code Here

Examples of org.apache.accumulo.core.bloomfilter.DynamicBloomFilter

      // <code>n</code> is the number of keys and <code>c</code> is the desired
      // max. error rate.
      // Our desired error rate is by default 0.005, i.e. 0.5%
      double errorRate = acuconf.getFraction(Property.TABLE_BLOOM_ERRORRATE);
      vectorSize = (int) Math.ceil(-HASH_COUNT * numKeys / Math.log(1.0 - Math.pow(errorRate, 1.0 / HASH_COUNT)));
      bloomFilter = new DynamicBloomFilter(vectorSize, HASH_COUNT, Hash.parseHashType(acuconf.get(Property.TABLE_BLOOM_HASHTYPE)), numKeys);
      
      /**
       * load KeyFunctor
       */
      try {

View Full Code Here

Examples of org.apache.accumulo.core.bloomfilter.DynamicBloomFilter

          String ClassName = null;
          DataInputStream in = null;
          
          try {
            in = reader.getMetaStore(BLOOM_FILE_NAME);
            DynamicBloomFilter tmpBloomFilter = new DynamicBloomFilter();
            
            // check for closed again after open but before reading the bloom filter in
            if (closed)
              return;
            
            /**
             * Load classname for keyFunctor
             */
            ClassName = in.readUTF();
            
            Class<? extends KeyFunctor> clazz = AccumuloVFSClassLoader.loadClass(ClassName, KeyFunctor.class);
            transformer = clazz.newInstance();
            
            /**
             * read in bloom filter
             */
            
            tmpBloomFilter.readFields(in);
            // only set the bloom filter after it is fully constructed
            bloomFilter = tmpBloomFilter;
          } catch (NoSuchMetaStoreException nsme) {
            // file does not have a bloom filter, ignore it
          } catch (IOException ioe) {

View Full Code Here

Examples of org.apache.accumulo.core.bloomfilter.DynamicBloomFilter

      // <code>n</code> is the number of keys and <code>c</code> is the desired
      // max. error rate.
      // Our desired error rate is by default 0.005, i.e. 0.5%
      double errorRate = acuconf.getFraction(Property.TABLE_BLOOM_ERRORRATE);
      vectorSize = (int) Math.ceil(-HASH_COUNT * numKeys / Math.log(1.0 - Math.pow(errorRate, 1.0 / HASH_COUNT)));
      bloomFilter = new DynamicBloomFilter(vectorSize, HASH_COUNT, Hash.parseHashType(acuconf.get(Property.TABLE_BLOOM_HASHTYPE)), numKeys);
      
      /**
       * load KeyFunctor
       */
      try {

View Full Code Here

Examples of org.apache.accumulo.core.bloomfilter.DynamicBloomFilter

          String ClassName = null;
          DataInputStream in = null;
          
          try {
            in = reader.getMetaStore(BLOOM_FILE_NAME);
            DynamicBloomFilter tmpBloomFilter = new DynamicBloomFilter();
            
            // check for closed again after open but before reading the bloom filter in
            if (closed)
              return;
            
            /**
             * Load classname for keyFunctor
             */
            ClassName = in.readUTF();
            
            Class<? extends KeyFunctor> clazz = AccumuloClassLoader.loadClass(ClassName, KeyFunctor.class);
            transformer = clazz.newInstance();
            
            /**
             * read in bloom filter
             */
            
            tmpBloomFilter.readFields(in);
            // only set the bloom filter after it is fully constructed
            bloomFilter = tmpBloomFilter;
          } catch (NoSuchMetaStoreException nsme) {
            // file does not have a bloom filter, ignore it
          } catch (IOException ioe) {

View Full Code Here

Examples of org.apache.hadoop.util.bloom.DynamicBloomFilter

      // max. error rate.
      // Our desired error rate is by default 0.005, i.e. 0.5%
      float errorRate = conf.getFloat("io.mapfile.bloom.error.rate", 0.005f);
      vectorSize = (int)Math.ceil((double)(-HASH_COUNT * numKeys) /
          Math.log(1.0 - Math.pow(errorRate, 1.0/HASH_COUNT)));
      bloomFilter = new DynamicBloomFilter(vectorSize, HASH_COUNT,
          Hash.getHashType(conf), numKeys);
    }

View Full Code Here

Examples of org.apache.hadoop.util.bloom.DynamicBloomFilter

    
    private void initBloomFilter(FileSystem fs, String dirName,
        Configuration conf) {
      try {
        DataInputStream in = fs.open(new Path(dirName, BLOOM_FILE_NAME));
        bloomFilter = new DynamicBloomFilter();
        bloomFilter.readFields(in);
        in.close();
      } catch (IOException ioe) {
        LOG.warn("Can't open BloomFilter: " + ioe + " - fallback to MapFile.");
        bloomFilter = null;

View Full Code Here

Examples of org.apache.hadoop.util.bloom.DynamicBloomFilter

      // max. error rate.
      // Our desired error rate is by default 0.005, i.e. 0.5%
      float errorRate = conf.getFloat("io.mapfile.bloom.error.rate", 0.005f);
      vectorSize = (int)Math.ceil((double)(-HASH_COUNT * numKeys) /
          Math.log(1.0 - Math.pow(errorRate, 1.0/HASH_COUNT)));
      bloomFilter = new DynamicBloomFilter(vectorSize, HASH_COUNT,
          Hash.getHashType(conf), numKeys);
    }

View Full Code Here

Examples of org.apache.hadoop.util.bloom.DynamicBloomFilter

    
    private void initBloomFilter(FileSystem fs, String dirName,
        Configuration conf) {
      try {
        DataInputStream in = fs.open(new Path(dirName, BLOOM_FILE_NAME));
        bloomFilter = new DynamicBloomFilter();
        bloomFilter.readFields(in);
        in.close();
      } catch (IOException ioe) {
        LOG.warn("Can't open BloomFilter: " + ioe + " - fallback to MapFile.");
        bloomFilter = null;

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.