Examples of HyperLogLogPlus

com.clearspring.analytics.stream.cardinality.HyperLogLogPlus
c.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/40671.pdf

Brief HyperLogLog++ Overview
Uses 64 bit hashing instead of 32 Has two representation modes: sparse and normal
'normal' is approximately the same as regular hyperloglog (still uses 64 bits)
'sparse' handles lower cardinality values with a highly accurate but poorly scaling strategy and leverages data compression to compete with 'normal' for as long as possible (sparse has the advantage on accuracy per unit of memory at low cardinality but quickly falls behind).

Examples of com.clearspring.analytics.stream.cardinality.HyperLogLogPlus

    
    Object result = udafEvaluator.terminate(agg);
    Assert.assertNotNull(result);
    
    byte[] b = ((JavaBinaryObjectInspector) finalOutputOi).getPrimitiveJavaObject(result);
    HyperLogLogPlus hll = HyperLogLogPlus.Builder.build( b );
    Long cardEst = hll.cardinality();
    
    LOG.info("cardEst = " + cardEst);
    
    int actualUniques = h.keySet().size();
    LOG.info("actualUniques = " + actualUniques);

View Full Code Here

Examples of com.clearspring.analytics.stream.cardinality.HyperLogLogPlus

    
      byte[] bref1 = this.binary1Inspector.getPrimitiveJavaObject(blobObj1);
      byte[] bref2 = this.binary2Inspector.getPrimitiveJavaObject(blobObj2);
    
      if(bref1 != null && bref2 != null) {
        HyperLogLogPlus hll1 = HyperLogLogPlus.Builder.build( bref1 );
        HyperLogLogPlus hll2 = HyperLogLogPlus.Builder.build( bref2 );
      
        ICardinality merged = hll1.merge(hll2);
      return merged.getBytes();
      } else {
        return null;

View Full Code Here

Examples of com.clearspring.analytics.stream.cardinality.HyperLogLogPlus

    return precision != 0;
  }


  public void init(int precision) {
    this.precision = precision;
    hll = new HyperLogLogPlus(precision);
  }

View Full Code Here

Examples of com.clearspring.analytics.stream.cardinality.HyperLogLogPlus

      ///HyperLogLogPlus hll = HyperLogLogPlus.Builder.build( bref.getData() );
      
      byte[] bref = this.binaryInspector.getPrimitiveJavaObject(blobObj);
      if(bref == null)
        return null;
      HyperLogLogPlus hll = HyperLogLogPlus.Builder.build( bref );
      
      
      return hll.cardinality();
    } catch(Exception e) {
      LOG.error("Error", e);
      throw new HiveException(e);
    }

View Full Code Here

Examples of com.clearspring.analytics.stream.cardinality.HyperLogLogPlus

    this.outputOption = outputOption;
  }


  @Override
  public ICardinality initialize() {
    return new HyperLogLogPlus(precision);
  }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.