Examples of org.apache.hadoop.hive.common.type.Decimal128

Package org.apache.hadoop.hive.common.type

Examples of org.apache.hadoop.hive.common.type.Decimal128

org.apache.hadoop.hive.common.type.Decimal128
This code was originally written for Microsoft PolyBase.
A 128-bit fixed-length Decimal value in the ANSI SQL Numeric semantics, representing unscaledValue / 10**scale where scale is 0 or positive.

This class is similar to {@link java.math.BigDecimal}, but a few things differ to conform to the SQL Numeric semantics.

Scale of this object is specified by the user, not automatically determined like {@link java.math.BigDecimal}. This means that underflow is possible depending on the scale. {@link java.math.BigDecimal} controls roundingbehaviors by MathContext, possibly throwing errors. But, underflow is NOT an error in ANSI SQL Numeric. "CAST(0.000000000....0001 AS DECIMAL(38,1))" is "0.0" without an error.

Because this object is fixed-length, overflow is also possible. Overflow IS an error in ANSI SQL Numeric. "CAST(10000 AS DECIMAL(38,38))" throws overflow error.

Each arithmetic operator takes scale as a parameter to control its behavior. It's user's (or query optimizer's) responsibility to give an appropriate scale parameter.

Finally, this class performs MUCH faster than java.math.BigDecimal for a few reasons. Its behavior is simple because of the designs above. This class is fixed-length without array expansion and re-allocation. This class is mutable, allowing reuse of the same object without re-allocation. This class and {@link UnsignedInt128} are designed such that minimal heap-objectallocations are required for most operations. The only exception is division. Even this class requires a few object allocations for division, though much fewer than BigDecimal.

    VectorExpression expr = new DecimalColAddDecimalColumn(0, 1, 2);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];


    // test without nulls
    expr.evaluate(b);
    assertTrue(r.vector[0].equals(new Decimal128("2.20", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-2.30", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("1.00", (short) 2)));


    // test nulls propagation
    b = getVectorizedRowBatch3DecimalCols();
    DecimalColumnVector c0 = (DecimalColumnVector) b.cols[0];
    c0.noNulls = false;
    c0.isNull[0] = true;
    r = (DecimalColumnVector) b.cols[2];
    expr.evaluate(b);
    assertTrue(!r.noNulls && r.isNull[0]);


    // Verify null output data entry is not 0, but rather the value specified by design,
    // which is the minimum non-0 value, 0.01 in this case.
    assertTrue(r.vector[0].equals(new Decimal128("0.01", (short) 2)));


    // test that overflow produces NULL
    b = getVectorizedRowBatch3DecimalCols();
    c0 = (DecimalColumnVector) b.cols[0];
    c0.vector[0].update("9999999999999999.99", (short) 2); // set to max possible value
    r = (DecimalColumnVector) b.cols[2];
    expr.evaluate(b); // will cause overflow for result at position 0, must yield NULL
    assertTrue(!r.noNulls && r.isNull[0]);


    // verify proper null output data value
    assertTrue(r.vector[0].equals(new Decimal128("0.01", (short) 2)));


    // test left input repeating
    b = getVectorizedRowBatch3DecimalCols();
    c0 = (DecimalColumnVector) b.cols[0];
    c0.isRepeating = true;
    r = (DecimalColumnVector) b.cols[2];
    expr.evaluate(b);
    assertTrue(r.vector[0].equals(new Decimal128("2.20", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("2.20", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("2.20", (short) 2)));


    // test both inputs repeating
    DecimalColumnVector c1 = (DecimalColumnVector) b.cols[1];
    c1.isRepeating = true;
    expr.evaluate(b);
    assertTrue(r.isRepeating);
    assertTrue(r.vector[0].equals(new Decimal128("2.20", (short) 2)));


    // test right input repeating
    b = getVectorizedRowBatch3DecimalCols();
    c1 = (DecimalColumnVector) b.cols[1];
    c1.isRepeating = true;
    c1.vector[0].update("2", (short) 2);
    r = (DecimalColumnVector) b.cols[2];
    expr.evaluate(b);
    assertTrue(r.vector[2].equals(new Decimal128("2", (short) 2)));
  }

View Full Code Here

    VectorExpression expr = new DecimalColSubtractDecimalColumn(0, 1, 2);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];


    // test without nulls
    expr.evaluate(b);
    assertTrue(r.vector[0].equals(new Decimal128("0.20", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-4.30", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("-1.00", (short) 2)));


    // test that underflow produces NULL
    b = getVectorizedRowBatch3DecimalCols();
    DecimalColumnVector c0 = (DecimalColumnVector) b.cols[0];
    c0.vector[0].update("-9999999999999999.99", (short) 2); // set to min possible value

View Full Code Here

    VectorExpression expr = new DecimalColMultiplyDecimalColumn(0, 1, 2);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];


    // test without nulls
    expr.evaluate(b);
    assertTrue(r.vector[0].equals(new Decimal128("1.20", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-3.30", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("0.00", (short) 2)));


    // test that underflow produces NULL
    b = getVectorizedRowBatch3DecimalCols();
    DecimalColumnVector c0 = (DecimalColumnVector) b.cols[0];
    c0.vector[0].update("9999999999999999.99", (short) 2); // set to max possible value

View Full Code Here

   * cases used in the source code template ColumnArithmeticScalarDecimal.txt.
   */
  @Test
  public void testDecimalColAddDecimalScalar() {
    VectorizedRowBatch b = getVectorizedRowBatch3DecimalCols();
    Decimal128 d = new Decimal128(1);
    VectorExpression expr = new DecimalColAddDecimalScalar(0, d, 2);


    // test without nulls
    expr.evaluate(b);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.vector[0].equals(new Decimal128("2.20", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-2.30", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("1.00", (short) 2)));


    // test null propagation
    b = getVectorizedRowBatch3DecimalCols();
    DecimalColumnVector in = (DecimalColumnVector) b.cols[0];
    r = (DecimalColumnVector) b.cols[2];
    in.noNulls = false;
    in.isNull[0] = true;
    expr.evaluate(b);
    assertTrue(!r.noNulls);
    assertTrue(r.isNull[0]);


    // test repeating case, no nulls
    b = getVectorizedRowBatch3DecimalCols();
    in = (DecimalColumnVector) b.cols[0];
    in.isRepeating = true;
    expr.evaluate(b);
    r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.isRepeating);
    assertTrue(r.vector[0].equals(new Decimal128("2.20", (short) 2)));


    // test repeating case for null value
    b = getVectorizedRowBatch3DecimalCols();
    in = (DecimalColumnVector) b.cols[0];
    in.isRepeating = true;

View Full Code Here

   * The template is used for division and modulo.
   */
  @Test
  public void testDecimalColDivideDecimalScalar() {
    VectorizedRowBatch b = getVectorizedRowBatch3DecimalCols();
    Decimal128 d = new Decimal128("2.00", (short) 2);
    VectorExpression expr = new DecimalColDivideDecimalScalar(0, d, 2);




    // test without nulls
    expr.evaluate(b);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.vector[0].equals(new Decimal128("0.60", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-1.65", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("0", (short) 2)));


    // test null propagation
    b = getVectorizedRowBatch3DecimalCols();
    DecimalColumnVector in = (DecimalColumnVector) b.cols[0];
    r = (DecimalColumnVector) b.cols[2];
    in.noNulls = false;
    in.isNull[0] = true;
    expr.evaluate(b);
    assertTrue(!r.noNulls);
    assertTrue(r.isNull[0]);


    // test repeating case, no nulls
    b = getVectorizedRowBatch3DecimalCols();
    in = (DecimalColumnVector) b.cols[0];
    in.isRepeating = true;
    expr.evaluate(b);
    r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.isRepeating);
    assertTrue(r.vector[0].equals(new Decimal128("0.60", (short) 2)));


    // test repeating case for null value
    b = getVectorizedRowBatch3DecimalCols();
    in = (DecimalColumnVector) b.cols[0];
    in.isRepeating = true;
    in.isNull[0] = true;
    in.noNulls = false;
    expr.evaluate(b);
    r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.isRepeating);
    assertTrue(!r.noNulls);
    assertTrue(r.isNull[0]);


    // test that zero-divide produces null for all output values
    b = getVectorizedRowBatch3DecimalCols();
    in = (DecimalColumnVector) b.cols[0];
    expr = new DecimalColDivideDecimalScalar(0, new Decimal128("0", (short) 2), 2);
    expr.evaluate(b);
    r = (DecimalColumnVector) b.cols[2];
    assertFalse(r.noNulls);
    assertTrue(r.isNull[0]);
    assertTrue(r.isRepeating);

View Full Code Here

   * for template ScalarDivideColumnDecimal.txt.
   */
  @Test
  public void testDecimalScalarDivideDecimalColumn() {
    VectorizedRowBatch b = getVectorizedRowBatch3DecimalCols();
    Decimal128 d = new Decimal128("3.96", (short) 2);  // 1.20 * 3.30
    VectorExpression expr = new DecimalScalarDivideDecimalColumn(d, 0, 2);


    // test without nulls
    expr.evaluate(b);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.vector[0].equals(new Decimal128("3.30", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-1.20", (short) 2)));
    assertFalse(r.noNulls); // entry 2 is null due to zero-divide
    assertTrue(r.isNull[2]);


    // test null propagation
    b = getVectorizedRowBatch3DecimalCols();
    DecimalColumnVector in = (DecimalColumnVector) b.cols[0];
    r = (DecimalColumnVector) b.cols[2];
    in.noNulls = false;
    in.isNull[0] = true;
    expr.evaluate(b);
    assertTrue(!r.noNulls);
    assertTrue(r.isNull[0]);


    // test repeating case, no nulls
    b = getVectorizedRowBatch3DecimalCols();
    in = (DecimalColumnVector) b.cols[0];
    in.isRepeating = true;
    expr.evaluate(b);
    r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.isRepeating);
    assertTrue(r.vector[0].equals(new Decimal128("3.30", (short) 2)));


    // test repeating case for null value
    b = getVectorizedRowBatch3DecimalCols();
    in = (DecimalColumnVector) b.cols[0];
    in.isRepeating = true;

View Full Code Here


  // Spot check Decimal Col-Scalar Modulo
  @Test
  public void testDecimalColModuloDecimalScalar() {
    VectorizedRowBatch b = getVectorizedRowBatch3DecimalCols();
    Decimal128 d = new Decimal128("2.00", (short) 2);
    VectorExpression expr = new DecimalColModuloDecimalScalar(0, d, 2);


    // test without nulls
    expr.evaluate(b);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.vector[0].equals(new Decimal128("1.20", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-1.30", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("0", (short) 2)));


    // try again with some different data values and divisor
    DecimalColumnVector in = (DecimalColumnVector) b.cols[0];
    in.vector[0].update("15.40", (short) 2);
    in.vector[1].update("-17.20", (short) 2);
    in.vector[2].update("70.00", (short) 2);
    d.update("4.75", (short) 2);


    expr.evaluate(b);
    assertTrue(r.vector[0].equals(new Decimal128("1.15", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-2.95", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("3.50", (short) 2)));


    // try a zero-divide to show a repeating NULL is produced
    d.update("0", (short) 2);
    expr.evaluate(b);
    assertFalse(r.noNulls);
    assertTrue(r.isNull[0]);
    assertTrue(r.isRepeating);
  }

View Full Code Here


  // Spot check decimal scalar-column modulo
  @Test
  public void testDecimalScalarModuloDecimalColumn() {
    VectorizedRowBatch b = getVectorizedRowBatch3DecimalCols();
    Decimal128 d = new Decimal128("2.00", (short) 2);
    VectorExpression expr = new DecimalScalarModuloDecimalColumn(d, 0, 2);


    // test without nulls
    expr.evaluate(b);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.vector[0].equals(new Decimal128("0.80", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("2.00", (short) 2)));
    assertFalse(r.noNulls); // entry 2 will be null due to zero-divide
    assertTrue(r.isNull[2]);


    // try again with some different data values
    DecimalColumnVector in = (DecimalColumnVector) b.cols[0];
    in.vector[0].update("0.50", (short) 2);
    in.vector[1].update("0.80", (short) 2);
    in.vector[2].update("0.70", (short) 2);


    expr.evaluate(b);
    assertTrue(r.vector[0].equals(new Decimal128("0.00", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("0.40", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("0.60", (short) 2)));
  }

View Full Code Here

  @Test
  public void testDecimalColDivideDecimalColumn() {
    VectorizedRowBatch b = getVectorizedRowBatch3DecimalCols();
    DecimalColumnVector in1 = (DecimalColumnVector) b.cols[1];
    for (int i = 0; i < 3; i++) {
      in1.vector[i] = new Decimal128("0.50", (short) 2);
    }
    VectorExpression expr = new DecimalColDivideDecimalColumn(0, 1, 2);
    expr.evaluate(b);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];


    // all divides are by 0.50 so the result column is 2 times col 0.
    assertTrue(r.vector[0].equals(new Decimal128("2.40", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-6.60", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("0", (short) 2)));


    // test null on left
    b.cols[0].noNulls = false;
    b.cols[0].isNull[0] = true;
    expr.evaluate(b);
    assertFalse(r.noNulls);
    assertTrue(r.isNull[0]);


    // test null on right
    b = getVectorizedRowBatch3DecimalCols();
    b.cols[1].noNulls = false;
    b.cols[1].isNull[0] = true;
    r = (DecimalColumnVector) b.cols[2];
    expr.evaluate(b);
    assertFalse(r.noNulls);
    assertTrue(r.isNull[0]);


    // test null on both sides
    b = getVectorizedRowBatch3DecimalCols();
    b.cols[0].noNulls = false;
    b.cols[0].isNull[0] = true;
    b.cols[1].noNulls = false;
    b.cols[1].isNull[0] = true;
    expr.evaluate(b);
    assertFalse(r.noNulls);
    assertTrue(r.isNull[0]);
    assertFalse(r.isNull[1]);
    assertFalse(r.isNull[2]);


    // test repeating on left
    b = getVectorizedRowBatch3DecimalCols();
    b.cols[0].isRepeating = true;
    expr.evaluate(b);
    r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.vector[2].equals(new Decimal128("1.20", (short) 2)));


    // test repeating on right
    b = getVectorizedRowBatch3DecimalCols();
    b.cols[1].isRepeating = true;
    expr.evaluate(b);
    r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.vector[2].equals(new Decimal128("0", (short) 2)));


    // test both repeating
    b = getVectorizedRowBatch3DecimalCols();
    b.cols[0].isRepeating = true;
    b.cols[1].isRepeating = true;
    expr.evaluate(b);
    r = (DecimalColumnVector) b.cols[2];
    assertTrue(r.isRepeating);
    assertTrue(r.vector[0].equals(new Decimal128("1.20", (short) 2)));


    // test zero-divide to show it results in NULL
    b = getVectorizedRowBatch3DecimalCols();
    ((DecimalColumnVector) b.cols[1]).vector[0].update(0);
    expr.evaluate(b);

View Full Code Here

  @Test
  public void testDecimalColModuloDecimalColumn() {
    VectorizedRowBatch b = getVectorizedRowBatch3DecimalCols();
    DecimalColumnVector in1 = (DecimalColumnVector) b.cols[1];
    for (int i = 0; i < 3; i++) {
      in1.vector[i] = new Decimal128("0.50", (short) 2);
    }
    VectorExpression expr = new DecimalColModuloDecimalColumn(0, 1, 2);
    expr.evaluate(b);
    DecimalColumnVector r = (DecimalColumnVector) b.cols[2];


    assertTrue(r.vector[0].equals(new Decimal128("0.20", (short) 2)));
    assertTrue(r.vector[1].equals(new Decimal128("-0.30", (short) 2)));
    assertTrue(r.vector[2].equals(new Decimal128("0", (short) 2)));
  }

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of org.apache.hadoop.hive.common.type.Decimal128

org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFMaxDecimal

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFMaxDecimal$Aggregation

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFMinDecimal

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFMinDecimal$Aggregation

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFStdPopDecimal

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFStdSampDecimal

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFVarPopDecimal

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFVarSampDecimal

org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFAvgDecimal

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.