List of publishStat() Examples

Examples of publishStat()

org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()
This method publishes a given statistic into a disk storage, possibly HBase or MySQL. @param fileID : a string identification the statistics to be published by all mappers/reducers and then gathered. The statID is unique per output partition per task, e.g.,: the output directory name (uniq per FileSinkOperator) + the partition specs (only for dynamic partitions) + taskID (last component of task file) @param stats : a map containing key-value pairs, where key is a string representing the statistic to be published, and value is a string representing the value for the given statistic @return true if successful, false otherwise

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()


      // update which should not take any effect
      fillStatMap("190", "1000");
      assertTrue(statsPublisher.publishStat("file_00000_a", stats));
      fillStatMap("290", "2000");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));


      // update that should take effect
      fillStatMap("500", "5000");
      assertTrue(statsPublisher.publishStat("file_00001_a", stats));
      fillStatMap("600", "6000");

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()

      fillStatMap("290", "2000");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));


      // update that should take effect
      fillStatMap("500", "5000");
      assertTrue(statsPublisher.publishStat("file_00001_a", stats));
      fillStatMap("600", "6000");
      assertTrue(statsPublisher.publishStat("file_00001_b", stats));


      // aggregate existing stats
      String rows0 = statsAggregator.aggregateStats("file_00000", StatsSetupConst.ROW_COUNT);

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()


      // update that should take effect
      fillStatMap("500", "5000");
      assertTrue(statsPublisher.publishStat("file_00001_a", stats));
      fillStatMap("600", "6000");
      assertTrue(statsPublisher.publishStat("file_00001_b", stats));


      // aggregate existing stats
      String rows0 = statsAggregator.aggregateStats("file_00000", StatsSetupConst.ROW_COUNT);
      assertEquals("500", rows0);
      String usize0 = statsAggregator.aggregateStats("file_00000",

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()

      assertNotNull(statsAggregator);
      assertTrue(statsAggregator.connect(conf));


      // publish stats
      fillStatMap("200", "");
      assertTrue(statsPublisher.publishStat("file_00000_a", stats));
      fillStatMap("300", "2000");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));




      // aggregate existing stats

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()


      // publish stats
      fillStatMap("200", "");
      assertTrue(statsPublisher.publishStat("file_00000_a", stats));
      fillStatMap("300", "2000");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));




      // aggregate existing stats
      String rows0 = statsAggregator.aggregateStats("file_00000", StatsSetupConst.ROW_COUNT);
      assertEquals("500", rows0);

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()

      assertEquals("2000", usize0);


      // update which should not take any effect - plus the map published is a supset of supported
      // stats
      fillStatMap("190", "");
      assertTrue(statsPublisher.publishStat("file_00000_a", stats));
      fillStatMap("290", "");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));


      // nothing changed
      rows0 = statsAggregator.aggregateStats("file_00000", StatsSetupConst.ROW_COUNT);

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()

      // update which should not take any effect - plus the map published is a supset of supported
      // stats
      fillStatMap("190", "");
      assertTrue(statsPublisher.publishStat("file_00000_a", stats));
      fillStatMap("290", "");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));


      // nothing changed
      rows0 = statsAggregator.aggregateStats("file_00000", StatsSetupConst.ROW_COUNT);
      assertEquals("500", rows0);
      usize0 = statsAggregator.aggregateStats("file_00000",

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()

      usize0 = statsAggregator.aggregateStats("file_00000",
          StatsSetupConst.RAW_DATA_SIZE);
      assertEquals("2000", usize0);


      fillStatMap("500", "");
      assertTrue(statsPublisher.publishStat("file_00000_a", stats));
      fillStatMap("500", "");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));


      // changed + the rawDataSize size was overwriten !!!
      rows0 = statsAggregator.aggregateStats("file_00000", StatsSetupConst.ROW_COUNT);

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()

      assertEquals("2000", usize0);


      fillStatMap("500", "");
      assertTrue(statsPublisher.publishStat("file_00000_a", stats));
      fillStatMap("500", "");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));


      // changed + the rawDataSize size was overwriten !!!
      rows0 = statsAggregator.aggregateStats("file_00000", StatsSetupConst.ROW_COUNT);
      assertEquals("1000", rows0);
      usize0 = statsAggregator.aggregateStats("file_00000",

View Full Code Here

Examples of org.apache.hadoop.hive.ql.stats.StatsPublisher.publishStat()

      assertNotNull(statsAggregator);
      assertTrue(statsAggregator.connect(conf));


      // publish stats
      fillStatMap("200", "1000");
      assertTrue(statsPublisher.publishStat("file_00000_a", stats));
      fillStatMap("300", "2000");
      assertTrue(statsPublisher.publishStat("file_00000_b", stats));


      fillStatMap("400", "3000");
      assertTrue(statsPublisher.publishStat("file_00001_a", stats));

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.