Examples of PairRDDFunctions


Examples of org.apache.spark.rdd.PairRDDFunctions

  private <T, K, V> void writeToDatasetHelper(T rdd, String datasetName, Class<K> kClass, Class<V> vClass) {
    Configuration hConf = setOutputDataset(datasetName);
    ClassTag<K> kClassTag = ClassTag$.MODULE$.apply(kClass);
    ClassTag<V> vClassTag = ClassTag$.MODULE$.apply(vClass);
    PairRDDFunctions pairRDD = new PairRDDFunctions<K, V>((RDD<Tuple2<K, V>>) rdd, kClassTag, vClassTag, null);
    pairRDD.saveAsNewAPIHadoopFile(datasetName, kClass, vClass, SparkDatasetOutputFormat.class, hConf);
  }
View Full Code Here
TOP
Copyright © 2018 www.massapi.com. All rights reserved.
All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.