Examples of parallelize()

Examples of org.apache.spark.api.java.JavaSparkContext.parallelize()

148
149
150
151
152
153
154
155
156
157
158
    // Alternatively, a JavaSchemaRDD can be created for a JSON dataset represented by
    // a RDD[String] storing one JSON object per string.
    List<String> jsonData = Arrays.asList(
          "{\"name\":\"Yin\",\"address\":{\"city\":\"Columbus\",\"state\":\"Ohio\"}}");
    JavaRDD<String> anotherPeopleRDD = ctx.parallelize(jsonData);
    JavaSchemaRDD peopleFromJsonRDD = sqlCtx.jsonRDD(anotherPeopleRDD);

    // Take a look at the schema of this new JavaSchemaRDD.
    peopleFromJsonRDD.printSchema();
    // The schema of anotherPeople is ...
View Full Code Here

Examples of org.apache.spark.api.java.JavaSparkContext.parallelize()

44
45
46
47
48
49
50
51
52
53
54
        List<Person> people = Arrays.asList(
                Person.newInstance(1, "John", new Date()),
                Person.newInstance(2, "Anna", new Date()),
                Person.newInstance(3, "Andrew", new Date())
        );
        JavaRDD<Person> rdd = sc.parallelize(people);
        javaFunctions(rdd, Person.class).saveToCassandra("test", "people");

        // use case: we want to read that data as an RDD of CassandraRows and convert them to strings...
        JavaRDD<String> cassandraRowsRDD = javaFunctions(sc).cassandraTable("test", "people")
                .map(new Function<CassandraRow, String>() {
View Full Code Here

Examples of org.apache.spark.api.java.JavaSparkContext.parallelize()

46
47
48
49
50
51
52
53
54
55
56
        List<Person> people = Arrays.asList(
                Person.newInstance(1, "John", new Date()),
                Person.newInstance(2, "Anna", new Date()),
                Person.newInstance(3, "Andrew", new Date())
        );
        JavaRDD<Person> rdd = sc.parallelize(people);
        javaFunctions(rdd)
                .writerBuilder("test", "people", mapToRows(Person.class))
                .saveToCassandra();

        // use case: we want to read that data as an RDD of CassandraRows and convert them to strings...
View Full Code Here
TOP
Copyright © 2018 www.massapi.com. All rights reserved.
All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.