Examples of com.cloudera.cdk.data.DatasetRepository.create()

com.cloudera.cdk.data.DatasetRepository.create()

Create a {@link Dataset} with the supplied {@code descriptor}. Depending on the underlying dataset storage, some schemas types or configurations may not be supported. If an illegal schema is supplied, an exception will be thrown by the implementing class. It is illegal to create a more than one dataset with a given name. If a duplicate name is provided, an exception is thrown. @param name The fully qualified dataset name @param descriptor A descriptor that describes the schema and otherproperties of the dataset @return The newly created dataset @throws IllegalArgumentException If {@code name} or {@code descriptor}is {@code null} @throws DatasetExistsException If a {@code Dataset} named {@code name}already exists. @throws ConcurrentSchemaModificationException If the {@code Dataset}schema is updated concurrently. @throws IncompatibleSchemaException If the schema is not compatible with existing datasets with shared storage (e.g. in the same HBase table). @throws DatasetRepositoryException


    // Create a dataset of events with the Avro schema in the repository
    DatasetDescriptor descriptor = new DatasetDescriptor.Builder()
        .schemaUri("resource:event.avsc")
        .build();
    repo.create("events", descriptor);


    return 0;
  }


  public static void main(String... args) throws Exception {

        .name("rating").type().intType().noDefault()
        .name("timeInSeconds").type().intType().noDefault()
        .endRecord();


    // create
    repo.create("ratings", new DatasetDescriptor.Builder()
//        .location("hdfs:u.data")
        .format(Formats.CSV)
        .property("cdk.csv.delimiter", "\t")
        .schema(ratingSchema)
        .build());

        .name("videoReleaseDate").type().stringType().noDefault()
        .name("imdbURL").type().stringType().noDefault()
        // ignore genre fields for now
        .endRecord();


    repo.create("movies", new DatasetDescriptor.Builder()
//        .location("hdfs:u.item")
        .format(Formats.CSV)
        .property("cdk.csv.delimiter", "|")
        .schema(movieSchema)
        .build());

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.