Examples of org.kitesdk.data.Dataset

org.kitesdk.data.Dataset

A logical representation of a set of data entities.

Logically, all datasets have two generic properties: a name, and a descriptor that holds information such as the dataset's schema and its partitioning information. Concrete implementations of {@code Dataset} can supportadditional properties, mandatory or otherwise, as needed. {@code Dataset}s are not normally instantiated directly, but managed by a repository (also implementation-specific).

Implementations of {@link Dataset} are immutable.

@param < E> The type of entities stored in this {@code Dataset}. @see View @see DatasetWriter @see DatasetReader @see PartitionStrategy @see DatasetDescriptor @see Schema

   * {@link org.apache.sqoop.lib.SqoopAvroRecord}. The output key is
   * {@link org.apache.avro.generic.GenericRecord}.
   */
  public static void configureImportJob(Configuration conf, Schema schema,
      String uri, boolean doAppend) throws IOException {
    Dataset dataset;
    if (doAppend) {
      try {
        dataset = Datasets.load(uri);
      } catch (DatasetNotFoundException ex) {
        dataset = createDataset(schema, uri);
      }
      Schema writtenWith = dataset.getDescriptor().getSchema();
      if (!SchemaValidationUtil.canRead(writtenWith, schema)) {
        throw new IOException(
            String.format("Expected schema: %s%nActual schema: %s",
                writtenWith, schema));
      }

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of org.kitesdk.data.Dataset

Related Classes of org.kitesdk.data.Dataset