Examples of PartitionKey

com.bazaarvoice.ostrich.partition.PartitionKey
com.cloudera.cdk.data.PartitionKey

A key for retrieving partitions from a {@link Dataset}.

A {@code PartitionKey} is a ordered sequence of values corresponding to the{@link FieldPartitioner}s in a {@link PartitionStrategy}. A {@link PartitionKey} may be obtained using{@link PartitionStrategy#partitionKey(Object)} or{@link PartitionStrategy#partitionKeyForEntity(Object)}.

Implementations of {@link PartitionKey} are typically not thread-safe; that is,the behavior when accessing a single instance from multiple threads is undefined.
@see PartitionStrategy @see FieldPartitioner @see Dataset
com.facebook.presto.spi.PartitionKey
com.mysql.clusterj.core.store.PartitionKey
This interface represents a partition key. A partition key consists of a table and values that make up the partition key. An instance of this interface can be used to specify the partition key used when starting a transaction.
org.kitesdk.data.spi.PartitionKey

A key for retrieving partitions from a {@link org.kitesdk.data.Dataset}.

A {@code PartitionKey} is an ordered sequence of values corresponding to the{@link org.kitesdk.data.spi.FieldPartitioner}s in a {@link org.kitesdk.data.PartitionStrategy}. You can obtain a {@link PartitionKey} using{@link PartitionKey#partitionKeyForEntity(org.kitesdk.data.PartitionStrategy,Object,EntityAccessor,PartitionKey)}.

Implementations of {@link PartitionKey} are typically not thread-safe; that is, the behavior when accessing a single instance from multiple threads is undefined.
@see org.kitesdk.data.PartitionStrategy @see org.kitesdk.data.spi.FieldPartitioner @see org.kitesdk.data.Dataset

Examples of org.kitesdk.data.spi.PartitionKey

        .setSubRecord1(subRecord1).setSubRecord2(subRecord2).build();


    // Test put and get
    dao.put(compositeRecord);


    PartitionKey key = new PartitionKey("1", "1");
    CompositeRecord returnedCompositeRecord = dao.get(key);
    assertEquals("field1_1", returnedCompositeRecord.getSubRecord1()
        .getField1());
    assertEquals("field1_2", returnedCompositeRecord.getSubRecord1()
        .getField2());
    assertEquals("field2_1", returnedCompositeRecord.getSubRecord2()
        .getField1());
    assertEquals("field2_2", returnedCompositeRecord.getSubRecord2()
        .getField2());


    // Test OCC
    assertFalse(dao.put(compositeRecord));
    assertTrue(dao.put(returnedCompositeRecord));


    // Test null field
    subRecord1 = SubRecord1.newBuilder().setKeyPart1("1").setKeyPart2("2")
        .setField1("field1_1").setField2("field1_2").build();
    compositeRecord = CompositeRecord.newBuilder().setSubRecord1(subRecord1)
        .build();
    dao.put(compositeRecord);
    key = new PartitionKey("1", "2");
    compositeRecord = dao.get(key);
    assertEquals(null, compositeRecord.getSubRecord2());
  }

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey

    compositeRecord.put("SubRecord1", subRecord1);
    compositeRecord.put("SubRecord2", subRecord2);


    // Test put and get
    dao.put(compositeRecord);
    PartitionKey key = new PartitionKey("1", "1");
    Map<String, SpecificRecord> returnedCompositeRecord = dao.get(key);
    assertEquals("field1_1", ((SubRecord1) returnedCompositeRecord.get("SubRecord1"))
        .getField1());
    assertEquals("field1_2", ((SubRecord1) returnedCompositeRecord.get("SubRecord1"))
        .getField2());
    assertEquals("field2_1", ((SubRecord2) returnedCompositeRecord.get("SubRecord2"))
        .getField1());
    assertEquals("field2_2", ((SubRecord2) returnedCompositeRecord.get("SubRecord2"))
        .getField2());


    // Test OCC
    assertFalse(dao.put(compositeRecord));
    assertTrue(dao.put(returnedCompositeRecord));


    // Test null field
    compositeRecord = new HashMap<String, SpecificRecord>();
    subRecord1 = SubRecord1.newBuilder().setKeyPart1("1").setKeyPart2("2")
        .setField1("field1_1").setField2("field1_2").build();
    compositeRecord.put("SubRecord1", subRecord1);
    dao.put(compositeRecord);
    key = new PartitionKey("1", "2");
    compositeRecord = dao.get(key);
    assertEquals(null, compositeRecord.get("SubRecord2"));
  }

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey

    possibleValues.add("field1_20");
    possibleValues.add("field1_21");
    possibleValues.add("field1_22");
    possibleValues.add("field1_23");
    possibleValues.add("field1_24");
    PartitionKey startKey = new PartitionKey("part1_20", "part2_20");
    PartitionKey stopKey = new PartitionKey("part1_25", "part2_25");


    EntityScannerBuilder<GenericRecord> builder = dao.getScannerBuilder()
        .setStartKey(startKey).setStopKey(stopKey);


    checkScannerYieldValues(builder.build(), possibleValues);

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey

    Dataset<Record> outputDataset = repo.create("ns", "out", new DatasetDescriptor.Builder()
        .schema(USER_SCHEMA).format(Formats.PARQUET).build());


    writeTestUsers(inputDataset, 10);


    PartitionKey key = new PartitionKey(0);
    Dataset<Record> inputPart0 =
        ((PartitionedDataset<Record>) inputDataset).getPartition(key, false);


    Pipeline pipeline = new MRPipeline(TestCrunchDatasets.class);
    PCollection<GenericData.Record> data = pipeline.read(

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey

    Dataset<Record> outputDataset = repo.create("ns", "out", new DatasetDescriptor.Builder()
        .schema(USER_SCHEMA).partitionStrategy(partitionStrategy).build());


    writeTestUsers(inputDataset, 10);


    PartitionKey key = new PartitionKey(0);
    Dataset<Record> inputPart0 =
        ((PartitionedDataset<Record>) inputDataset).getPartition(key, false);
    Dataset<Record> outputPart0 =
        ((PartitionedDataset<Record>) outputDataset).getPartition(key, true);

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey

    Dataset<Record> outputDataset = repo.create("ns", "out", new DatasetDescriptor.Builder()
        .schema(USER_SCHEMA).partitionStrategy(partitionStrategy).build());


    writeTestUsers(inputDataset, 10);


    PartitionKey key = new PartitionKey(0);
    Dataset<Record> inputPart0 =
        ((PartitionedDataset<Record>) inputDataset).getPartition(key, false);


    Pipeline pipeline = new MRPipeline(TestCrunchDatasets.class);
    PCollection<GenericData.Record> data = pipeline.read(

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey

  }


  @Override
  public E mapToEntity(Result result) {
    boolean allNull = true;
    PartitionKey partitionKey;
    if (keySerDe == null) {
      partitionKey = null;
    } else {
      partitionKey = keySerDe.deserialize(result.getRow());
    }
    EntityComposer.Builder<E> builder = getEntityComposer().getBuilder();
    for (FieldMapping fieldMapping : entitySchema.getColumnMappingDescriptor()
        .getFieldMappings()) {
      Object fieldValue;
      if (fieldMapping.getMappingType() == MappingType.KEY) {
        if (partitionKey != null) {
          // KEY field mappings are always associated with identity mappers,
          // which is enforced by the DatasetDescriptor. Get the value from the
          // key at the correct position.
          fieldValue = partitionKey.get(
              keySchema.position(fieldMapping.getFieldName()));
        } else {
          // This should never happen. The partitionKey is null only when
          // a keySerDe hasn't been set, which indicates we have an entity
          // without a key. That means there should be no KEY mapping types, but

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey


  @Override
  public boolean delete(E entity) {
    VersionCheckAction checkAction = entityMapper.mapFromEntity(entity)
        .getVersionCheckAction();
    PartitionKey key = entityMapper.mapToKey(entity);
    return clientTemplate.delete(key, entityMapper.getRequiredColumns(),
        checkAction, entityMapper.getKeySerDe());
  }

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey

   *          The last name of users to scan.
   */
  public void printUserProfileActionsForLastName(String lastName) {
    // Create a partial key that will allow us to start the scanner from the
    // first user record that has last name equal to the one provided.
    PartitionKey startKey = new PartitionKey("lastName");


    // Get the scanner with the start key. Null for stopKey in the getScanner
    // method indicates that the scanner will scan to the end of the table. Our
    // loop will break out when it encounters a record without the last name.

View Full Code Here

Examples of org.kitesdk.data.spi.PartitionKey

    // Get the timestamp we'll use to set the value of the profile_updated
    // action.
    long ts = System.currentTimeMillis();


    // Construct the key we'll use to fetch the user.
    PartitionKey key = new PartitionKey(lastName, firstName);


    // Get the profile and actions entity from the composite dao.
    UserProfileActionsModel profileActionsModel = userProfileActionsDao
        .get(key);

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.