List of openForWrite() Examples

Examples of openForWrite()

cascading.tap.Hfs.openForWrite()
cascading.tap.Tap.openForWrite()
Method openForWrite opens the resource represented by this Tap instance for writing.
This method is used internally and does not honor the {@link SinkMode} setting. If SinkMode is{@link SinkMode#REPLACE}, this call may fail. See {@link #openForWrite(cascading.flow.FlowProcess)}.
{@code output} value may be null, if so, sub-classes must inquire with the underlying {@link Scheme}via {@link Scheme#sinkConfInit(cascading.flow.FlowProcess,Tap,Object)} to get the properoutput type and instantiate it before calling {@code super.openForWrite()}. @param flowProcess of type FlowProcess @param output of type Output @return TupleEntryCollector @throws java.io.IOException when the resource cannot be opened
cascading.tap.hadoop.Hfs.openForWrite()
com.scaleunlimited.cascading.local.DirectoryTap.openForWrite()

Examples of cascading.tap.hadoop.Hfs.openForWrite()

      maxHashes = BloomProps.getMaxBloomHashes(conf);
      minHashes = BloomProps.getMinBloomHashes(conf);


      for (int i = minHashes; i <= maxHashes; i++) {
        Hfs tap = new Hfs(new SequenceFile(new Fields("split", "filter")), partsRoot + "/" + i + "/");
        numHashesToCollector.put(i, tap.openForWrite(new HadoopFlowProcess(conf)));
      }


    } catch (IOException e) {
      throw new RuntimeException(e);
    }

View Full Code Here

Examples of cascading.tap.hadoop.Hfs.openForWrite()


  @Test
  public void testSimpleCombiner() throws IOException {


    Hfs source = new Hfs(new SequenceFile(new Fields("key", "value")), INPUT_PATH);
    TupleEntryCollector tc = source.openForWrite(CascadingUtil.get().getFlowProcess());
    tc.add(new Tuple("k0", 1));
    tc.add(new Tuple("k0", 2));
    tc.add(new Tuple("k1", 1));
    tc.add(new Tuple("k1", -3));
    tc.add(new Tuple("k1", 10));

View Full Code Here

Examples of cascading.tap.hadoop.Hfs.openForWrite()


  @Test
  public void testSimpleCombinerWithMemoryLimit() throws IOException {


    Hfs source = new Hfs(new SequenceFile(new Fields("key", "value")), INPUT_PATH);
    TupleEntryCollector tc = source.openForWrite(CascadingUtil.get().getFlowProcess());
    tc.add(new Tuple("key0", 1));
    tc.add(new Tuple("key0", 2));
    tc.add(new Tuple("key1", 1));
    tc.add(new Tuple("key1", -3));
    tc.add(new Tuple("key0", 10));

View Full Code Here

Examples of cascading.tap.hadoop.Hfs.openForWrite()


  @Test
  public void testComplexCombiner() throws IOException {


    Hfs source = new Hfs(new SequenceFile(new Fields("key", "value")), INPUT_PATH);
    TupleEntryCollector tc = source.openForWrite(CascadingUtil.get().getFlowProcess());
    tc.add(new Tuple("k0", 1));
    tc.add(new Tuple("k0", 2));
    tc.add(new Tuple("k1", 1));
    tc.add(new Tuple("k1", 4));
    tc.add(new Tuple("k1", 10));

View Full Code Here

Examples of cascading.tap.hadoop.Hfs.openForWrite()

  private final String OUTPUT_PATH = getTestRoot() + "/output";


  @Test
  public void testMain() throws IOException {
    Hfs source = new Hfs(new SequenceFile(new Fields("key", "value")), INPUT_PATH);
    TupleEntryCollector tc = source.openForWrite(CascadingUtil.get().getFlowProcess());
    tc.add(new Tuple("k0", 1));
    tc.add(new Tuple("k0", 2));
    tc.add(new Tuple("k1", 10));
    tc.add(new Tuple("k1", -2));
    tc.add(new Tuple("k1", -9));

View Full Code Here

Examples of cascading.tap.hadoop.Hfs.openForWrite()


  @Test
  public void testMain() throws IOException {


    Hfs source = new Hfs(new SequenceFile(new Fields("key", "a", "b")), INPUT_PATH);
    TupleEntryCollector tc = source.openForWrite(CascadingUtil.get().getFlowProcess());
    tc.add(new Tuple("k0", 1, 1));
    tc.add(new Tuple("k0", 2, 5));
    tc.add(new Tuple("k1", 1, 7));
    tc.add(new Tuple("k1", -2, 10));
    tc.add(new Tuple("k1", -2, -9));

View Full Code Here

Examples of cascading.tap.hadoop.Hfs.openForWrite()

  private final String OUTPUT_PATH = getTestRoot() + "/output";


  @Test
  public void testMain() throws IOException {
    Hfs source = new Hfs(new SequenceFile(new Fields("key")), INPUT_PATH);
    TupleEntryCollector tc = source.openForWrite(CascadingUtil.get().getFlowProcess());
    tc.add(new Tuple("k0"));
    tc.add(new Tuple("k0"));
    tc.add(new Tuple("k1"));
    tc.add(new Tuple("k1"));
    tc.add(new Tuple("k1"));

View Full Code Here

Examples of com.scaleunlimited.cascading.local.DirectoryTap.openForWrite()

        final Fields testFields = new Fields("id", "name", "price", "inStock");
        String out = getTestDir() + "testIndexSink/out";


        DirectoryTap solrSink = new DirectoryTap(new SolrScheme(testFields, SOLR_CORE_DIR), out, SinkMode.REPLACE);
        
        TupleEntryCollector writer = solrSink.openForWrite(new LocalFlowProcess());


        for (int i = 0; i < 100; i++) {
            writer.add(new Tuple(i, "product #" + i, i * 1.0f, true));
        }

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.