Examples of MboxSplitterFunction

com.scaleunlimited.helpful.operations.MboxSplitterFunction
Cascading function to split a single mbox file into N entries, one per each email. This then lets us parse them as individual emails, which works better with the current Tika API, as we can then use the metadata returned from the Parser.parse() call to get subject, etc.

Examples of com.scaleunlimited.helpful.operations.MboxSplitterFunction

    when(_funcCall.getOutputCollector()).thenReturn(_collector);
  }
  
  @Test
  public void testSplitterWithNonMbox() {
    MboxSplitterFunction splitter = new MboxSplitterFunction();
    
    FetchedDatum datum = new FetchedDatum("baseUrl", "redirectedUrl", 0, new HttpHeaders(), new ContentBytes(), "text/ascii", 0);
    TupleEntry value = new TupleEntry(datum.getTupleEntry());
    
    when(_funcCall.getArguments()).thenReturn(value);
    splitter.operate(_process, _funcCall);
    
    verify(_collector).add(value);
  }

View Full Code Here

Examples of com.scaleunlimited.helpful.operations.MboxSplitterFunction

    verify(_collector).add(value);
  }
  
  @Test
  public void testSplitterTwoEmails() throws UnsupportedEncodingException {
    MboxSplitterFunction splitter = new MboxSplitterFunction();


    final String mboxString = "From 1\r\rContent 1\r\rFrom 2\r\rContent 2";
    byte[] mboxContent = mboxString.getBytes("us-ascii");
    FetchedDatum datum = new FetchedDatum("baseUrl", "redirectedUrl", 0, new HttpHeaders(), new ContentBytes(mboxContent), "application/mbox", 0);
    TupleEntry value = new TupleEntry(datum.getTupleEntry());
    
    when(_funcCall.getArguments()).thenReturn(value);
    splitter.operate(_process, _funcCall);


    verify(_collector, times(2)).add(any(TupleEntry.class));
  }

View Full Code Here

Examples of com.scaleunlimited.helpful.operations.MboxSplitterFunction

    @SuppressWarnings("serial")
  private static class SplitEmails extends SubAssembly {


    public SplitEmails(FetchPipe fetchPipe) {
            Pipe splitPipe = new Pipe(SPLITTER_PIPE_NAME, fetchPipe.getContentTailPipe());
            splitPipe = new Each(splitPipe, new MboxSplitterFunction());
            // TODO KKr - code currently relies on splitPipe being first tail pipe.
            setTails(splitPipe, fetchPipe.getStatusTailPipe());
      }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.