Examples of AvroSource

org.apache.flume.source.AvroSource

A {@link Source} implementation that receives Avro events from clients thatimplement {@link AvroSourceProtocol}.

This source forms one half of Flume's tiered collection support. Internally, this source uses Avro's NettyTransceiver to listen for, and handle events. It can be paired with the builtin AvroSink to create tiered collection topologies. Of course, nothing prevents one from using this source to receive data from other custom built infrastructure that uses the same Avro protocol (specifically {@link AvroSourceProtocol}).

Events may be received from the client either singly or in batches.Generally, larger batches are far more efficient, but introduce a slight delay (measured in millis) in delivery. A batch submitted to the configured {@link Channel}atomically (i.e. either all events make it into the channel or none).

Configuration options

Parameter	Description	Unit / Type	Default
`bind`	The hostname or IP to which the source will bind.	Hostname or IP / String	none (required)
`port`	The port to which the source will bind and listen for events.	TCP port / int	none (required)
`threads`	Max number of threads assigned to thread pool, 0 being unlimited	Count / int	0(optional)

Metrics

TODO

Examples of org.apache.flume.source.AvroSource


    @Before
    public void setUp() throws Exception {
        File file = new File("target/file-channel");
        boolean result = deleteFiles(file);
        primarySource = new AvroSource();
        primarySource.setName("Primary");
        altSource = new AvroSource();
        altSource.setName("Alternate");
        primaryChannel = new MemoryChannel();
        primaryChannel.setName("Primary Memory");
        alternateChannel = new MemoryChannel();
        alternateChannel.setName("Alternate Memory");

View Full Code Here

Examples of org.apache.flume.source.AvroSource


    @Before
    public void setUp() throws Exception {
        File file = new File("target/file-channel");
        boolean result = deleteFiles(file);
        primarySource = new AvroSource();
        primarySource.setName("Primary");
        altSource = new AvroSource();
        altSource.setName("Alternate");
        primaryChannel = new MemoryChannel();
        primaryChannel.setName("Primary Memory");
        alternateChannel = new MemoryChannel();
        alternateChannel.setName("Alternate Memory");

View Full Code Here

Examples of org.apache.flume.source.AvroSource


    }


    @Before
    public void setUp() throws Exception {
        eventSource = new AvroSource();
        channel = new MemoryChannel();


        Configurables.configure(channel, new Context());


        avroLogger = (Logger) LogManager.getLogger("avrologger");

View Full Code Here

Examples of org.apache.flume.source.AvroSource

  private Properties props;


  @Before
  public void initiate() throws Exception{
    int port = 25430;
    source = new AvroSource();
    ch = new MemoryChannel();
    Configurables.configure(ch, new Context());


    Context context = new Context();
    context.put("port", String.valueOf(port));

View Full Code Here

Examples of org.apache.flume.source.AvroSource

    URL schemaUrl = getClass().getClassLoader().getResource("myrecord.avsc");
    Files.copy(Resources.newInputStreamSupplier(schemaUrl),
        new File("/tmp/myrecord.avsc"));


    int port = 25430;
    source = new AvroSource();
    ch = new MemoryChannel();
    Configurables.configure(ch, new Context());


    Context context = new Context();
    context.put("port", String.valueOf(port));

View Full Code Here

Examples of org.apache.flume.source.AvroSource


    }


    @Before
    public void setUp() throws Exception {
        eventSource = new AvroSource();
        channel = new MemoryChannel();


        Configurables.configure(channel, new Context());


        avroLogger = (Logger) LogManager.getLogger("avrologger");

View Full Code Here

Examples of org.apache.flume.source.AvroSource


    }


    @Before
    public void setUp() throws Exception {
        eventSource = new AvroSource();
        channel = new MemoryChannel();


        Configurables.configure(channel, new Context());


        avroLogger = (Logger) LogManager.getLogger("avrologger");

View Full Code Here

Examples of org.apache.flume.source.AvroSource


    @Before
    public void setUp() throws Exception {
        File file = new File("target/file-channel");
        boolean result = deleteFiles(file);
        primarySource = new AvroSource();
        primarySource.setName("Primary");
        altSource = new AvroSource();
        altSource.setName("Alternate");
        primaryChannel = new MemoryChannel();
        primaryChannel.setName("Primary Memory");
        alternateChannel = new MemoryChannel();
        alternateChannel.setName("Alternate Memory");

View Full Code Here

Examples of org.apache.flume.source.AvroSource


    @Before
    public void setUp() throws Exception {
        File file = new File("target/file-channel");
        boolean result = deleteFiles(file);
        primarySource = new AvroSource();
        primarySource.setName("Primary");
        altSource = new AvroSource();
        altSource.setName("Alternate");
        primaryChannel = new MemoryChannel();
        primaryChannel.setName("Primary Memory");
        alternateChannel = new MemoryChannel();
        alternateChannel.setName("Alternate Memory");

View Full Code Here

Examples of org.apache.flume.source.AvroSource


    }


    @Before
    public void setUp() throws Exception {
        eventSource = new AvroSource();
        channel = new MemoryChannel();


        Configurables.configure(channel, new Context());


        avroLogger = (Logger) LogManager.getLogger("avrologger");

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.