Examples of TwitterSource

DataAcquisition.TwitterSource
@author Muhammad Zahran
org.apache.flink.streaming.connectors.twitter.TwitterSource
Implementation of {@link SourceFunction} specialized to emit tweets fromTwitter. It can connect to Twitter Streaming API, collect tweets and
org.apache.flume.source.twitter.TwitterSource
Demo Flume source that connects via Streaming API to the 1% sample twitter firehose, continously downloads tweets, converts them to Avro format and sends Avro events to a downstream Flume sink. Requires the consumer and access tokens and secrets of a Twitter developer account

Examples of DataAcquisition.TwitterSource

  }


  void setSource(String query, DataSources ds) {


    if (ds == DataSources.Twitter) {
      dataSource = new TwitterSource();
      ((TwitterSource)(dataSource)).setSearchQuery(query);
      dataSource.harvest();
    } else {
      String site = "";
      if (ds == DataSources.Amazon) {

View Full Code Here

Examples of org.apache.flink.streaming.connectors.twitter.TwitterSource


  private static DataStream<String> getTextDataStream(
      StreamExecutionEnvironment env) {
    if (fromFile) {
      // read the text file from given input path
      return env.addSource(new TwitterSource(path));
    } else {
      // get default test text data
      return env.fromElements(TwitterStreamData.TEXTS);
    }
  }

View Full Code Here

Examples of org.apache.flume.source.twitter.TwitterSource

    context.put("consumerSecret", consumerSecret);
    context.put("accessToken", accessToken);
    context.put("accessTokenSecret", accessTokenSecret);
    context.put("maxBatchDurationMillis", "1000");


    TwitterSource source = new TwitterSource();
    source.configure(context);


    Map<String, String> channelContext = new HashMap();
    channelContext.put("capacity", "1000000");
    channelContext.put("keep-alive", "0"); // for faster tests
    Channel channel = new MemoryChannel();
    Configurables.configure(channel, new Context(channelContext));


    Sink sink = new LoggerSink();
    sink.setChannel(channel);
    sink.start();
    DefaultSinkProcessor proc = new DefaultSinkProcessor();
    proc.setSinks(Collections.singletonList(sink));
    SinkRunner sinkRunner = new SinkRunner(proc);
    sinkRunner.start();


    ChannelSelector rcs = new ReplicatingChannelSelector();
    rcs.setChannels(Collections.singletonList(channel));
    ChannelProcessor chp = new ChannelProcessor(rcs);
    source.setChannelProcessor(chp);
    source.start();


    Thread.sleep(5000);
    source.stop();
    sinkRunner.stop();
    sink.stop();
  }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.