Examples of com.cloudera.flume.core.BackOffFailOverSink

com.cloudera.flume.core.BackOffFailOverSink
This failover sink initially opens the primary and the backup attempts to to append to the primary. If the primary fails, it falls back and appends to the secondary. When the next message appears, it will continue to send to the secondary until an initialBackoff time has elapsed. Once this has elapsed, an attempt to reopen the primary and append to the primary. If the primary fails again, backoff adjusted and we fall back to the secondary again. If we reach the secondary and it fails, the append calls will throw an exception. These can be chained if multiple failovers are desired. (failover to another failover). To save on open handles or connections, one can wrap the secondary with a lazyOpen decorator

      EventSink tsnk = new CompositeSink(context, failoverSpec);
      if (cur == null) {
        cur = tsnk;
        continue;
      }
      cur = new BackOffFailOverSink(tsnk, cur, policy);
    }
    return cur;
  }

View Full Code Here

    Clock.sleep(20);
    c1Src.next();


    // get handle to roller in dfo log roller to provide data
    AgentFailChainSink afcs = (AgentFailChainSink) snk.getSink();
    BackOffFailOverSink bofos = (BackOffFailOverSink) ((CompositeSink) afcs.snk)
        .getSink();
    DiskFailoverDeco dfo = (DiskFailoverDeco) bofos.getBackup();
    DiskFailoverManager dfm = dfo.getFailoverManager();
    RollSink dfoWriter = dfo.getDFOWriter();
    dfoWriter.rotate(); // allow dfo retry thread to go.


    // give data some time to show up.

View Full Code Here

      CommonTree main = backupNodes.get(0);
      CommonTree backup = backupNodes.get(1);
      try {
        EventSink mainSink = buildEventSink(context, main, sinkFactory);
        EventSink backupSink = buildEventSink(context, backup, sinkFactory);
        return new BackOffFailOverSink(mainSink, backupSink);
      } catch (FlumeSpecException ife) {
        LOG.error("Failed to build Failover sink", ife);
        throw ife;
      }
    }

View Full Code Here

    Clock.sleep(20);
    c1Src.next();


    // get handle to roller in dfo log roller to provide data
    AgentFailChainSink afcs = (AgentFailChainSink) snk.getSink();
    BackOffFailOverSink bofos = (BackOffFailOverSink) ((CompositeSink) afcs.snk)
        .getSink();
    DiskFailoverDeco dfo = (DiskFailoverDeco) bofos.getBackup();
    RollSink dfoWriter = dfo.getDFOWriter();
    dfoWriter.rotate(); // allow dfo retry thread to go.


    // give data some time to show up.
    Clock.sleep(1000);

View Full Code Here

      CommonTree main = backupNodes.get(0);
      CommonTree backup = backupNodes.get(1);
      try {
        EventSink mainSink = buildEventSink(context, main, sinkFactory);
        EventSink backupSink = buildEventSink(context, backup, sinkFactory);
        return new BackOffFailOverSink(mainSink, backupSink);
      } catch (FlumeSpecException ife) {
        LOG.error("Failed to build Failover sink", ife);
        throw ife;
      }
    }

View Full Code Here

      CommonTree main = backupNodes.get(0);
      CommonTree backup = backupNodes.get(1);
      try {
        EventSink mainSink = buildEventSink(context, main, sinkFactory);
        EventSink backupSink = buildEventSink(context, backup, sinkFactory);
        return new BackOffFailOverSink(mainSink, backupSink);
      } catch (FlumeSpecException ife) {
        LOG.error("Failed to build Failover sink", ife);
        throw ife;
      }
    }

View Full Code Here

      EventSink tsnk = new CompositeSink(context, failoverSpec);
      if (cur == null) {
        cur = tsnk;
        continue;
      }
      cur = new BackOffFailOverSink(tsnk, cur, policy);
    }
    return cur;
  }

View Full Code Here

TOP

Related Classes of com.cloudera.flume.core.BackOffFailOverSink

com.cloudera.flume.agent.TestAgentFailChainSink

com.cloudera.flume.conf.FlumeBuilder

com.cloudera.flume.conf.SinkFactory.SinkBuilder

com.cloudera.flume.master.availability.FailoverChainSink

com.cloudera.flume.reporter.ReportEvent

com.cloudera.util.CappedExponentialBackoff

java.io.IOException

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.