Examples of com.mapr.franz.server.ProtoLogger

com.mapr.franz.server.ProtoLogger
Logs topics to files that are of limited size using a protobuf envelope format. Below the base directory of the logger, each topic has its own directory named after the topic. Each of these directories contains files whose names are the offset from the beginning of the topic stream expressed in hex. As each file reaches maximum size, the logger moves to the next file. If we find existing files on startup, the previous process who was writing to this topic may still have some pending writes. To avoid problems with this, we never open an existing file and always leave a bit of a gap in stream offset when opening the next file unless we know that we were the previous writer.


    // TODO implement some sort of statistics that records (a) number of
    // clients, (b) transactions per topic, (c) bytes per topic


    public SimpleCatcherService(int port, String basePath) throws FileNotFoundException, SocketException {
        logger = new ProtoLogger(basePath);
        recorder = new History(1, 10, 60, 300)
                .logTicks()
                .addListener(new History.Listener() {
                    @Override
                    public void tick(double t, int interval, int uniqueTopics, int messages) {

View Full Code Here

        // this is evil because it prevents running more than one server per JVM
        CatcherImpl.serverId = us.getProto().getServerId();
        this.us = us;
        this.instance = instance;
        this.basePath = basePath;
        logger = new ProtoLogger(this.basePath);
    }

View Full Code Here

    @Test
    public void testFileRollover() throws IOException {
        Path homeDir = Files.createTempDirectory("logger");
        Path statusPath = Files.createTempFile("status", "dat");


        ProtoLogger p = new ProtoLogger(homeDir.toString());
        p.setMaxLogFile(500);


        for (int i = 0; i < 1000; i++) {
            p.write("topic-1", ByteString.copyFromUtf8(i + ""));
        }
        p.close();




        ProtoSpout ps = new ProtoSpout(new ProtoSpout.TupleParser() {
            @Override
            public List<Object> parse(ByteString buffer) {

View Full Code Here

TOP

Related Classes of com.mapr.franz.server.ProtoLogger

com.mapr.franz.hazel.CatcherImpl

com.mapr.franz.simple.SimpleCatcherService

com.mapr.ProtoSpoutTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.