Examples of TableMapLogEvent

com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

In row-based mode, every row operation event is preceded by a Table_map_log_event which maps a table definition to a number. The table definition consists of database name, table name, and column definitions. The Post-Header has the following components:

Post-Header for Table_map_log_event
Name	Format	Description
table_id	6 bytes unsigned integer	The number that identifies the table.
flags	2 byte bitfield	Reserved for future use; currently always 0.

The Body has the following components:

Body for Table_map_log_event
Name	Format	Description
database_name	one byte string length, followed by null-terminated string	The name of the database in which the table resides. The name is represented as a one byte unsigned integer representing the number of bytes in the name, followed by length bytes containing the database name, followed by a terminating 0 byte. (Note the redundancy in the representation of the length.)
table_name	one byte string length, followed by null-terminated string	The name of the table, encoded the same way as the database name above.
column_count	packed_integer "Packed Integer"	The number of columns in the table, represented as a packed variable-length integer.
column_type	List of column_count 1 byte enumeration values	The type of each column in the table, listed from left to right. Each byte is mapped to a column type according to the enumeration type enum_field_types defined in mysql_com.h. The mapping of types to numbers is listed in the table Table_table_map_log_event_column_types "below" (along with description of the associated metadata field).
metadata_length	packed_integer "Packed Integer"	The length of the following metadata block
metadata	list of metadata for each column	For each column from left to right, a chunk of data who's length and semantics depends on the type of the column. The length and semantics for the metadata for each column are listed in the table Table_table_map_log_event_column_types "below".
null_bits	column_count bits, rounded up to nearest byte	For each column, a bit indicating whether data in the column can be NULL or not. The number of bytes needed for this is int((column_count+7)/8). The flag for the first column from the left is in the least-significant bit of the first byte, the second is in the second least significant bit of the first byte, the ninth is in the least significant bit of the second byte, and so on.

The table below lists all column types, along with the numerical identifier for it and the size and interpretation of meta-data used to describe the type.

Table_map_log_event column types: numerical identifier and metadata
Name	Identifier	Size of metadata in bytes	Description of metadata
MYSQL_TYPE_DECIMAL	0	0	No column metadata.
MYSQL_TYPE_TINY	1	0	No column metadata.
MYSQL_TYPE_SHORT	2	0	No column metadata.
MYSQL_TYPE_LONG	3	0	No column metadata.
MYSQL_TYPE_FLOAT	4	1 byte	1 byte unsigned integer, representing the "pack_length", which is equal to sizeof(float) on the server from which the event originates.
MYSQL_TYPE_DOUBLE	5	1 byte	1 byte unsigned integer, representing the "pack_length", which is equal to sizeof(double) on the server from which the event originates.
MYSQL_TYPE_NULL	6	0	No column metadata.
MYSQL_TYPE_TIMESTAMP	7	0	No column metadata.
MYSQL_TYPE_LONGLONG	8	0	No column metadata.
MYSQL_TYPE_INT24	9	0	No column metadata.
MYSQL_TYPE_DATE	10	0	No column metadata.
MYSQL_TYPE_TIME	11	0	No column metadata.
MYSQL_TYPE_DATETIME	12	0	No column metadata.
MYSQL_TYPE_YEAR	13	0	No column metadata.
MYSQL_TYPE_NEWDATE	14	–	This enumeration value is only used internally and cannot exist in a binlog.
MYSQL_TYPE_VARCHAR	15	2 bytes	2 byte unsigned integer representing the maximum length of the string.
MYSQL_TYPE_BIT	16	2 bytes	A 1 byte unsigned int representing the length in bits of the bitfield (0 to 64), followed by a 1 byte unsigned int representing the number of bytes occupied by the bitfield. The number of bytes is either int((length+7)/8) or int(length/8).
MYSQL_TYPE_NEWDECIMAL	246	2 bytes	A 1 byte unsigned int representing the precision, followed by a 1 byte unsigned int representing the number of decimals.
MYSQL_TYPE_ENUM	247	–	This enumeration value is only used internally and cannot exist in a binlog.
MYSQL_TYPE_SET	248	–	This enumeration value is only used internally and cannot exist in a binlog.
MYSQL_TYPE_TINY_BLOB	249	–	This enumeration value is only used internally and cannot exist in a binlog.
MYSQL_TYPE_MEDIUM_BLOB	250	–	This enumeration value is only used internally and cannot exist in a binlog.
MYSQL_TYPE_LONG_BLOB	251	–	This enumeration value is only used internally and cannot exist in a binlog.
MYSQL_TYPE_BLOB	252	1 byte	The pack length, i.e., the number of bytes needed to represent the length of the blob: 1, 2, 3, or 4.
MYSQL_TYPE_VAR_STRING	253	2 bytes	This is used to store both strings and enumeration values. The first byte is a enumeration value storing the real type, which may be either MYSQL_TYPE_VAR_STRING or MYSQL_TYPE_ENUM. The second byte is a 1 byte unsigned integer representing the field size, i.e., the number of bytes needed to store the length of the string.
MYSQL_TYPE_STRING	254	2 bytes	The first byte is always MYSQL_TYPE_VAR_STRING (i.e., 253). The second byte is the field size, i.e., the number of bytes in the representation of size of the string: 3 or 4.
MYSQL_TYPE_GEOMETRY	255	1 byte	The pack length, i.e., the number of bytes needed to represent the length of the geometry: 1, 2, 3, or 4.

@author Changyuan.lh @version 1.0

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = getSchemaNameAndTableName(table);
            if (nameFilter != null && !nameFilter.filter(fullname)) { // check
                                                                      // name
                                                                      // filter
                return null;
            }
            EventType eventType = null;
            int type = event.getHeader().getType();
            if (LogEvent.WRITE_ROWS_EVENT_V1 == type || LogEvent.WRITE_ROWS_EVENT == type) {
                eventType = EventType.INSERT;
            } else if (LogEvent.UPDATE_ROWS_EVENT_V1 == type || LogEvent.UPDATE_ROWS_EVENT == type) {
                eventType = EventType.UPDATE;
            } else if (LogEvent.DELETE_ROWS_EVENT_V1 == type || LogEvent.DELETE_ROWS_EVENT == type) {
                eventType = EventType.DELETE;
            } else {
                throw new CanalParseException("unsupport event type :" + event.getHeader().getType());
            }


            Header header = createHeader(binlogFileName,
                event.getHeader(),
                table.getDbName(),
                table.getTableName(),
                eventType);
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = getSchemaNameAndTableName(table);
            if (nameFilter != null && !nameFilter.filter(fullname)) { // check name filter
                return null;
            }
            EventType eventType = null;
            int type = event.getHeader().getType();
            if (LogEvent.WRITE_ROWS_EVENT_V1 == type || LogEvent.WRITE_ROWS_EVENT == type) {
                eventType = EventType.INSERT;
            } else if (LogEvent.UPDATE_ROWS_EVENT_V1 == type || LogEvent.UPDATE_ROWS_EVENT == type) {
                eventType = EventType.UPDATE;
            } else if (LogEvent.DELETE_ROWS_EVENT_V1 == type || LogEvent.DELETE_ROWS_EVENT == type) {
                eventType = EventType.DELETE;
            } else {
                throw new CanalParseException("unsupport event type :" + event.getHeader().getType());
            }


            Header header = createHeader(binlogFileName, event.getHeader(), table.getDbName(), table.getTableName(),
                                         eventType);
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = getSchemaNameAndTableName(table);
            if (nameFilter != null && !nameFilter.filter(fullname)) { // check name filter
                return null;
            }


            Header header = createHeader(binlogFileName, event.getHeader(), table.getDbName(), table.getTableName());
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);
            EventType eventType = null;
            if (LogEvent.WRITE_ROWS_EVENT == event.getHeader().getType()) {

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = getSchemaNameAndTableName(table);
            if (nameFilter != null && !nameFilter.filter(fullname)) { // check name filter
                return null;
            }


            Header header = createHeader(binlogFileName, event.getHeader(), table.getDbName(), table.getTableName());
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);
            EventType eventType = null;
            if (LogEvent.WRITE_ROWS_EVENT == event.getHeader().getType()) {

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = getSchemaNameAndTableName(table);
            // check name filter
            if (nameFilter != null && !nameFilter.filter(fullname)) {
                return null;
            }
            EventType eventType = null;
            int type = event.getHeader().getType();
            if (LogEvent.WRITE_ROWS_EVENT_V1 == type || LogEvent.WRITE_ROWS_EVENT == type) {
                eventType = EventType.INSERT;
            } else if (LogEvent.UPDATE_ROWS_EVENT_V1 == type || LogEvent.UPDATE_ROWS_EVENT == type) {
                eventType = EventType.UPDATE;
            } else if (LogEvent.DELETE_ROWS_EVENT_V1 == type || LogEvent.DELETE_ROWS_EVENT == type) {
                eventType = EventType.DELETE;
            } else {
                throw new CanalParseException("unsupport event type :" + event.getHeader().getType());
            }


            Header header = createHeader(binlogFileName, event.getHeader(), table.getDbName(), table.getTableName(),
                                         eventType);
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = getSchemaNameAndTableName(table);
            if (nameFilter != null && !nameFilter.filter(fullname)) { // check name filter
                return null;
            }


            Header header = createHeader(binlogFileName, event.getHeader(), table.getDbName(), table.getTableName());
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);
            EventType eventType = null;
            int type = event.getHeader().getType();

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = table.getDbName() + "." + table.getTableName();
            // check name filter
            if (nameFilter != null && !nameFilter.filter(fullname)) {
                return null;
            }
            EventType eventType = null;
            int type = event.getHeader().getType();
            if (LogEvent.WRITE_ROWS_EVENT_V1 == type || LogEvent.WRITE_ROWS_EVENT == type) {
                eventType = EventType.INSERT;
            } else if (LogEvent.UPDATE_ROWS_EVENT_V1 == type || LogEvent.UPDATE_ROWS_EVENT == type) {
                eventType = EventType.UPDATE;
            } else if (LogEvent.DELETE_ROWS_EVENT_V1 == type || LogEvent.DELETE_ROWS_EVENT == type) {
                eventType = EventType.DELETE;
            } else {
                throw new CanalParseException("unsupport event type :" + event.getHeader().getType());
            }


            Header header = createHeader(binlogFileName,
                event.getHeader(),
                table.getDbName(),
                table.getTableName(),
                eventType);
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);


            rowChangeBuider.setEventType(eventType);
            RowsLogBuffer buffer = event.getRowsBuf(charset.name());
            BitSet columns = event.getColumns();
            BitSet changeColumns = event.getColumns();
            TableMeta tableMeta = null;
            if (tableMetaCache != null) {// 入错存在table meta cache
                tableMeta = tableMetaCache.getTableMeta(table.getDbName(), table.getTableName());
                if (tableMeta == null) {
                    throw new CanalParseException("not found [" + fullname + "] in db , pls check!");
                }
            }

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

                logPosition.position = header.getLogPos();
                return event;
            }
        case LogEvent.TABLE_MAP_EVENT:
            {
                TableMapLogEvent mapEvent = new TableMapLogEvent(header,
                        buffer, descriptionEvent);
                /* updating position in context */
                logPosition.position = header.getLogPos();
                context.putTable(mapEvent);
                return mapEvent;

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = getSchemaNameAndTableName(table);
            if (nameFilter != null && !nameFilter.filter(fullname)) { // check name filter
                return null;
            }
            EventType eventType = null;
            int type = event.getHeader().getType();
            if (LogEvent.WRITE_ROWS_EVENT_V1 == type || LogEvent.WRITE_ROWS_EVENT == type) {
                eventType = EventType.INSERT;
            } else if (LogEvent.UPDATE_ROWS_EVENT_V1 == type || LogEvent.UPDATE_ROWS_EVENT == type) {
                eventType = EventType.UPDATE;
            } else if (LogEvent.DELETE_ROWS_EVENT_V1 == type || LogEvent.DELETE_ROWS_EVENT == type) {
                eventType = EventType.DELETE;
            } else {
                throw new CanalParseException("unsupport event type :" + event.getHeader().getType());
            }


            Header header = createHeader(binlogFileName, event.getHeader(), table.getDbName(), table.getTableName(),
                                         eventType);
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);

View Full Code Here

Examples of com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent

        return createEntry(header, EntryType.TRANSACTIONEND, transactionEnd.toByteString());
    }


    private Entry parseRowsEvent(RowsLogEvent event) {
        try {
            TableMapLogEvent table = event.getTable();
            if (table == null) {
                // tableId对应的记录不存在
                throw new TableIdNotFoundException("not found tableId:" + event.getTableId());
            }


            String fullname = table.getDbName() + "." + table.getTableName();
            // check name filter
            if (nameFilter != null && !nameFilter.filter(fullname)) {
                return null;
            }
            if (nameBlackFilter != null && nameBlackFilter.filter(fullname)) {
                return null;
            }


            EventType eventType = null;
            int type = event.getHeader().getType();
            if (LogEvent.WRITE_ROWS_EVENT_V1 == type || LogEvent.WRITE_ROWS_EVENT == type) {
                eventType = EventType.INSERT;
            } else if (LogEvent.UPDATE_ROWS_EVENT_V1 == type || LogEvent.UPDATE_ROWS_EVENT == type) {
                eventType = EventType.UPDATE;
            } else if (LogEvent.DELETE_ROWS_EVENT_V1 == type || LogEvent.DELETE_ROWS_EVENT == type) {
                eventType = EventType.DELETE;
            } else {
                throw new CanalParseException("unsupport event type :" + event.getHeader().getType());
            }


            Header header = createHeader(binlogFileName,
                event.getHeader(),
                table.getDbName(),
                table.getTableName(),
                eventType);
            RowChange.Builder rowChangeBuider = RowChange.newBuilder();
            rowChangeBuider.setTableId(event.getTableId());
            rowChangeBuider.setIsDdl(false);


            rowChangeBuider.setEventType(eventType);
            RowsLogBuffer buffer = event.getRowsBuf(charset.name());
            BitSet columns = event.getColumns();
            BitSet changeColumns = event.getColumns();
            boolean tableError = false;
            TableMeta tableMeta = null;
            if (tableMetaCache != null) {// 入错存在table meta cache
                tableMeta = getTableMeta(table.getDbName(), table.getTableName(), true);
                if (tableMeta == null) {
                    tableError = true;
                    if (!filterTableError) {
                        throw new CanalParseException("not found [" + fullname + "] in db , pls check!");
                    }

View Full Code Here

0 1

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.