This class represents a Databus event stored in a {@link java.nio.ByteBuffer}.
Binary Serialization Format
The table below summarizes the serialization format of Databus events as stored into memory and sent over the wire.
| Field | Offset | Type | Size | Description |
| Header (61 bytes for events with long keys, 57 bytes + key length) |
| Version | 0 | byte | 1 | A value denoting the version of the message format |
| HeaderCrc | 1 | int | 4 | CRC computed over the header |
| Length | 5 | int | 4 | Total length of the event in bytes (fixed-length header + variable-length payload) |
| Attributes | 9 | short | 2 | Event attributes bitmap (see below) |
| Sequence | 11 | long | 8 | Sequence number for the event window in which this event was generated |
| PhysicalPartitionId | 19 | short | 2 | physical partition-id -> represents a sequence generator |
| LogicalPartitionId | 21 | short | 2 | logical partition-id -> represents a logical partition of the physical stream |
| NanoTimestamp | 23 | long | 8 | Time (in nanoseconds) at which the event was generated |
| srcId | 31 | short | 2 | Databus source id for the event |
| SchemaId* | 33 | byte[] | 16 | hash for the schema used to generate the event |
| ValueCrc | 49 | int | 4 | a CRC computed over the variable-length payload of the event |
| Key | 53 | long | 8 | key value for events with long keys |
| KeySize | 53 | int | 4 | key length for byte[]-valued keys |
| Variable-size payload |
| Key | 57 | byte[] | 4 or KeySize | 32 LSB from key value for long-typed keys or key value for byte[] keys |
| Value | 61 or 57 + KeySize | byte[] | Length - Offset(Value) | Serialized event payload |
* For REPL_DBUS we are using schemaid field to pass Schema Version. So in this case SchemaId row of the table should be replaced with:
| SchemaVersion | 33 | short | 2 | schema version for the event |
| SchemaId(not valid) | 35 | byte[] | 14 | hash for the schema used to generate the event |
JSON serialization format
The table below summarizes the JSON serialization format.
| Attribute | Type | Description | Optional |
| opcode | String | UPSERT or DELETE | Yes |
| keyBytes | String | Base-64 encoding of the key byte sequence for string keys | One of the two needs to be present |
| key | Long | key value for numeric keys |
| sequence | Long | event sequence number | No |
| logicalPartitionId | Short | logical partition id | No |
| physicalPartitionId | Short | physical partition id | No |
| timestampInNanos | Long | creation timestamp in nanoseconds since the Unix Epoch | No |
| srcId | Short | source id | No |
| schemaId | String | Base-64 encoding of the event serialization schema hash id | No |
| valueEnc | String | value encoding format: JSON or JSON_PLAIN | No |
| endOfPeriod | Boolean | true iff the event denotes end of event window | Yes; default is false |
| value | String | Literal value string for JSON_PLAIN encoding or Base-64 encoding of the value byte sequence for JSON encoding | Yes; default is false |
Event attributes
The table below summarizes the Databus event attribute bits
| Attribute | Bit N | Description |
| OpCode0 | 0 | Bit 0 of event opcode |
| OpCode1 | 1 | bit 1 of event opcode |
| Trace | 2 | The event is a trace event |
| ByteKey | 3 | The event has a byte[] key |
| EoP | 4 | The event is the last event in a event window |
| ExtReplEvent | 8 | The event was generated through external replication (e.g. originating in a different data center) |
Event opcodes
Currently, Databus supports two choices of event opcodes
Databus source ids
The possible values for Databus source ids are partitioned into several ranges. In general, all positive source ids are used to uniquely represent a Databus source. The source ids are used in Databus data messages. All non-positive values are reserved for Databus system use. These source ids are used in Databus control messages.
- [1, {@link java.lang.Short.MAX_VALUE}] - data source ids
- [ {@link java.lang.Short.MIN_VALUE}, 0] - system source ids
- [ {@link com.linkedin.databus.core.DbusEventInternalWritable#PRIVATE_RANGE_MAX_SRCID} + 1, 0] - global system source ids. These controlmessages will be transmitted to other Databus components over the network
- [ {@link java.lang.Short.MIN_VALUE}, {@link com.linkedin.databus.core.DbusEventInternalWritable#PRIVATE_RANGE_MAX_SRCID}] - private system source ids. These messages are used for internal communication inside a Databus component and are not transmitted to other Databus components.