private Ip4 ip = new Ip4(); // Preallocat IP version 4 header private Ip4.Timestamp timestamp = new Ip4.Timestamp(); // Optional header public void nextPacket(PcapPacket packet, Object user) { if (packet.hasHeader(ip) && ip.hasSubHeader(timestamp)) { System.out.println("ip.version=%d\n", ip.version); System.out.println("timestamp optional header length=%d\n", timstamp.length()); } A couple of points about the sub header example. Notice that we preallocated a Timestamp header, which is defined from within Ip4 class itself, but is a separate class on its own none the less. Next we first check if Ip4 header is present at all in the packet, peer it if exists (combined hasHeader and getHeader accessor method) and as a second step we check with the Ip4 header if it has an optional header using ip.hasSubHeader(timestamp). If the method returns true, it also peers the sub header timestamp with the appropriate packet data buffer where the optional header resides. JPacket packet = // From out handler TextFormatter out = new TextFormatter(System.out); out.format(packet); // Send pretty output to stdout // Or save time System.out.println(packet.toString()); // Use internal TextFormatter
Each part of the packet is managed independently, that is either part can be initialized or not. Either part can point to any memory location, including a large single buffer of contigues bytes that contains all 3 parts, header, state and packet data. There are various methods supplied by PcapPacket that allow an external buffer to be peered with all 3 parts of the packet. There are also many methods for transfering (deep copy) the data to and from buffers.
All of these components are stored in native memory in native C structures that are peered with the packet API classes. The classes, managed by JMemory class are referencing native memory locations. Any native method that is called upon in the PcapPacket class or its base classes, will perform those operations on the peered structure and data.
These temporary packets are only suitable for immediate use. That is if the packets are processed immediately when received and then discarded, they do not need to be preserved. If a packet is to be put on a queue and for later processing, the packet needs to preserve its state. That requires a physical copy of all 3 components of the packet to a new memory location. The most efficient way to store the new packet is to allocate a memory buffer large enough to hold all of the packets state and data out of a JMemoryPool. The JPacket provides a default singleton memory pool out of which all packets allocate memory out of for the required space.
PcapPacket.transferTo methods that perform deep copies of the packet. For efficiency reasons, each transferTo method are designed to copy data into a memory buffer of larger size. The packet state and data are copied to the buffer with the following layout within the buffer: +----------+-----+----+ |PcapHeader|State|Data| +----------+-----+----+
The buffer to which this copy takes place can be an external buffer or an internally allocated one by the packet class itself. As stated before, packet's use an interal singleton memory pool to allocate memory out of more efficiently. This memory allocates large native memory blocks which are then sub divided further and given out by the memory pool on a per request basis. All the copies are done natively by low level native copy routines, not in java space for maximum performace.
The easiest way to copy packet contents as received, for example, from PcapPacketHandler, is to pass the temporary packet to the PcapPacket constructor which will automatically allocate new space for the packet state and data and perform a deep copy. The new packet immediately becomes usable and is permanently stored in memory with its state and data, until garbage collected. Here is an example of a PcapPacketHandler that copies the temporary packet to new permanent one:
pulic void nextPacket(PcapPacket packet, Queue<PcapPacket> queue) { PcapPacket permanent = new PcapPacket(packet); queue.offer(permanent); }
Alternative is to reused another packet and transfer the temporary packets state and data to it or create a new unitiatialized packet with new PcapPacket(JMemory.Type.POINTER) constructor and subsequently perform PcapPacket.transferTo(PcapPacket) call to copy the contents. In the first case where an existing packet is being reused, if that packet already contains a large enough memory buffer to hold the state and data of the temporary packet, that buffer is reused. Otherwise a new buffer is allocated out of the default memory pool. Here is an exmaple: *
final PcapPacket permanent = new PcapPacket(Type.POINTER); pulic void nextPacket(PcapPacket packet, Queue<PcapPacket> queue) { permanent.transferStateAndData(packet); // Or packet.transferTo(permanent); } In either case, any existing buffer previously allocated in the permanent packet if its big enough to hold the state and data of the packet, is reused, saving time on memory allocation. You can also manually allocate a large buffer and reuse a packet: final PcapPacket permanent = new PcapPacket(64 * 1024); // Preallocate 64K pulic void nextPacket(PcapPacket packet, Queue<PcapPacket> queue) { permanent.transferStateAndData(packet); // Or packet.transferTo(permanent); } In this example, the packet buffer will always be large enough and resused. But still this is a semi permanentn state. Yet another alternative is to store the contents of the packet in an external buffer such as ByteBuffer, JBuffer or simply a byte[] and then at an appropriate time, transfer the data back or peer the external buffer with a packet object. Only the byte[] buffer type and ByteBuffer backed by a byte array, can not be peered directly with a packet as only buffer sources that are native memory based can be peered. All external buffer types can be copied back into a packet, if peering is not required. New memory space is allocated for the copy. Here is an example:
pulic void nextPacket(PcapPacket packet, Queue<PcapPacket> queue) { JBuffer jbuf = new JBuffer(packet.getTotalSize()); packet.transferTo(jbuf); // Or ByteBuffer bbuf = ByteBuffer.allocateDirect(packet.getTotalSize()); packet.transferTo(bbuf); // Or byte[] babuf = new byte[packet.getTotalSize())]; packet.transferTo(babuf); } In all 3 cases, complete the packet's state and data buffer are copied to external buffer. JMemory class prevents buffer overrun attacks and any access to memory that has not been allocated. a direct reference. Here is an example: pulic void nextPacket(PcapPacket packet, Queue<PcapPacket> queue) { JBuffer jbuf = new JBuffer(packet.getTotalSize()); packet.transferTo(jbuf); // Or ByteBuffer bbuf = ByteBuffer.allocateDirect(packet.getTotalSize()); packet.transferTo(bbuf); // Or byte[] babuf = new byte[packet.getTotalSize())]; packet.transferTo(babuf); PcapPacket p1 = new PcapPacket(jbuf); // Deep copy PcapPacket p2 = new PcapPacket(Type.POINTER); // Uninitialized bbuf.flip(); // Have to flip the buffer to access the just written contents p2.peer(bbuf); // No copies, peered directly with external buffer PcapPacket p3 = new PcapPacket(Type.POINTER); // Uninitialized p3.transferStateAndData(babuf); // Deep copy - byte[] buffers can not be peered PcapPacket p4 = new PcapPacket(Type.POINTER); // Uninitialized p4.peer(p3); // both point at same internal memory space } The above example demonstrates 3 different ways that data from an external buffer can be either copied or peered with a new packet object. In all cases the data and state were transfered from the temporary packet received by the handler to a more permenant buffer and then packet. An interesting scenerio occures with packet p4. Lets take a closer look. First, p3 is created unitialized, meaning that packet header, state and data are null pointers at this time, they don't point to anything and any accessor method used will immediately throw a NullPointerException. Second, the byte[] external buffer is copied into newly allocate memory space by p3. The packet is intiailized to pointer at its internal buffer for the header, state and packet data. Then we create p4, also unitialized and in the following step we peer p4 to p3. That is p4 points at the exact same memory location for packet's header, state and data. No new memory was allocated and changing the contents in either packet, p3 or p4, will have immediate effect on the other packet. Another words, both p3 and p4 are peered to the same internal memory space.
@author Mark Bednarczyk @author Sly Technologies, Inc. @see JMemoryPool | |
| |
| |
| |
| |
| |
| |