Examples of SnappyOutputStream

org.iq80.snappy.SnappyOutputStream
This class implements an output stream for writing Snappy compressed data. The output format is the stream header "snappy\0" followed by one or more compressed blocks of data, each of which is preceded by a seven byte header.
The first byte of the header is a flag indicating if the block is compressed or not. A value of 0x00 means uncompressed, and 0x01 means compressed.
The second and third bytes are the size of the block in the stream as a big endian number. This value is never zero as empty blocks are never written. The maximum allowed length is 32k (1 << 15).
The remaining four byes are crc32c checksum of the user input data masked with the following function: {@code ((crc >>> 15) | (crc << 17)) + 0xa282ead8 }
An uncompressed block is simply copied from the input, thus guaranteeing that the output is never larger than the input (not including the header).
org.xerial.snappy.SnappyOutputStream
This class implements a stream filter for writing compressed data using Snappy.
The input data is blocked into 32kb size (in default), and each block is compressed and then passed to the given {@link OutputStream}.
The output data format is:
1. snappy codec header defined in {@link SnappyCodec} (8 bytes)
2. compressed block 1 : a pair of (compressed data size [4 byte integer. Big-endian], compressed data...)
3. compressed block 2
4. ...
Note that the compressed data created by {@link SnappyOutputStream} cannot beuncompressed by {@link Snappy#uncompress(byte[])} since the output formats of{@link Snappy#compress(byte[])} and {@link SnappyOutputStream} are different.Use {@link SnappyInputStream} for uncompress the data generated by{@link SnappyOutputStream}. @author leo

Examples of org.xerial.snappy.SnappyOutputStream

                    out.flush();
                    logger.trace("Upgrading OutputStream to be compressed");
                    if (targetVersion < MessagingService.VERSION_21)
                    {
                        // Snappy is buffered, so no need for extra buffering output stream
                        out = new DataOutputStreamPlus(new SnappyOutputStream(socket.getOutputStream()));
                    }
                    else
                    {
                        // TODO: custom LZ4 OS that supports BB write methods
                        LZ4Compressor compressor = LZ4Factory.fastestInstance().fastCompressor();

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

    BSPMessageCompressor<M> {


  @Override
  public byte[] compress(byte[] bytes) {
    ByteArrayOutputStream bos = null;
    SnappyOutputStream sos = null;
    DataOutputStream dos = null;
    byte[] compressedBytes = null;


    try {
      bos = new ByteArrayOutputStream();
      sos = new SnappyOutputStream(bos);
      dos = new DataOutputStream(sos);


      dos.write(bytes);
      dos.close(); // Flush the stream as no more data will be sent.


      compressedBytes = bos.toByteArray();


    } catch (IOException ioe) {
      LOG.error("Unable to compress", ioe);
    } finally {
      try {
        sos.close();
        bos.close();
      } catch (IOException e) {
        LOG.warn("Failed to close compression streams.", e);
      }
    }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

                    CompactEndpointSerializationHelper.serialize(FBUtilities.getBroadcastAddress(), out);
                    if (shouldCompressConnection())
                    {
                        out.flush();
                        logger.trace("Upgrading OutputStream to be compressed");
                        out = new DataOutputStream(new SnappyOutputStream(new BufferedOutputStream(socket.getOutputStream())));
                    }
                }


                return true;
            }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

                {
                    out.flush();
                    logger.trace("Upgrading OutputStream to be compressed");
                    if (targetVersion < MessagingService.VERSION_21)
                    {
                        out = new DataOutputStream(new SnappyOutputStream(new BufferedOutputStream(socket.getOutputStream())));
                    }
                    else
                    {
                        LZ4Compressor compressor = LZ4Factory.fastestInstance().fastCompressor();
                        Checksum checksum = XXHashFactory.fastestInstance().newStreamingHash32(LZ4_HASH_SEED).asChecksum();

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream


  @Override
  public BSPCompressedBundle compressBundle(BSPMessageBundle<M> bundle) {
    BSPCompressedBundle compMsgBundle = null;
    ByteArrayOutputStream bos = null;
    SnappyOutputStream sos = null;
    DataOutputStream dos = null;


    try {
      bos = new ByteArrayOutputStream();
      sos = new SnappyOutputStream(bos);
      dos = new DataOutputStream(sos);


      bundle.write(dos);
      dos.close(); // Flush the stream as no more data will be sent.


      byte[] data = bos.toByteArray();
      compMsgBundle = new BSPCompressedBundle(data);


    } catch (IOException ioe) {
      LOG.error("Unable to compress", ioe);
    } finally {
      try {
        sos.close();
        bos.close();
      } catch (IOException e) {
        LOG.warn("Failed to close compression streams.", e);
      }
    }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

                    CompactEndpointSerializationHelper.serialize(FBUtilities.getBroadcastAddress(), out);
                    if (shouldCompressConnection())
                    {
                        out.flush();
                        logger.trace("Upgrading OutputStream to be compressed");
                        out = new DataOutputStream(new SnappyOutputStream(new BufferedOutputStream(socket.getOutputStream())));
                    }
                }


                return true;
            }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

                    CompactEndpointSerializationHelper.serialize(FBUtilities.getBroadcastAddress(), out);
                    if (shouldCompressConnection())
                    {
                        out.flush();
                        logger.trace("Upgrading OutputStream to be compressed");
                        out = new DataOutputStream(new SnappyOutputStream(new BufferedOutputStream(socket.getOutputStream())));
                    }
                }


                return true;
            }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

                    CompactEndpointSerializationHelper.serialize(FBUtilities.getBroadcastAddress(), out);
                    if (shouldCompressConnection())
                    {
                        out.flush();
                        logger.trace("Upgrading OutputStream to be compressed");
                        out = new DataOutputStream(new SnappyOutputStream(new BufferedOutputStream(socket.getOutputStream())));
                    }
                }


                return true;
            }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

                    CompactEndpointSerializationHelper.serialize(FBUtilities.getBroadcastAddress(), out);
                    if (shouldCompressConnection())
                    {
                        out.flush();
                        logger.debug("Upgrading OutputStream to be compressed");
                        out = new DataOutputStream(new SnappyOutputStream(new BufferedOutputStream(socket.getOutputStream())));
                    }
                }


                return true;
            }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

                    CompactEndpointSerializationHelper.serialize(FBUtilities.getBroadcastAddress(), out);
                    if (shouldCompressConnection())
                    {
                        out.flush();
                        logger.trace("Upgrading OutputStream to be compressed");
                        out = new DataOutputStream(new SnappyOutputStream(new BufferedOutputStream(socket.getOutputStream())));
                    }
                }


                return true;
            }

View Full Code Here

0 1 2 3

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.