Examples of SnappyOutputStream

org.iq80.snappy.SnappyOutputStream
This class implements an output stream for writing Snappy compressed data. The output format is the stream header "snappy\0" followed by one or more compressed blocks of data, each of which is preceded by a seven byte header.
The first byte of the header is a flag indicating if the block is compressed or not. A value of 0x00 means uncompressed, and 0x01 means compressed.
The second and third bytes are the size of the block in the stream as a big endian number. This value is never zero as empty blocks are never written. The maximum allowed length is 32k (1 << 15).
The remaining four byes are crc32c checksum of the user input data masked with the following function: {@code ((crc >>> 15) | (crc << 17)) + 0xa282ead8 }
An uncompressed block is simply copied from the input, thus guaranteeing that the output is never larger than the input (not including the header).
org.xerial.snappy.SnappyOutputStream
This class implements a stream filter for writing compressed data using Snappy.
The input data is blocked into 32kb size (in default), and each block is compressed and then passed to the given {@link OutputStream}.
The output data format is:
1. snappy codec header defined in {@link SnappyCodec} (8 bytes)
2. compressed block 1 : a pair of (compressed data size [4 byte integer. Big-endian], compressed data...)
3. compressed block 2
4. ...
Note that the compressed data created by {@link SnappyOutputStream} cannot beuncompressed by {@link Snappy#uncompress(byte[])} since the output formats of{@link Snappy#compress(byte[])} and {@link SnappyOutputStream} are different.Use {@link SnappyInputStream} for uncompress the data generated by{@link SnappyOutputStream}. @author leo

Examples of org.xerial.snappy.SnappyOutputStream


    @Test
    public void snappyCompress() throws IOException
    {
        FileInputStream fi = new FileInputStream("/tmp/compress-test.txt");
        SnappyOutputStream out = new SnappyOutputStream(new BufferedOutputStream(new FileOutputStream("/tmp/test0.snp")));
        BufferedInputStream origin = new BufferedInputStream(fi, 1024);
        byte data[] = new byte[1024];
        int count;
        while ((count = origin.read(data, 0, 1024)) != -1)
        {
            out.write(data, 0, count);
        }
        IOUtils.closeQuietly(origin);
        IOUtils.closeQuietly(fi);
        IOUtils.closeQuietly(out);

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream


    public ChunkedStream(InputStream is, long chunkSize) throws IOException
    {
        this.origin = is;
        this.bos = new ByteArrayOutputStream();
        this.compress = new SnappyOutputStream(bos);
        this.chunkSize = chunkSize;
    }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

        if (s == null) {
            File f = new VoltFile(path, name);
            f.delete();
            RandomAccessFile ras = new RandomAccessFile(f, "rw");
            ras.seek(8);
            SnappyOutputStream sos = new SnappyOutputStream(new FileOutputStream(ras.getFD()));
            DataOutputStream dos = new DataOutputStream(sos);
            s = new Stuff();
            s.dos = dos;
            s.count = ras.getChannel().map(MapMode.READ_WRITE, 0, 8);
            s.count.order(ByteOrder.nativeOrder());

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream


  @Override
  public BSPCompressedBundle compressBundle(BSPMessageBundle<M> bundle) {
    BSPCompressedBundle compMsgBundle = null;
    ByteArrayOutputStream bos = null;
    SnappyOutputStream sos = null;
    DataOutputStream dos = null;


    try {
      bos = new ByteArrayOutputStream();
      sos = new SnappyOutputStream(bos);
      dos = new DataOutputStream(sos);


      bundle.write(dos);
      dos.close(); // Flush the stream as no more data will be sent.


      byte[] data = bos.toByteArray();
      compMsgBundle = new BSPCompressedBundle(data);


    } catch (IOException ioe) {
      LOG.error("Unable to compress", ioe);
    } finally {
      try {
        sos.close();
        bos.close();
      } catch (IOException e) {
        LOG.warn("Failed to close compression streams.", e);
      }
    }

View Full Code Here

Examples of org.xerial.snappy.SnappyOutputStream

        if (obj == null) {
            return null;
        }
        
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        SnappyOutputStream snappy;
        try {
            snappy = new SnappyOutputStream(out);
            snappy.write(StringUtils.getBytesUtf8(obj));
            snappy.close();
            return ByteBuffer.wrap(out.toByteArray());
        } catch (IOException e) {
            throw new RuntimeException("Error compressing column data", e);
        }        
    }

View Full Code Here

0 1 2 3

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.