Examples of com.yahoo.ycsb.ByteIterator

com.yahoo.ycsb.ByteIterator
YCSB-specific buffer class. ByteIterators are designed to support efficient field generation, and to allow backend drivers that can stream fields (instead of materializing them in RAM) to do so.
YCSB originially used String objects to represent field values. This led to two performance issues.
First, it leads to unnecessary conversions between UTF-16 and UTF-8, both during field generation, and when passing data to byte-based backend drivers.
Second, Java strings are represented internally using UTF-16, and are built by appending to a growable array type (StringBuilder or StringBuffer), then calling a toString() method. This leads to a 4x memory overhead as field values are being built, which prevented YCSB from driving large object stores.
The StringByteIterator class contains a number of convenience methods for backend drivers that convert between Map<String,String> and Map<String,ByteBuffer>. @author sears

                array[i] = (byte)k.charAt(j);
                i++;
            }
            array[i] = '\t'; // XXX would like to use sane delimiter (null, 254, 255, ...) but java makes this nearly impossible
            i++;
            ByteIterator v = values.get(k);
            i = v.nextBuf(array, i);
            array[i] = '\t';
            i++;
        }
        array[array.length-1] = 0;
        ByteBuffer buf = ByteBuffer.wrap(array);

View Full Code Here

        {
            byte[] k = key.getBytes(UTF8);
            m_writeBuf.putInt(k.length);
            m_writeBuf.put(k);


            ByteIterator v = columns.get(key);
            int len = (int) v.bytesLeft();
            m_writeBuf.putInt(len);
            v.nextBuf(m_workingData, m_writeBuf.position());
            m_writeBuf.position(m_writeBuf.position() + len);
        }


        byte[] data = new byte[m_writeBuf.position()];
        System.arraycopy(m_workingData, 0, data, 0, data.length);

View Full Code Here

        HashMap<String, ByteIterator> values = new HashMap<String, ByteIterator>();


        for (int i = 0; i < 4; i++)
        {
            String fieldkey = "field" + i;
            ByteIterator data = new RandomByteIterator(fieldlengthgenerator.nextInt());
            values.put(fieldkey, data);
        }
        return values;
    }

View Full Code Here

                System.out.print("Reading key: " + key);
            }


            Column column;
            String name;
            ByteIterator value;
            
            assert results != null && !results.isEmpty();
/*            for (ColumnOrSuperColumn oneresult : results)
            {

View Full Code Here

            {
                tuple = new HashMap<String, ByteIterator>();


                Column column;
                String name;
                ByteIterator value;
                for (ColumnOrSuperColumn onecol : oneresult.columns)
                {
                    column = onecol.column;
                    name = new String(column.name.array(), column.name.position() + column.name.arrayOffset(),
                            column.name.remaining());

View Full Code Here

TOP

Related Classes of com.yahoo.ycsb.ByteIterator

com.impetus.kundera.ycsb.benchmark.OracleNosqlClient

com.impetus.kundera.ycsb.benchmark.ThriftClient

com.yahoo.ycsb.db.MapKeeperClient

com.yahoo.ycsb.db.VoltClient4

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.