Examples of eu.stratosphere.core.memory.MemorySegment

eu.stratosphere.core.memory.MemorySegment
This class represents a piece of memory allocated from the memory manager. The segment is backed by a byte array and features random put and get methods for the basic types that are stored in a byte-wise fashion in the memory.
Comments on the implementation: We make heavy use of operations that are supported by native instructions, to achieve a high efficiency. Multi byte types (int, long, float, double, ...) are read and written with "unsafe" native commands. Little-endian to big-endian conversion and vice versa are done using the static reverseBytes methods in the boxing data types (for example {@link Integer#reverseBytes(int)}). On x86/amd64, these are translated by the jit compiler to bswap intrinsic commands. Below is an example of the code generated for the {@link MemorySegment#putLongBigEndian(int,long)}function by the just-in-time compiler. The code is grabbed from an oracle jvm 7 using the hotspot disassembler library (hsdis32.dll) and the jvm command -XX:+UnlockDiagnosticVMOptions -XX:CompileCommand=print,*UnsafeMemorySegment.putLongBigEndian. Note that this code realizes both the byte order swapping and the reinterpret cast access to get a long from the byte array.
```
 [Verified Entry Point] 0x00007fc403e19920: sub    $0x18,%rsp 0x00007fc403e19927: mov    %rbp,0x10(%rsp)    ;*synchronization entry ; - eu.stratosphere.nephele.services.memorymanager.UnsafeMemorySegment::putLongBigEndian@-1 (line 652) 0x00007fc403e1992c: mov    0xc(%rsi),%r10d    ;*getfield memory ; - eu.stratosphere.nephele.services.memorymanager.UnsafeMemorySegment::putLong@4 (line 611) ; - eu.stratosphere.nephele.services.memorymanager.UnsafeMemorySegment::putLongBigEndian@12 (line 653) 0x00007fc403e19930: bswap  %rcx 0x00007fc403e19933: shl    $0x3,%r10 0x00007fc403e19937: movslq %edx,%r11 0x00007fc403e1993a: mov    %rcx,0x10(%r10,%r11,1)  ;*invokevirtual putLong ; - eu.stratosphere.nephele.services.memorymanager.UnsafeMemorySegment::putLong@14 (line 611) ; - eu.stratosphere.nephele.services.memorymanager.UnsafeMemorySegment::putLongBigEndian@12 (line 653) 0x00007fc403e1993f: add    $0x10,%rsp 0x00007fc403e19943: pop    %rbp 0x00007fc403e19944: test   %eax,0x5ba76b6(%rip)        # 0x00007fc4099c1000 ;   {poll_return} 0x00007fc403e1994a: retq  
```

    Envelope env = new Envelope(random.nextInt(), new JobID(), new ChannelID());
    if (bufferSize > 0) {
      byte[] data = new byte[bufferSize];
      random.nextBytes(data);


      env.setBuffer(spy(new Buffer(new MemorySegment(data), bufferSize, RECYCLER)));
    }


    if (events != null && events.length > 0) {
      env.serializeEventList(Arrays.asList(events));
    }

View Full Code Here

  
  @SuppressWarnings("unchecked")
  private <T extends Comparable<T>> void assertNormalizableKey(NormalizableKey<T> key1, NormalizableKey<T> key2, int len) {
    
    byte[] normalizedKeys = new byte[2*len];
    MemorySegment wrapper = new MemorySegment(normalizedKeys);
    
    key1.copyNormalizedKey(wrapper, 0, len);
    key2.copyNormalizedKey(wrapper, len, len);
    
    for (int i = 0; i < len; i++) {

View Full Code Here


  // --------------------------------- Normalized key tests -------------------------------------
  
  // Help Function for setting up a memory segment and normalize the keys of the data array in it
  public MemorySegment setupNormalizedKeysMemSegment(T[] data, int normKeyLen, TypeComparator<T> comparator) {
    MemorySegment memSeg = new MemorySegment(new byte[2048]);


    // Setup normalized Keys in the memory segment
    int offset = 0;
    for (T e : data) {
      comparator.putNormalizedKey(e, memSeg, offset, normKeyLen);

View Full Code Here

    try {
      TypeComparator<T> comparator = getComparator(true);
      T[] data = getSortedData();
      int normKeyLen = getNormKeyLen(halfLength, data, comparator);


      MemorySegment memSeg1 = setupNormalizedKeysMemSegment(data, normKeyLen, comparator);
      MemorySegment memSeg2 = setupNormalizedKeysMemSegment(data, normKeyLen, comparator);


      for (int i = 0; i < data.length; i++) {
        assertTrue(MemorySegment.compare(memSeg1, memSeg2, i * normKeyLen, i * normKeyLen, normKeyLen) == 0);
      }
    } catch (Exception e) {

View Full Code Here

      T[] data = getSortedData();
      // Get the normKeyLen on which we are testing
      int normKeyLen = getNormKeyLen(halfLength, data, comparator);


      // Write the data into different 2 memory segements
      MemorySegment memSegLow = setupNormalizedKeysMemSegment(data, normKeyLen, comparator);
      MemorySegment memSegHigh = setupNormalizedKeysMemSegment(data, normKeyLen, comparator);


      boolean fullyDetermines = !comparator.isNormalizedKeyPrefixOnly(normKeyLen);
      
      // Compare every element with every bigger element
      for (int l = 0; l < data.length - 1; l++) {

View Full Code Here

0 1 2 3 4 5 6

TOP

Related Classes of eu.stratosphere.core.memory.MemorySegment

eu.stratosphere.api.common.typeutils.ComparatorTestBase

eu.stratosphere.nephele.services.iomanager.BlockChannelReader

eu.stratosphere.nephele.services.iomanager.BlockChannelWriter

eu.stratosphere.nephele.services.iomanager.ChannelReaderInputView

eu.stratosphere.nephele.services.iomanager.ChannelWriterOutputView

eu.stratosphere.nephele.services.iomanager.IOManagerTest

eu.stratosphere.nephele.services.memorymanager.spi.DefaultMemoryManager

eu.stratosphere.nephele.util.TestBufferProvider

eu.stratosphere.pact.runtime.hash.CompactingHashTable

eu.stratosphere.pact.runtime.hash.CompactingHashTable$EntryIterator

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.