Examples of com.google.appengine.tools.mapreduce.impl.sort.SortWorker$IndexedComparator

com.google.appengine.tools.mapreduce.impl.sort.SortWorker

Sorts a set of keyValues by a lexicographical comparison of the bytes of the key. On beginSlice a large buffer is allocated to store the data.

The data is stored in the values buffer. Once this is full the data is sorted in place and written out. The class cannot be used again until beginSlice is called again.

The in place sort is achieved as follows: Each new item is added to the beginning of the buffer, and a pointer is written to the end. Once the buffer is filled the pointers at the end are sorted where the order is determined by comparing the keys they point to. Once they are ordered the data can be read out by reading through the pointers and emitting the value that corresponds to them.

This class is NOT threadSafe.

      for (int i = 0; i < readers.size(); i++) {
        sortTasks.add(new SortShardTask(mrJobId,
            i,
            readers.size(),
            readers.get(i),
            new SortWorker(settings.getMaxSortMemory(), settings.getSortBatchPerEmitBytes()),
            writers.get(i),
            settings.getSortReadTimeMillis()));
      }
      ShardedJobSettings shardedJobSettings =
          settings.toShardedJobSettings(shardedJobId, getPipelineKey());

Examples of com.google.appengine.tools.mapreduce.impl.sort.SortWorker$IndexedComparator

Related Classes of com.google.appengine.tools.mapreduce.impl.sort.SortWorker$IndexedComparator