Examples of com.hazelcast.mapreduce.Reducer

com.hazelcast.mapreduce.Reducer
The abstract Reducer class is used to build reducers for the {@link Job}.
Reducers may be distributed inside of the cluster but there is always only one Reducer per key.
Reducers are called in a threadsafe way so internal locking is not required.

Due to the fact that there is only one Reducer per key mapped values needs to be transmitted to one of the cluster nodes. To reduce the traffic costs between the nodes a {@link Combiner} implementation can be added to the call which runs alongsidethe mapper to pre-reduce mapped values into intermediate results.

A simple Reducer implementation could look like that sum-function implementation:
```
 public class SumReducer implements Reducer<String, Integer, Integer> { private int sum = 0; public void reduce( String key, Integer value ) { sum += value; } public Integer finalizeReduce() { return sum; } } 
```
@param < KeyIn> key type of the resulting keys @param < ValueIn> value type of the incoming values @param < ValueOut> value type of the reduced values @since 3.2

        }
    }


    private void reduceChunk(Map<Key, Chunk> chunk) {
        for (Map.Entry<Key, Chunk> entry : chunk.entrySet()) {
            Reducer reducer = supervisor.getReducerByKey(entry.getKey());
            if (reducer != null) {
                Chunk chunkValue = entry.getValue();
                if (chunkValue instanceof List) {
                    for (Object value : (List) chunkValue) {
                        reducer.reduce(value);
                    }
                } else {
                    reducer.reduce(chunkValue);
                }
            }
        }
    }

View Full Code Here

        }
        return result;
    }


    public <KeyIn, ValueIn, ValueOut> Reducer<KeyIn, ValueIn, ValueOut> getReducerByKey(Object key) {
        Reducer reducer = reducers.get(key);
        if (reducer == null && configuration.getReducerFactory() != null) {
            reducer = configuration.getReducerFactory().newReducer(key);
            Reducer oldReducer = reducers.putIfAbsent(key, reducer);
            if (oldReducer != null) {
                reducer = oldReducer;
            } else {
                reducer.beginReduce(key);
            }

View Full Code Here

        }
        return result;
    }


    public <KeyIn, ValueIn, ValueOut> Reducer<ValueIn, ValueOut> getReducerByKey(Object key) {
        Reducer reducer = reducers.get(key);
        if (reducer == null && configuration.getReducerFactory() != null) {
            reducer = configuration.getReducerFactory().newReducer(key);
            Reducer oldReducer = reducers.putIfAbsent(key, reducer);
            if (oldReducer != null) {
                reducer = oldReducer;
            } else {
                reducer.beginReduce();
            }

View Full Code Here

        }
    }


    private void reduceChunk(Map<Key, Chunk> chunk) {
        for (Map.Entry<Key, Chunk> entry : chunk.entrySet()) {
            Reducer reducer = supervisor.getReducerByKey(entry.getKey());
            if (reducer != null) {
                Chunk chunkValue = entry.getValue();
                if (chunkValue instanceof CombinerResultList) {
                    for (Object value : (List) chunkValue) {
                        reducer.reduce(value);
                    }
                } else {
                    reducer.reduce(chunkValue);
                }
            }
        }
    }

View Full Code Here

        }
        return result;
    }


    public <KeyIn, ValueIn, ValueOut> Reducer<ValueIn, ValueOut> getReducerByKey(Object key) {
        Reducer reducer = reducers.get(key);
        if (reducer == null && configuration.getReducerFactory() != null) {
            reducer = configuration.getReducerFactory().newReducer(key);
            Reducer oldReducer = reducers.putIfAbsent(key, reducer);
            if (oldReducer != null) {
                reducer = oldReducer;
            } else {
                reducer.beginReduce();
            }

View Full Code Here

TOP

Related Classes of com.hazelcast.mapreduce.Reducer

com.hazelcast.mapreduce.impl.task.JobSupervisor

com.hazelcast.mapreduce.impl.task.ReducerTask

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.