Examples of org.apache.flink.compiler.costs.DefaultCostEstimator

org.apache.flink.compiler.costs.DefaultCostEstimator
A default cost estimator that has access to basic size and cardinality estimates.
This estimator works with actual estimates (as far as they are available) and falls back to setting relative costs, if no estimates are available. That way, the estimator makes sure that plans with different strategies are costed differently, also in the absence of estimates. The different relative costs in the absence of estimates represent this estimator's heuristic guidance towards certain strategies.
For robustness reasons, we always assume that the whole data is shipped during a repartition step. We deviate from the typical estimate of (n - 1) / n (with n being the number of nodes), because for a parallelism of 1, that would yield a shipping of zero bytes. While this is usually correct, the runtime scheduling may still choose to move tasks to different nodes, so that we do not know that no data is shipped.

  // ------------------------------------------------------------------------  
  
  @Before
  public void setup() {
    this.dataStats = new DataStatistics();
    this.withStatsCompiler = new PactCompiler(this.dataStats, new DefaultCostEstimator());
    this.withStatsCompiler.setDefaultDegreeOfParallelism(DEFAULT_PARALLELISM);
    
    this.noStatsCompiler = new PactCompiler(null, new DefaultCostEstimator());
    this.noStatsCompiler.setDefaultDegreeOfParallelism(DEFAULT_PARALLELISM);
  }

View Full Code Here

    this.configuration = config;
    configuration.setString(ConfigConstants.JOB_MANAGER_IPC_ADDRESS_KEY, jobManagerAddress.getAddress().getHostAddress());
    configuration.setInteger(ConfigConstants.JOB_MANAGER_IPC_PORT_KEY, jobManagerAddress.getPort());
    
    this.userCodeClassLoader = userCodeClassLoader;
    this.compiler = new PactCompiler(new DataStatistics(), new DefaultCostEstimator());
  }

View Full Code Here

    if (port < 0) {
      throw new CompilerException("Cannot find port to job manager's RPC service in the global configuration.");
    }


    this.userCodeClassLoader = userCodeClassLoader;
    this.compiler = new PactCompiler(new DataStatistics(), new DefaultCostEstimator());
  }

View Full Code Here

   * it has no access to another cost estimator.
   * <p>
   * The address of the job manager (to obtain system characteristics) is determined via the global configuration.
   */
  public PactCompiler() {
    this(null, new DefaultCostEstimator());
  }

View Full Code Here

   * 
   * @param stats
   *        The statistics to be used to determine the input properties.
   */
  public PactCompiler(DataStatistics stats) {
    this(stats, new DefaultCostEstimator());
  }

View Full Code Here

  // ------------------------------------------------------------------------  
  
  @Before
  public void setup() {
    this.dataStats = new DataStatistics();
    this.withStatsCompiler = new PactCompiler(this.dataStats, new DefaultCostEstimator());
    this.withStatsCompiler.setDefaultDegreeOfParallelism(DEFAULT_PARALLELISM);
    
    this.noStatsCompiler = new PactCompiler(null, new DefaultCostEstimator());
    this.noStatsCompiler.setDefaultDegreeOfParallelism(DEFAULT_PARALLELISM);
  }

View Full Code Here

TOP

Related Classes of org.apache.flink.compiler.costs.DefaultCostEstimator

org.apache.flink.client.program.Client

org.apache.flink.compiler.CompilerTestBase

org.apache.flink.compiler.PactCompiler

org.apache.flink.test.compiler.util.CompilerTestBase

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.