Examples of org.jamesii.core.experiments.BaseExperiment

org.jamesii.core.experiments.BaseExperiment
Base class for experiments. A computer based experiment with a model (aka simulation) is a set of computations done with a model. Thereby the initial parameters (for the model and the computation algorithms used) may vary from computation to computation. A base experiment contains the definition how to modify which parameters to be used for the computation / model. Basically the {@link #getExperimentVariables() experimentVariables} serves as a "for" loop.Thereby the items per level are progressed synchronous, and they constant while the sub levels are modified according to the rules defined in there.
For a simple experiment you only need to pass a fixed set of parameters which might even be empty, and an URI where the model can be found ( {@link #setModelLocation(URI)}). If you are interest in the trajectory of the simulation you have to add an instrumenter in addition. Everything else is a can, but not a must. Thus you can specify a computation task start time (this value will be used instead of 0 as initial value for the "simulation time") and the default factory to be used for determining the computation task end {@link #setComputationTaskStopPolicyFactory(ComputationTaskStopPolicyFactory)}, to use a distributed setup or a sequential on (default), the number of replications ( {@link #setRepeatRuns(Integer)}, {@link #setReplicationCriterionFactory(ParameterizedFactory)}), and you can specify the data sink to be used ( {@link #setDataStorageFactory(DataStorageFactory)}). May of these can be/or have to be parameterized, but this depends on the factory selected (e.g., you typically need to give username etc. for a database used as data sink).
Check out the {@link org.jamesii.core.experiments.variables.ExperimentVariables} class toread more about how to define which variables.
Using this class you can setup
- "normal" production runs, sometimes called parameter scans
- optimization experiments
- validation experiments
- benchmark experiments
A BaseExperiment does not create computation jobs directly. A base experiment only creates {@link TaskConfiguration} objects which are then transferred tothe {@link org.jamesii.core.experiments.taskrunner.ITaskRunner} to be used toexecute the tasks. This allows to use different execution modes with the same experiment definition (and thus to make the experiment definition scalable), and it eases the parallel creation of job configurations, which is supported as long as the "next" configurations do not depend on results of previous runs (as this might be the case for some optimization algorithms, however of some of those parallel variants exist as well). In addition even the software to be used for computation can be exchanged. By default always the {@link org.jamesii.core.experiments.tasks.setup.internalsimrun.SimulationRunFactory}is used but this can be changed by using the {@link #setSetupFactory(TaskSetupFactory)} method of instances of this class. @author Jan Himmelspach @author Roland Ewald


    // load symbolic model
    final IModel model = wizard.getValue(ModelLoader.MODEL);
    experiment = wizard.getValue(EXPERIMENT);
    if (experiment == null) {
      wizard.putValue(EXPERIMENT, experiment = new BaseExperiment());
    }


    ParameterBlock params =
        new ParameterBlock(model, ExperimentSetupEditorFactory.MODEL);

View Full Code Here


    // Execute all jobs
    for (ComparisonJob job : jobs) {


      // Configure base experiment
      BaseExperiment exp = new BaseExperiment();
      IProblemDefinition problem = job.getProblem();
      IProblemScheme bm = problem.getProblemScheme();
      exp.setModelLocation(bm.getUri());
      exp.setDefaultSimStopTime(SimulationProblemDefinition
          .getSimStopTime(problem));
      exp.setFixedModelParameters(new HashMap<String, Object>(problem
          .getSchemeParameters()));
      exp.setTaskRunnerFactory(new ParameterizedFactory<TaskRunnerFactory>(
          new AdaptiveTaskRunnerFactory(), (new ParameterBlock())
              .addSubBl(AdaptiveTaskRunnerFactory.PORTFOLIO,
                  new ArrayList<>(job.getConfigs().values()))
              .addSubBl(AdaptiveTaskRunnerFactory.POLICY,
                  policyFactory.getClass().getName())
              .addSubBl(ParallelComputationTaskRunnerFactory.NUM_CORES,
                  numOfThreads)));


      Map<Long, ParameterBlock> configs = job.getConfigs();
      exp.setRepeatRuns(repFactor * configs.size());


      // Set up performance recorder
      // TODO: Make recording optional
      PerfDBRecorder perfRec = new PerfDBRecorder();
      exp.getExecutionController().addExecutionListener(perfRec);
      perfRec.getNewVerRTConfigIDs().addAll(configsWithAlgo);


      // Execute experiment, record performance
      ComparisonJobResultListener cjrl =
          new ComparisonJobResultListener(job, perfType);
      perfRec.addListener(cjrl);
      perfRec.start();
      exp.execute();
      perfRec.stop();


      outputHandler.output(job.getProblem(), job);
    }

View Full Code Here

    observers.add(policyObserver);
    List<String> blackList = new ArrayList<>();
    blackList.add("SomeFactory");


    // Configure experiment with given model, policy, etc.
    BaseExperiment experiment = new BaseExperiment();
    experiment.setModelLocation(new URI(modelLocation));
    experiment.setBackupEnabled(false);
    experiment.setRepeatRuns(replications);
    experiment.setDefaultSimStopTime(stopTime);


    // Define experiment
    List<ExperimentVariable<?>> expModelVars =
        new ArrayList<>();
    expModelVars.add(new ExperimentVariable<>("someVariable",
        new IncrementModifierInteger(3, 1, 5)));
    expModelVars.add(new ExperimentVariable<>("anotherVariable",
        new IncrementModifierInteger(1, 1, 2)));
    experiment.setupVariables(expModelVars);


    // Get all possible options
    File optionFile = new File(OPTION_FILE);
    List<ParameterBlock> options = null;
    if (optionFile.exists()) {
      options = (List<ParameterBlock>) Files.load(OPTION_FILE);
    } else {
      options = createParameterBlocks(modelLocation, blackList);
      Files.save(options, OPTION_FILE);
    }


    // Set parameters of runner
    policyParams.setValue(factory.getClass().getName());
    ParameterBlock runnerParams = new ParameterBlock();
    runnerParams
        .addSubBl(AdaptiveTaskRunnerFactory.POLICY_OBSERVERS, observers)
        .addSubBl(AdaptiveTaskRunnerFactory.POLICY, policyParams)
        .addSubBl(AdaptiveTaskRunnerFactory.BLACK_LIST, blackList)
        .addSubBl(AdaptiveTaskRunnerFactory.PORTFOLIO, options)
        .addSubBl(ParallelComputationTaskRunnerFactory.NUM_CORES, 4);


    experiment
        .setTaskRunnerFactory(new ParameterizedFactory<TaskRunnerFactory>(
            new AdaptiveTaskRunnerFactory(), runnerParams));


    // Execute experiment
    experiment.execute();


    // Analyse and reset observer
    policyObserver.reset();
  }

View Full Code Here


  /**
   * Test simple experiment execution.
   */
  public void testSimpleExperimentExecution() throws Exception {
    BaseExperiment exp = new BaseExperiment();
    exp.setModelLocation(new URI("java://"
        + BogusModel.class.getCanonicalName()));
    exp.setDefaultSimStopTime(100);
    exp.execute();


    for (int load : new int[] { 1, 2, 4, 8, 16, 32, 64 }) {
      executeExperimentWithLoad(load);
    }
  }

View Full Code Here

   * 
   * @throws Exception
   *           the exception
   */
  private void executeExperimentWithLoad(final int load) throws Exception {
    BaseExperiment exp = new BaseExperiment();
    exp.setModelLocation(new URI("java://"
        + BogusModel.class.getCanonicalName()));
    exp.setDefaultSimStopTime(100);
    exp.setProcessorFactoryParameters(FlexibleBogusSimulatorFactory.class
        .getName());
    exp.getParameters()
        .getParameterBlock()
        .addSubBlock(FlexibleBogusSimulatorFactory.SIM_PROPERTIES,
            new IBogusSimulatorProperties() {
              private static final long serialVersionUID =
                  -8801113732776797227L;


              @Override
              public int getLoadPerSteps(Map<String, Serializable> modelContent) {
                return load;
              }
            });
    exp.execute();
  }

View Full Code Here

   * @throws Exception
   */
  public Double executePerformanceExploration(IPerformanceDatabase perfDB,
      Long rngSeed, boolean switchOffExploration) throws Exception {


    experiment = new BaseExperiment();
    experiment.setModelLocation(new URI(model));


    PerfDBRecorder perfRecorder = new PerfDBRecorder();
    perfRecorder.getInstanceGenerator().setCurrentMode(
        ProblemInstanceSelectionMode.CONSTANT);

View Full Code Here

    List<Double> performances = new ArrayList<>();
    for (Map<String, Object> parameterSetup : setup.getSecondValue()) {
      System.err.println("Executing: " + Strings.dispMap(parameterSetup));
      StopWatch sw = new StopWatch();
      sw.start();
      BaseExperiment exp = new BaseExperiment();
      exp.getFixedModelParameters().putAll(parameterSetup);
      exp.setModelLocation(exploration.getExperiment().getModelLocation());
      exp.setDefaultSimStopTime(defaultSimStopTime);
      exp.execute();
      sw.stop();
      performances.add(sw.elapsedSeconds());
    }


    return new RealWorldSelectorPerformanceEntry(setup.getFirstValue(),

View Full Code Here

        for (double[] currentPortfolio : new double[][] { gaPortfolio,
        /* completePortfolio, */stochasticPortfolio }) {
          for (int policyRep = 0; policyRep < POLICY_REPS; policyRep++) {
            System.err.println("Portfolio:"
                + Strings.dispArray(currentPortfolio));
            BaseExperiment be = new BaseExperiment();
            be.setDefaultSimStopTime(STOP_TIME);
            be.setModelLocation(problem.getProblemScheme().getUri());


            // @see CyclicChainSystem.REACT_PER_SPECIES
            be.getFixedModelParameters().put("ReactPerSpecies", 1);
            be.setRepeatRuns(ACTUAL_REPLICATIONS);


            List<ParameterBlock> portfolioBlocks =
                getParamBlocksForPortfolio(currentPortfolio, portPerfData);
            System.err.println("Size:" + portfolioBlocks.size());


            List<IObserver<? extends IMinBanditPolicy>> observers =
                new ArrayList<>();
            observers.add(new SimplePolicyObserver());


            ParameterBlock pb = new ParameterBlock();
            pb.addSubBl(AdaptiveTaskRunnerFactory.PORTFOLIO, portfolioBlocks)
                .addSubBl(AdaptiveTaskRunnerFactory.POLICY_OBSERVERS, observers)
                .addSubBl(AdaptiveTaskRunnerFactory.POLICY,
                    new ParameterBlock(UCB2Factory.class.getName()));
            pb.addSubBl(ParallelComputationTaskRunnerFactory.NUM_CORES, 1);
            be.setTaskRunnerFactory(new ParameterizedFactory<TaskRunnerFactory>(
                new AdaptiveTaskRunnerFactory(), pb));


            StopWatch sw = new StopWatch();
            sw.start();
            be.execute();
            sw.stop();
            System.err.println("Execution Time:" + sw.elapsedMilliseconds());
            execTimes.add(sw.elapsedMilliseconds() / 1000.0);
            System.err.println("Execution Times:"
                + Strings.dispIterable(execTimes));

View Full Code Here

   *          the arguments
   */
  public static void main(String[] args) {
    try {
      ApplicationLogger.setLogLevel(Level.ALL);
      BaseExperiment exp = new BaseExperiment();
      exp.setModelLocation(new URI(
          "java://org.jamesii.examples.ca.tutorial.BogusModel"));
      ParameterBlock pb = new ParameterBlock();
      pb.addSubBlock(SimTimeStopFactory.SIMEND, 10.0);


      exp.setComputationTaskStopPolicyFactory(new ParameterizedFactory<ComputationTaskStopPolicyFactory<?>>(
          new SimTimeStopFactory(), pb));
      exp.setRepeatRuns(1);
      exp.execute();
    } catch (Exception ex) {
      SimSystem.report(ex);
    }
  }

View Full Code Here

        .compareTo(deserialisedVersion.getParameters().getParameterBlock()) == 0);
  }


  @Override
  public BaseExperiment getTestObject() throws Exception {
    BaseExperiment exp = new BaseExperiment();
    exp.setProcessorFactoryParameters( "test.ProcessorFactory");
    return exp;
  }

View Full Code Here

0 1

TOP

Related Classes of org.jamesii.core.experiments.BaseExperiment

org.jamesii.asf.integration.PerformanceExplorationTask

org.jamesii.asf.integration.SelectorDeploymentTask

org.jamesii.asf.integration.SimpleTestExperiments

org.jamesii.core.cmdparameters.Parameters

org.jamesii.core.experiments.BaseExperimentSerializationTest

org.jamesii.core.experiments.replication.MaximizingReplicationCriterionFactory

org.jamesii.core.experiments.replication.RepNumberCriterionFactory

org.jamesii.core.experiments.taskrunner.TestEmptyExperimentErrorHandling

org.jamesii.core.experiments.util.ExperimentBackup

org.jamesii.core.experiments.variables.DynamicExperimentConfigurator

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.