Sequential Parameter Optimization Workshop


Table of Contents

  1. Sequential Parameter Optimization Workshop
    1. Table of Contents
    2. Overview
    3. Topics Covered
    4. Schedule
    5. Installation
      1. Q-1. How can R be installed on my local computer?
      2. Q-2. How can SPOT be installed on my local computer?
      3. Q-3. Where do I find the most recent SPOT package?
    6. Introductory Tasks and Hands-on Exercises
      1. T-1. Tuning Simulated Annealing
    7. Tasks for the Experts
      1. ET-1. Writing SPOT interfaces
      2. ET-2. Performing experiments in the SPOT environment
      3. ET-3. Performing SPOT meta runs
    8. Workshop Organizer
    9. References

Overview

  Experimental analysis of algorithms is becoming more and more important. Make sure that you or your students have the experience you need to be able to perform a proper analysis. You supply your laptop; we will supply the tools and techniques. Parts of this workshop use R, a freely available language and environment for statistical computing and graphics. R provides a wide variety of statistical and graphical techniques. The software can be downloaded from http://cran.r-project.org. 

The workshop will be presented at an introductory level and is meant for any EC researchers who want to compare their newly designed EC system with established EC systems to see if there is an improvement, or who want to determine which EC system performs better on their chosen problem; i.e., nearly everyone. It is vital, if our community is to be taken seriously, for us to continue to educate ourselves, and especially new Graduate students, on the statistical and experimental design techniques that are considered rigor for experiments performed in any field that is stochastic in nature, as is EC.

Topics Covered

Schedule

Installation

Q-1. How can R be installed on my local computer?

Precompiled binary distributions of the base R system and contributed packages are available from the following resources:
 The R FAQ provides further help.

Q-2. How can SPOT be installed on my local computer?

You need to have a running R system. Follow the instructions from Q-1 first.
> install.packages("SPOT")
> library(SPOT)
> demo(spotDemo07RandomForestSann,ask=F)

Note, the option "ask=F" is necessary to avoid user interaction each time the output window is redrawn.
> install.packages("Rcmdr")
 > library(Rcmdr)

Q-3. Where do I find the most recent SPOT package?

If you are an experienced R user, you can download the most recent SPOT version from our WWW page.
Note, this version has to be installed manually. The R FAQ provides further help. It is highly recommended that beginners proceed as decribed in Q-2.

Introductory Tasks and Hands-on Exercises

T-1. Tuning Simulated Annealing

How simulated annealing can be tuned is explained in this tutorial. Here, two algorithm parameters, namely TEMP and TMAX are tuned.
Click here to go to the on-line version of this tutorial. The pdf can be downloaded here.

Tasks for the Experts

ET-1. Writing SPOT interfaces

How your algorithm can be integrated into the SPOT framework is explained in this tutorial.
Click here to go to the on-line version of this tutorial. The pdf can be downloaded here.

ET-2. Performing experiments in the SPOT environment

This tutorial illustrates how experiments can be set up. It discusses problems related to experimental designs (factorial designs and space filling designs; regions of interest) and further settings that may influence experimental results.
Click here to go to the on-line version of this tutorial. The pdf can be downloaded here.

ET-3. Performing SPOT meta runs

Click here to go to the on-line version of this tutorial. The pdf can be downloaded here.

Workshop Organizer



 Thomas Bartz-Beielstein, Cologne University of Applied Science.
Dr. Thomas Bartz-Beielstein is a professor for Applied Mathematics at Cologne University of Applied Sciences (CUAS). He is head of the FIWA research team at CUAS, which develops tools from Genetic and Evolutionary Computation (GEC) for applications from finance and water industry.
Prof. Bartz-Beielstein has published more than several dozen research papers, presented tutorials and workshops about interactive and automatic tuning  of randomized and deterministic algorithms, and has edited several books in the field of GEC, e.g., "Experimental Research in EC" and "Experimental Methods for the Analysis of Optimization Algorithms". His research interests include optimization, simulation, and statistical analysis of complex real-world problems.
Prof. Bartz-Beielstein is the driving force in the development of the Sequential Parameter Optimization methodology and the related Toolbox (SPOT). SPOT was applied as a tuner for numerous optimization algorithms such as evolution strategies, differential evolution, or particle swarm optimization.




References