Stephan Weibelzahl

While empirical evaluations are a common research method in some areas of Artificial Intelligence (AI), others still neglect this approach. This article outlines both the opportunities and the limits of empirical evaluations for AI techniques exemplified by the evaluation of adaptive systems.

Using the so called layered evaluation approach, we demonstrate that empirical evaluations are able to identify errors in AI systems that would otherwise remain undiscovered. To encourage new evaluations we implemented an online database of studies that are concerned with empirical evaluations of adaptive systems (EASy-D).