Stephan Weibelzahl

Software and interactive systems that adapt their behavior to the user are often referred to as Adaptive Systems. These systems infer the user's goals, knowledge or preferences by observing the user's actions. A synposis of 43 published studies demonstrated that only few of the existing systems are evaluated empirically. Most studies failed to show an advantage of the user model. A new framework is proposed that categorizes existing studies and defines an evaluation procedure which is able to uncover failures and maladaptations in the user model. It consists of four layers: evaluation of input data, evaluation of inference, evaluation of adaptation decision and evaluation of total interaction. Exemplary, the framework has been applied to the HTML-Tutor, an onlinecourse that adapts to the learners' knowledge. Several empirical studies are described that test the accuracy of the user models, and explore the effects of adaptation to knowledge respectively prior knowledge. Generalization issues of the approach are discussed.