Evaluation

This work package will evaluate the creativity produced by the prototypes derived in ConCreTe using human raters. Rating of creativity will continue throughout the project. Evaluating creative artifacts and systems is notoriously difficult, due to the subjectivity of definition of the terms involved. Therefore, we will investigate the notion of creativity by implementing a self-organizing metric of terms related to creativity, derived from text corpora (e.g., WordNet). Our emergent semantic map will be used for human ratings of creativity of the ConCreTe prototypes and applications. We will also use human ratings to compare the use of ratings of creativity based on our emergent semantic map to ratings of creativity based on criteria reported in the literature. We will further compare our emergent semantic map with the ConCreTe conceptual space, to relate our creativity metric with the conceptual development in that space.

Objectives

1. Throughout the project we assess the results produced by the prototypes of creativity derived in ConCreTe as useful, original and surprising using human raters.
2. We develop an emergent metric (semantic map) of terms related to creativity, based on a self-organizing procedure, for the evaluation of creative systems.
3. We apply our emergent metric to evaluate the creativity of the propotypes derived in ConCreTe, using human raters.
4. We compare our emergent metric for rating of creativity with other rating procedures in the literature, using human raters and the prototypes of creativity derived in ConCreTe.
5. We compare our emergent semantic map with the ConCreTe conceptual space.
6. We will design new methods for indirect evaluation of our cognitive architecture as a model of human creativity.