Evaluating Plan-Property Dependencies: A Web-Based Platform and User Study
Rebecca Eifler, Martim Brandao, Amanda Coles, Jeremy Frank and Joerg Hoffmann
Abstract: The trade-offs between different desirable plan properties -- e.g.
PDDL temporal plan preferences -- are often difficult to
understand. Recent work addresses this by iterative planning with
explanations elucidating the dependencies between such plan
properties. Users can ask questions of the form ``Why does the plan
not satisfy property p?'', which are answered by ``Because then we
would have to forego q''. It has been shown that such dependencies
can be computed reasonably efficiently. But is this form of
explanation actually useful for users? We run a large crowd-worker
user study (N=100 in each of 3 domains) evaluating that question. To
enable such a study in the first place, we contribute a Web-based
platform for iterative planning with explanations, running in standard
browsers. Comparing users with vs. without access to the explanations,
we find that the explanations enable users to identify better
trade-offs between the plan properties, indicating an improved
understanding of the planning task.
*This password protected talk video will only be available after it was presented at the conference.
