Obj.5 – Relying on an iterative A/B testing approach, inform teachers through a reporting system on the decision-making process for improving the design of virtual labs.

Activities: Based on both shallow and deep analytics obtained from Obj.2 and Obj.3, pinpoint and report diverse data-driven metrics logged by the engagement of learners with the system, so that teachers (i.e., lab designers) that use the authoring environment (Obj.4) get informed about the strong or weak points of a lab as well as the differences between two versions of a lab and proceed to pro-active decisions so as to build new labs that fulfill users’ requirements or improve existing ones.

The statistical significance of an A/B test will be validated using hypothesis tests along with diverse measures of chance likelihood, e.g., t-test, X-squared test, etc. As being faithful merely to quantitative results may prove to be hazardous in the decision-making process, it is important to also keep an eye to qualitative aspects of an A/B test, like for instance the context of a trial, e.g. surrounding tasks of learners, in order to avoid blind interpretation of the underlying statistics. The decrease of the churn out rate as well as the increase of the engagement of learners with a virtual lab will be the tangible measure of the effectiveness of an iterative A/B test.