Adapt/Minerva project
Twente part

Evaluation study of building adaptive products

A project funded under the Socrates-Minerva Programme of the Directorate for Education and Culture, European Commission

Final report
(online version)
Information about the Twente experiments

Rik Min, Italo De Diana & Niels Primus

March, 29, 2005
(online version)

Center for Telematics and Information Technology
CTIT
University of Twente
2002 - 2005
Code: EU/E Opener 4006.1301

Abstract

The University of Twente participates in the Adapt/Minerva project of the EU, among others in cooperation with the University of Nottingham and the Technical University Eindhoven. These three actors are, among other tasks, involved in the evaluation of adaptive multimedia products generated by systems, tools and languages. The group of the University of Twente has carried out an experiment to find out what the characteristics of the used tools and generated products are.

Three aspects of evaluation are involved: system evaluation, in respect to the used authoring tools, product evaluation in respect to the resulting courseware products, and process evaluation in respect to the application of tools in order to develop products. The multimedial courseware products involved are based upon an interactive dynamical website for the simulation of a transistor circuit.

In the experiment 6 kind of content elements were used: text, visuals, sounds, linear animation, video and an 'intelligent' part: the six dimensions in our test product. That means: the test products contains adaptive text and other kinds of content as an adaptive gateway to an interactive task-oriented micro-world. These content elements were applied in a process of translating the existing courseware product into new versions developed by applying authoring tools.

Introduction

The University of Twente participates in the EU project, Adapt/Minerva, together with the University of Nottingham, Southhampton, the Technical University Eindhoven, Trento, and the Politecnico di Milano.

The University of Twente is involved in evaluation research. In this research a special courseware product is made adaptive in three different ways by applying three different authoring tools. We are discussing the transformation and evaluation of the courseware product as caused by the application of the authoring tools. The original courseware products have educational simulation as the product core. Furthermore, the products offer text, graphics, animations, and audio and video. The product is Web-based. In the Twente part of the Minerva project 'Adapt' we study how this type of product can be transformed into adaptive educational simulation products by using one or more authoring tools for building adaptive materials. Moreover, we study the final products within their respective delivery systems.

ITS systems

ITS systems have been researched and developed for many years already. Also our department in Twente (Dijkstra, Krammer, Merrienboer and Tekinerdogan) was involved heavily in research on ITS. Unfortunately, until the present, ITS has proven too complex for creating useful and applicable courseware. The creation of good domain models and the online creation and application of student models is a very complex process. (Winkels, 1992). The trend the last years in fact is the reverse: open courseware, courseware that is not adaptive but that offers many attractive (intelligent) parts: two-way e-learning, learning materials with intelligent components such as simulations or micro-worlds, also called ‘higher order courseware’. Such courseware that can be applied in more natural modes of adaptation, without the need to create complicated student models. In a way, simulations are already adaptive products. The teacher can decide on what level and with what assignment a student can work. If a student cannot make an assignment, he is not ready for it, see the publications of Veermans, Winkels, De Vries and Dijkstra, University of Twente (Veermans, 2003; Winkels, 1992; De Vries, 1997; Stoop, 2001; Futo & Gergely, 1992 and Min, 1996 – 2004) An overview of ITS research in Twente in the departments of Toegepaste Onderwijskunde (TO) and Computer Science (INF), can be found in Tekinerdogan (Tekinerdogan et al, 1995).

Currently we sense again an upsurge of ITS research. Maybe due to the fact that technologies and techniques become available such as web-technology and XML. The 5 layer-authoring model of Cristea & Calvi (2003) corresponds with the basic ideas of ITS systems. This layer-model resembles, as a description of the authoring process, the 6 step-model of courseware development, as described by De Diana (De Diana, 1999).

In the Minerva project we want to connect authoring tools with delivery tools. In some tools these two functions are combined in one system and in other tools they are not. Our partners are working on transformation tools. Because in the project we move away from specialized tools of authoring for only specific delivery systems and try to understand what is in common to all adaptive hypermedia systems. (See our interim report 2004)

Evaluation

In this report we make a distinction between system evaluation, product evaluation and process evaluation. With system evaluation we mean a functional evaluation of the authoring tools involved. With product evaluation we mean an evaluation of the products that result from applying the authoring tools. With process evaluation we mean a discription of the process from which products result by applying authoring tools.

System evaluation: authoring tools

Authoring tools are software tools that support the development of courseware by offering dedicated functions and methods. Sometimes authoring tools generate language code such as XML or HTML or even Java. Such code needs specific delivery systems. So the two functionalities, editing and presenting the content are split up. That is a good division of labor between the two main parts of the authoring process.

In the Minerva project several authoring tools are involved and or under construction. We evaluate by applying three authoring systems, WHURLE, MOT and AHA! and 2 conversion systems.

The goal of these type of tools and/or the type of products, is personification of learning and teaching.

Table 1. The six methods; more characteristics
method WHURLE (system)	method MOT (system)	method AHA! (system)	method MOT - WHURLE (conversion)	method MOT - AHA! (conversion)	method JavaScript (stand alone)
A-system	A-system	A-system	conversion system	conversion system	programming language
has a delivery system	has no delivery system	has a delivery system	not applicable	not applicable	a browser
3 levels (beginners, intermediates, experts)	more than 3 levels (infinite)	more than 3 levels (infinite)	3 levels (beginners, intermediates, experts)	more than 3 levels (infinite)
DM, GM, UM, AM & PM	DM, GM, UM, AM & PM	DM, GM, UM, AM & PM	not applicable	not applicable	no, only a UM and DM

In our project part we apply authoring tools to transform courseware products into adaptive products. Here we discuss the systems that are being developed in Eindhoven and Nottingham and that we have used.

WHURLE

The authoring system WHURLE is developed at the University of Nottingham. The name is an acronym for Web-based Hierarchical Universal Reactive Learning Environment. It is classified as an system for adaptive hypermedia products.

The adaptive products don’t work with adaptive nodes but with information that is being selected on the basis of the level of knowledge of beginners, more advanced or experts. The chunks present their content in relation to the kind of users (the user model). The user model is built in a pre-test session. Traversal of the learning material, the courseware, depends on the student and the learning style that is measured in advance.

WHURLE interpretes Xml-files and generates Dhtml-files. One of the disadvantages of the system is that it needs a dedicated server and a database system.

AHA!

The AHA!-system is a tool and developed at the Eindhoven University of Technology. The AHA!-system is classified as both an authoring and a delivery system for adaptive hypermedia products.

AHA! has a well developed set of graphical editors: a concept-editor and a form-editor. These editors can handle objects. The logic is based on ‘event-condition-action-rules’ (with prerequisites). Each object is connected with each attribute by rules. The AHA!-system generates XML-files. See table 1 and 2. One of the disadvantages of the system is that it needs a dedicated server and a database system.

MOT:

This system is developed by Cristea at the Eindhoven University of Technology. MOT means 'My Online Teacher' and is classified as an adaptive hypermedia system. MOT is a authoring tool and not a delivery tool. MOT supports a five layer model for authoring adaptivity: a domain model, a goal and constraints model, a lesson model, an adaptation model and a presentation model. MOT has a nice parallel interface to edit text, but doesn't have graphical oriented WYSIWYG tools at this moment. The MOT system is based on LAOS and LAG, and creates MySQL-files. See table 1 and 2. One of the disadvantages of the system is that it needs a dedicated server and a database system.

Table 2. The six methods; more characteristics
method WHURLE (system)	method MOT (system)	method AHA! (system)	method MOT - WHURLE (conversion system)	method MOT - AHA! (conversion system)	method JavaScript (stand alone)
TomCat 3.3 Apache Cocoon (Xml-kit) MySQL	MySQL Xml	TomCat Apache MySQL Xml	not applicable	not applicable	client actions; no server actions
Dhtml	output in Html	Xml	not applicable	output in Xml	not applicable
no system editors	one editor	4 editors: Form Editor Concept Editor Graph Authoring Tool Test Editor	depends on MOT	depends on MOT	no system editors
no manual	no manual	online manual available	no manual	no manual	no manual
3 dimensional lesson 'Transistor', version N. Primus.	3 dimensional lesson 'Transistor', version N. Primus.	3 dimensional lesson 'Transistor', version N. Primus.	3 dimensional lesson 'Transistor', version N. Primus.	lessons: not applicable	6 dimensional lesson 'Transistor plus', version R. Min.

MOT to WHURLE (MOT2WHURLE converter)

This system for the conversion (translation) from a MOT product ('half products') to WHURLE products is developed by Brailsford, Moore, Cristea and Stewart, at the University of Nottingham in the Nottingham/Eindhoven part of this Adapt project. WHURLE uses the Xml-files from MOT and presents the content in Dhtml. See table 1 and 2.

MOT to AHA! (MOT2AHA! converter)

This system for MOT to AHA! conversion is developed in Eindhoven by Cristea and De Bra. AHA! uses the files from MOT. See table 1 and 2.

JavaScript system

Min has developed one adaptive test lesson, built with JavaScript to demonstrate the different paths in the courseware (for beginners, advanced users and experts) and to use this as an working example for the members in the other project groups (Eindhoven en Nottingham). The demonstration application has three parts and starts with a mc-questionnaire, for classifying a 'user model' and continues to the 'best' part of the courseware. After the lesson, and the 'test' in using and working with the 'transistor simulation', the user is lead to the mc-questionnaire again for his 'grade'. See table 1 and 2; and figure 1 and 5.

Results of the system evaluation

Niels Primus has written a MSc-thesis on system evaluation. The results of this thesis we present here. Tables in his report shows only MOT offers adaptive multimedia presentation and adaptation of modality. In a broader sense, all systems should perhaps reconsider the set of techniques they currently adopt. There certainly is a gain in using more different methods and techniques, as this allows teachers to create more diverse lessons.

MOT scores high on both technological and educational criteria and a bit less on end-user criteria. This shows MOT is a technology driven system, that has to be translated into a more user-friendly application. The idea of the developers of MOT was, in fact, just this, as they did not add a presentation module for teaching purposes (an interface for students) on purpose. The presentation for teachers, however, should be made clearer, as teachers and course developers are, generally spoken, not technological experts. A novice user can not, for instance, be expected to understand the difference between concepts and lessons.

At first glance, AHA! seems the more all round system. It receives high values in all three categories, but, of course, still contains some flaws. This system is also a technology driven system, though better evolved in the direction of the end-user. In fact, AHA! is the only system, of these three, to receive a positive total assessment in the category of end-user criteria.

WHURLE more or less comes out of the test as the inferior system of the three. The reason for this could partly lie in the fact that the system was not tested as MOT and AHA! were, for creating lessons. The WHURLE test lesson was created in MOT and then transferred to the WHURLE system. The reason for this is the complicated way of creating lessons in WHURLE in combination with the fact that the system is not (yet) designed to create lessons. In fact, WHURLE does not provide an editor or interface for creating course materials and adaptive structures. WHURLE unarguably contains certain flaws, such as the unreliability of the system in general and the improper features of certain presentation aspects.

All three systems score well on both educational and technological criteria, this is explained by the fact that they were all constructed from some great ideas of concepts and adaptive linking mechanisms. These aspects are translated into a solid technical foundation, as can be derived from the relative high values in the category of technological criteria.

In order for AEH authoring applications (systems) to be widely accepted by the educational world, usability is an issue of major importance. MOT, AHA! and WHURLE all are technology driven systems, developed by IT researchers and, as such, are not the most user-friendly from the perspective of teachers or courseware developers.

All in all, considering the fact that the three systems are still under development; MOT, AHA! and WHURLE will be 'good' AEH authoring systems eventually. The basic conceptual structures are solid and thought-out. However, the developers have to reconsider and perhaps extent the adaptive features, improve the general possibilities, such as the editors, and work on the user-friendliness of the systems [Niels Primus, 2005].

Product evaluation: the test product

In our evaluation procedure we have the following products or partial products:

the original courseware products;
the intended transformed products;
the transformed products.

Figure 1. The original 3 dimensional test lesson TransistorARNS (master students version A. Rhema & N. Smeda, 2003).

We can describe each of these products by a set of characteristics, such as:

short overall description;
product goals;
technical qualities such as product organization and instructional approach;
content to be studied;
media utilization;
performance evaluation.

It should be noted that not all of these descriptors can be applied to the product-as-intended.

Due to the application of the authoring system, the original product is transformed into the final product. The transformation is guided by the intentions for the final product. In our evaluation we can study:

the characteristics of the final product in respect to those of the original product;
the characteristics of the final product in respect to the intended characteristics;
the performance of the final product.

We have chosen to take the content from an original piece of courseware about transistors and RC circuits. See figure 1 and 2.

Figure xxx. The flow chart of the demonstration lesson. You see the path for beginners, the path for intermediates and the path for experts.

The multimedial product and the content we have chosen for the experiments is a task-oriented interactive dynamical website for the simulation of a transistor circuit and the connected instruction. This instruction has to be adaptive. Beginners follow another trajectory than more advanced learners. Experts can go immediately to the simulation part and experiment with the simulation.

The group of the University of Twente has initially developed a special pre-prototype in JavaScript to make an inventory of problems involved in the transformation process. The derived information is used for experiments performed by the other partners: one in Nottingham; two in Eindhoven.

Each experiment uses the same 6 dimensional multi medial raw data content: text, visuals, sounds, linear animation, video and an intelligent part. That means: the product contains adaptive text and other dimensional content as an adaptive gateway to an interactive task-oriented micro-world. The ‘6st’ dimension is the dimension about the intelligent parts of the courseware (here the class-file with the simulation applet). The simulation, the micro-world and the tasks or exercises themselves are also adaptive and even the feedback of the simulation - we hope - will be made adaptive.

The courseware contains the following parts:

The content of the original courseware product is an instruction divided in three chapters about the history of transistor for beginners, the physics for more advanced learners, the principles of AC/DC current and voltages in relation to resistors and capacitors for electro-technical students. The electronical circuit with a common emitter is explained for beginners.

Figure 3. The 4e dimension: animation. You see the animation for beginners.

Figure 4. The 3e and the 5e dimension: audio and video. You see the coach for all the users of the last part of the lesson, the simulation.

After the instruction part the simulation part is open for learning the influences of the resistors and capacitors in the circuit to the amplifier capacity of the whole transistor circuit. The influences of the circuit determinants to the amplifier capacity is so complex that only text, books and even lectures are not sufficient to understand these theoretical principles.

Figure 5. The flow chart of the 6 dimensional demonstration lesson. You see the animation for beginners, the speaking video coach for all the users, parallel to the problem solving part with the 'open' simulation environment.

Results of the product evaluation

AHA! and MOT are adaptive hypermedia systems in the sense that they can generate adaptive courseware (lessons). Translator systems have been developed in order to translate files from MOT, AHA! and WHURLE. MOT files can be translated to AHA! and to WHURLE. Adaptive courseware also can built by scripting languages; directly.

Figure 6. The test lesson TransistorARNS in MOT.

Niels Primus has developed 4 protypes: one in WHURLE, one in MOT, one in AHA! and from MOT to WHURLE. See figure 6, 7 and 8.

Figure 7. The test lesson TransistorARNS in AHA!.

Figure 8. The test lesson TransistorARNS in WHURLE (translated from MOT).

Rik Min has developed courseware, writen in JavaScript, containing three lesson parts. See figure 9 and 10.

Figure 9. Screendump of the first part of the adaptive demonstration lesson in JavaScript: the common part for beginners as well for intermediates and experts.

The lesson, writen in JavaScript, as shown in figure 9 and 10 is an minimum test with 'content' in 6 dimensions; not only in text and pictures, but also in animation, in audio and in video. The lesson contains about 23 chuncks: 10 chunks with text (exclusive the mc-questionnaire); 10 chunks with a picture; one chunk with an animation; one chunk with a movie (with audio), and one chunk with a micro world (a simulation).

The JavaScript version of Transistor is used an example (reference) for the four other products. This example is so clear - for us - that all we want to have in this project - is incorporated. It contains a good working mc-questionaire and a good working differentation system. The other prototypes - for testing the systems - do'nt have these separation capabilities.

Figure 10. Screendump of the last part of the adaptive demonstration lesson in JavaScript: the simulation part with the audiovisual coach; six dimensional.

This 6 dimensional product of Rik Min was a target and a reference for Niels Primus experiemnts with the 4 systems. Niels Primus only tested 3 of the 6 dimensions with the systems as described above.

The JavaScript product of Min is the only product that has and is tested with a credit system. The multple choice questionaire in the heading of the 6 dimensional courseware gives a good differentiation between 'beginners', 'mediates' and 'experts'.

Process evaluation: estimates of development time

In the process evaluation we study how the transformation process has been organized (intentions) and how the process was actually carried out. We consider that the process can be split up in transformation steps and that for each step it can be studied:

the purpose of the transformation step;
how the authoring tool was involved, that is what functions were used;
what authoring functions were lacking or incomplete in respect to our purpose;
how the usability was of the authoring functions applied;
what product parts were lacking, had to be changed or to be deleted;
what the results were of the step;
What the time (duration) involved in the transformation step.

Our group has planned an evaluation of three - in principle similar - products: one build by an expert in WHURLE in the UK; one build by an expert in MOT in Eindhoven and one build by an expert in AHA!, also in Eindhoven. We want to have data with the transformation steps (from raw text information to the real adaptive multimedial product), the time the designer spends to the steps (duration times), and the total time of the design and developing process in relation to the raw text data. The data we want to collect, are the steps the designer have to taken (the quality aspect) and the time this step have cost (the quantity aspect), as shown in the list.

Table 3. Time estimates
method WHURLE (system)	method MOT (system)	method AHA! (system)	method MOT - WHURLE (conversion)	method MOT - AHA! (conversion)	method JavaScript (stand alone)
text, pictures and applet	text, pictures and applet	text, pictures and applet	text, pictures and applet	not tested	text, pictures, sound, animation, video and applet
not applicable	1 hours (preparation)	1 hours (preparation)	irrelevant	not applicable	not applicable
not applicable	7 hours (authoring)	11 hours (authoring)	30 minutes (authoring)	not applicable	not applicable
not applicable	8 hours (total)	12 hours (total)	30 minutes (total)	not applicable	not applicable

Specialists in the authoring systems are the best courseware builders. Non-specialists need much more time to develop a lesson and if one non-specialist is working with 3 different tools, the chance that one of the tools is being neglected and discriminated in the evaluation is very high. Thas is the reason that in this evaluation plan specialists of Eindhoven (MOT and AHA!) and Nottingham (WHURLE) make the chosen piece of courseware with their own tools. Our hypothesis is that these 3 teams do this chosen engineerings job by the Twente team, in 4 to 8 hours (see table 3).

Results of the process evaluation

During the product evaluation Niels Primus has registered duration times of several authoring steps as involved in the questionnaire our group has produced in 2004. These time estimates show the results of working with the various authoring tools involved.

Conclusions

From the systems-, products- and process-evaluation the following information can be gained.

In order for AEH authoring applications to be widely accepted by the educational world, usability is an issue of major importance. MOT, AHA! and WHURLE all are technology driven systems, developed by IT researchers and, as such, are not the most user-friendly from the perspective of teachers or courseware developers.

Niels Primus has developed 4 protypes: one in WHURLE, one in MOT, one in AHA! and from MOT to WHURLE. See figure 6, 7 and 8.

The 6 dimensional product of Rik Min was a target and a reference for Niels Primus experiments with the 4 systems. Niels Primus only tested 3 of the 6 dimensions with the systems as described above. The JavaScript product of Min is the only product that has and is tested with a credit system. The multple choice questionaire in the heading of the 6 dimensional courseware gives a good differentiation between 'beginners', 'mediates' and 'experts'.

We have studied the performance of the adapted courseware products, based on structured walkthroughs. AHA! has proven to be the most complete and most user friendly system. WHURLE is more a system for classical courseware and not perfect for developing adapitive products. WHURLE works well with MOT and is well suited for presenting lessons created in MOT. MOT is theoretically well designed and in combination with AHA! and WHURLE offers high potentional.

References & literature

ADAPT project
Online available on: http://wwwis.win.tue.nl/~acristea/HTML/Minerva/index.html

AHA!
Information can be find on: http://aha.win.tue.nl (P. de Bra, Eindhoven University of Technology)

Brusilovsky P. (2002)
Adaptive hypermedia, User Modeling and User Adapted Interaction, Ten Year Anniversary Issue (Alfred Kobsa, ed.) 11 (1/2), pp. 87-110.

Cristea, A.I., Calvi, L. (2003)
The three Layers of Adaptation Granularity, In Proceedings of UM’03, Pittsburgh, US, Springer.

Cristea, A.I., Floes, D., Stash, N., de Bra, P. (2003)
MOT meets AHA!, In Proceedings of PEG 2003, St. Petersburg, Russia.

Cristea, A.I. & Aroyo, L. (2002)
Adaptive Authoring of Adaptive Educational Hypermedia, In Proceedings of AH 2002. Adaptive Hypermedia and Adaptive Web-Based Systems, LNCS 2347, Springer, pp. 122-132.

Cristea, A.I. & De Mooij, A. (2003)
LAOS: Layered WWW AHS Authoring Model and their corresponding Algebraic Operators, In Proceedings of WWW’03, Budapest, Hungary, ACM, IW3C2.

De Bra, P. (2002)
Adaptive Educational Hypermedia on the Web. Communications of the ACM, 45(5), pp. 60-61.

Diana, I. De, (1999)
The 6-steps model for courseware authoring. In: Courseware Engineering Architecture (CEA), collegediktaat, vakcode 193530. University of Twente.

Futo, I. & T. Gergely (1990)
Artificial Intelligence in Simulation, Ellis Horwood Limited. ISBN 0-13-048257-9.

Min, F.B.M., (2004) Demonstration lesson of adaptive courseware: 'Transistor', built with JavaScript. CTIT. University of Twente. Online available on: http://projects.edte.utwente.nl/pi/Examples/TransistorARNS/mc.html

Min, F.B.M., and I. De Diana (2004) Information about the Twente courseware. CTIT. University of Twente. Twente part of the Adapt/Minerva project. Interim report. Also online available on: http://projects.edte.utwente.nl/pi/Projecten/intrimReport.htm

Min, F.B.M., and I. De Diana (2003)
Adaptive products, What? Which? How? A contrudution to the Minerva/Adapt project. Workshop Politecnico di Milano, Milan.

Min, F.B.M., (1996 – 2004)
An serie of 6 e-books about multimedia, courseware and simulation components for higher order e-learning environments. University of Twente. Online available on: http://projects.edte.utwente.nl/pi/Book/Home.html

MOT
Information can be find on: http://e-learning.dsp.pub.ro/mot/ and http://e-learning.dsp.pub.ro/motadapt/ (A. Christea, Eindhoven University of Technology)

Primus, N.
A generic framework for evaluating Adaptive Educational Hypermedia authoring systems; Evaluating MOT, AHA! and WHURLE to recommend on the development of AEH authoring systems Also online available on: http://projects.edte.utwente.nl/pi/Papers/primus.html

Stoop, E.M.L.A. (2001)
Detection of Misfitting Item-Score Patterns in Computerized Adaptive Testing. PhD thesis. University of Twente.

Tekinerdogan, B., H. Krammer, J. Merrienboer & M. Schonewille (1995)
A Generic Domain Model for an Imperative Programming Languages Tutor Shell. In Proceedings of the European Conference on Computer Assisted Language Learning, Valencia, Spain.

Veermans, K. (2003)
Intelligent support for discovery learning. PhD-thesis. Twente University Press. University of Twente, Enschede.

Vries, S.A. de (1997)
Aanpasbare Courseware: ontwerp en gebruik binnen het voortgezette onderwijs (about adaptable courseware; design and use). PhD thesis. University of Twente.

WHURLE
Information can be find on: http://sourceforge.net/projects/whurle (C. Stewart, A. Moore and T.J. Brailsford, Universiteit Nottingham)

Winkels, R. (1992)
Explorations in intelligent tutoring and help. IOS Press, Amsterdam, Wageningen, Tokyo.

Enschede, March, 29, 2005

Appendix

Questionnaire

A. Product characteristic:

.... Old situation: .... New situation: ....

Modality:

Does your product have texts? .... yes .... ………

Does your product have images? .... yes .... ………

pictures .... yes .... ………
photographs .... yes .... ………
schemes .... yes .... ………

Does your product have linear animations? .... No, but ready for use (in Dutch) .... ………

dynamical gif-files .... yes .... ………
Flash animations .... yes .... ………

Does your product have sounds? .... No, but ready for use (in Dutch) .... ………………

sound-files .... yes .... ………………

Does your product have videos? .... No, but ready for use (in Dutch) .... ………………

QuickTime-movies .... yes ....………………
others .... no .... ………………

Does your product have intelligent components? .... yes .... ………………

rule base .... no, but ready for use .... ………………
expert system .... no .... ………………
(model-driven) simulation .... yes .... ………………
scripts .... no .... ………………

Does your product have other elements/components? .... no .... ………………………………

Models:

Does your product have a domain model? .... no .... ………………

Does your product have a user model? .... no .... ………

Does your product have an adaptation model? .... no .... ………

Does your product have a presentation model? .... no .... ………

Does your product have a lesson model? .... no .... ………

Does your product have other models? .... no .... ………

Instruction events:

Is your product better now on Gaining attention (event of instruction 1)? ………………………………………