Introduction

One of the more popular standards that has emerged in the field of e-assessment is that of Question and Test Interoperability (QTI), developed by the IMS Consortium. The QTI specification describes a data model for representing questions and tests and the reporting of results, thereby allowing the exchange of data (item, test, and results) between tools (such as authoring tools, item banks, test constructional tools, learning environments, and assessment delivery systems). Wide take-up of QTI would facilitate not only the sharing of questions and tests across institutions, but would also enable investment in the development of common tools. QTI is now in its second version (QTIv2), designed for compatibility with other IMS specifications, but despite community enthusiasm there have been only a few real but incomplete examples of QTIv2 being used, and no definitive reference implementation.

At the 2005 JISC/CETIS conference the UK assessment community confirmed that kick-starting the use of the IMS Question and Test Interoperability version 2 specifications was a high priority. However there was little incentive to move towards the new specification as existing public implementations are incomplete and difficult to use. The conference concluded that there needed to be a robust set of tools and services that conformed to the QTIv2 specification to facilitate this migration.

A central function that many systems require is that of rendering questions and responding to users answers. The R2Q2 project produced a core set of web services to provide this functionality.

Question and Test Interoperability

The IMS QTI Specification is a standard for representing questions and tests in XML to allow interchange. A QTI questions is broken down into three parts: ItemBody declares the content of the question itself, ResponseDeclaration declares a variable to store the studentís answer, and OutcomeVariables declares other resulting variables, in this case a score variable to hold the value of the result. In R2Q2 we focus on rendering and responding to the 16 different types of interactions described in version 2 of the QTI specification (QTIv2). These are:

1. Choice

9. Hotspot

2. Order

10. Select point

3. Associate

11. Graphic

4. Match

12. Graphic Order

5. Inline Choice

13. Graphic Associate

6. Text Entry

14. Graphic Gap Match

7. Extended Text

15. Position object

8. Hot Text

16. Slider

The list of different question types can be combined with templated questions or adaptive responses, providing an author with numerous alternative methods for writing questions appropriate to the needs of the students. A templated question provides variables in the item body that are instantiated when the question is rendered (for example, inserting different values into the text of maths problems). An adaptive question has a branching structure, and the parts that students see depends on their answers to each part of the branch. Up to sixty-four different combinations are possible.

Service Oriented Architectures

A Service-Oriented Architectures (SOA) allows large complex systems to be modularised in such a way that they are composed of independent software components that offer services to one another through well-defined interfaces. A service approach is ideally suited to loosely coupled systems, where services may be developed by different organisations. This is the philosophy behind JISCís e-Framework.

Due to the nature of the loose coupling in a SOA, applications can be developed and deployed incrementally. In addition, new features can be easily added after the system is deployed. This modularity and extensibility made an SOA approach especially suitable as a platform for an assessment system with evolving requirements and standards. The R2Q2 architecture implements the SOA of the e-Framework.

R2Q2 Design

The methodology used was an agile approach to design and implementation, which included the FREMA approach to designing Web services. We interviewed experts and stakeholders for a wide picture as to how the R2Q2 service might be used. In addition, we looked at the outputs from previous projects in this area and identified what not to do (lessons learned). An overview of the R2Q2 service requests and responses is shown in Figure 1.

overview

 

Figure 1 R2Q2 Overview

In the R2Q2 project we provided a definitive render and response processing engine for QTI version 2 question types. This was achieved by taking the requirements and refactoring them such that the main functions were divided between several co-operating Web Services. Figure 2 shows the core functions of the R2Q2 web service.

overview

 

 

 

Figure 2 The R2Q2 Service Architecture

The R2Q2 engine is a loosely coupled architecture comprising of three interoperable services. All the interactions with and within the R2Q2 engine are managed by an internal component called the Router.

The Router is responsible for parsing and passing the various components of the item (QTIv2) to the responsible web services. It also manages the interactions of external software with the system, and it is therefore the only component that handles state.

The Processor service processes the user responses and generates feedback. The Processor compares the userís answer with a set of rules and generates response variables based on those rules. The Renderer service then renders the item (and any feedback) to the user given these response variables.

We validated the system by using test questions from Graham Smith (these are recognised as being the gold standard for QTIv2). We also used the questions developed as part of the CATS project and questions from E3AN which have be upgraded to version two.

Conclusions

R2Q2 is a definitive response and rendering engine for QTIv2 questions. While this only deals with an Item in QTI terms, it is essential to all processing of QTI questions; that is, it forms the core component of all future systems. Due to the design and use of internal Web services, the system facilitates future enhancement and can be changed to suit any application.

The R2Q2 rendering and response engine of QTIv2 questions is expected to support two main categories of stakeholder:

More information, demonstrator and code can be obtained from the website the project Website http://www.r2q2.ecs.soton.ac.uk/.