The e-Framework Reference Model for Assessment (FREMA) project has built a community resource site for e-Framework Services within the Assessment domain, developed agile engineering methods for service construction, and developed two Service Usage Models (SUMs) which describe how services within the domain can fit together to collectively support a scenario. These SUMs are Summative On-line Assessment and Peer Review.

Summative On-line Assessment includes core topics such as Item Banking, Question Delivery, Mark Creation, Grade Allocation, and Feedback. There are many existing tools within this space, and ongoing efforts to build these into service architectures that can be used with the e-Framework (for example, the R2Q2 project).

Peer Review is a less well supported area. FREMA makes a distinction between Peer Review, the marking of a student’s work by their peers, and Peer Assessment, the marking of collaborative group work by a tutor (which may be modified by peer reflection). Peer Assessment can be a significantly complex process, and it is currently being investigated by the JISC funded WebPA project (led by Loughborough University) with the aim of producing a hardened web system to support it. Peer Review is a more straightforward process. In Peer Review, which can be both summative and formative, it is the peers who directly assess one another, in the form of a review, and who in turn may have those reviews assessed by a tutor. Peer Review is an important tool for giving feedback to large student cohorts when tutor time is limited, however it is also an important learning activity in its own right, and encourages students to develop the higher level functions from Bloom’s taxonomy: critique and evaluation of their peers, and ultimately reflection on their own work.

This project will undertake the development of an initial set of services from the Peer Review usecase. The aim is to provide lightweight services which may be reused within other group-oriented SUMs.

In order to create particularly lightweight services, the project will look at creating REST services to support the resource submission and distribution phases of Peer Review. These will be developed using the same agile methodology that the FREMA and R2Q2 projects have already used to successfully create SOAP services.

The aims of the project are to:

Project Description

The Peer Review SUM includes many use cases, for example: The early stages of this process, concerning group allocation, have been mostly addressed by the JISC JoinIn demonstrator project that completed in August 20061. JoinIn have developed a number of services to manage users and groups, and are compliant with the IMS Enterprise specification.

The aim of this project is to develop lightweight resource distribution services that can be used with IMS Enterprise group data from the JoinIn services to fulfil the early stages of the Peer Review SUM. Together the services could be used to organise groups, and manage the flow of resources around group members (in particular, items for review, and the resulting reviews). These functions are important as they are generic services that could be reused within any SUM that deals with group work. They also have interesting characteristics that make a demonstrator desirable: This project with therefore concentrate on developing REST services that can be used alongside IMS Enterprise compliant group systems to support this highly re-usable part of the Peer Review SUM.

Project Plan

Project Management

The project will be led by the Learning Technologies Group at Southampton. Southampton will retain a senior researcher (Soton RF) to manage the project day to day, to organize the liaison with the wider community, to arrange the dissemination and to take responsibility for reporting. Southampton will also employ two programmers (RAs) to work on implementing the prototypes.

The project will begin with an initial project start-up face-to-face meeting with all those taking part in the project. Further team meetings will occur at monthly intervals to monitor progress against objectives. There will be a final, project closure meeting. Public versions of the minutes of these meetings will be published on the project Website. Each of the work packages will require formal review and sign-off meetings, and these are spaced a month intervals. There will be weekly technical meetings of the project staff. Financial reports will be supplied by ECS financial management, and a Final Report will be produced at the end of the Project.
1 JoinIn website:

Work plan and Deliverables

The project comprises four work packages (W1-W4) and seven deliverables (D1-D7). The deliverables are described in the work package entries below.

Workpackage 1: Refinement of Peer Review SUM

FREMA has developed a Service Usage Model for Peer Review. In this early workpackage the services created in the JoinIn project will be mapped to this SUM, and new complimentary services for resource submission and distribution will be identified. At this stage the existing Service Expressions may also be refactored into a more granular design. The mapping and these refactored expressions will be recorded on the FREMA Community Wiki. Work will be commissioned from the JoinIn project to ensure accurate and effective exploitation of the JoinIn products within the PeerPigeon project, especially for deliverable 1, but including work related to subsequent deliverables 2 and 3.

Deliverable 1 (D1): Mapping of JoinIn Services to the FREMA Peer Review SUM

Deliverable 2 (D2): Refactored Service Expressions (SRCs and documentation) for the selected Resource Submission and Distribution Services

Workpackage 2: Development of Resource Distribution Service(s)

The refinement of the Peer Review SUM will have identified service expressions that support Resource Submission and Distribution. This workpackage will realise these designs into working REST services by defining concrete operations and data models. At least two patterns of distribution will be implemented, based on the evidence for Peer Review located in FREMA. The code for these services will be made available on the project web site and documentation and descriptions will be added to the FREMA Community Wiki.

Deliverable 3 (D3): Code for Resource Submission and Distribution Service(s)

Deliverable 4 (D4): Documentation for Resource Submission and Distribution Service(s)

Workpackage 3: Simple Web-based Demonstrator

This workpackage will develop a simple web-based demonstrator that allows users to access the functionality of the services identified and developed in WP1 and WP2. This will also serve as a validation of the service design, and final test harness for the system as a whole. The demonstrator and its code will be made available on the project web site and documentation and descriptions will be added to the FREMA Community Wiki.

Deliverable 5 (D5): Simple Web-based demonstrator (both running demonstrator and code)

Deliverable 6 (D6): Documentation and User guide for demonstrator

Workpackage 4: Final report

This workpackage will complete the report and entries in the FREMA Community Wiki.

Deliverable 7 (D7): Final report