IntroductionThe e-Framework Reference Model for Assessment (FREMA) project has built a community resource site for e-Framework Services within the Assessment domain, developed agile engineering methods for service construction, and developed two Service Usage Models (SUMs) which describe how services within the domain can fit together to collectively support a scenario. These SUMs are Summative On-line Assessment and Peer Review.
Summative On-line Assessment includes core topics such as Item Banking, Question Delivery, Mark Creation, Grade Allocation, and Feedback. There are many existing tools within this space, and ongoing efforts to build these into service architectures that can be used with the e-Framework (for example, the R2Q2 project).
Peer Review is a less well supported area. FREMA makes a distinction between Peer Review, the marking of a student’s work by their peers, and Peer Assessment, the marking of collaborative group work by a tutor (which may be modified by peer reflection). Peer Assessment can be a significantly complex process, and it is currently being investigated by the JISC funded WebPA project (led by Loughborough University) with the aim of producing a hardened web system to support it. Peer Review is a more straightforward process. In Peer Review, which can be both summative and formative, it is the peers who directly assess one another, in the form of a review, and who in turn may have those reviews assessed by a tutor. Peer Review is an important tool for giving feedback to large student cohorts when tutor time is limited, however it is also an important learning activity in its own right, and encourages students to develop the higher level functions from Bloom’s taxonomy: critique and evaluation of their peers, and ultimately reflection on their own work.
This project will undertake the development of an initial set of services from the Peer Review usecase. The aim is to provide lightweight services which may be reused within other group-oriented SUMs.
In order to create particularly lightweight services, the project will look at creating REST services to support the resource submission and distribution phases of Peer Review. These will be developed using the same agile methodology that the FREMA and R2Q2 projects have already used to successfully create SOAP services.
The aims of the project are to:
- Validate core parts of the FREMA Peer Review SUM
- Develop lightweight REST services for resource submission and distribution within groups
- Develop these REST services within an e-Framework Reference Model (FREMA)
- Demonstrate an agile software engineering method applied to REST services
Project DescriptionThe Peer Review SUM includes many use cases, for example:
- Create Student Pool – assigning students (and their profiles) into a pool from which groups will be chosen.
- Group Allocation from Pool – assigning students from a pool to a group. Different algorithms for group composition could be used (for example, random allocation, or based on profile information such as marks or background).
- Submission of Resources –group members submit resources to the group (to be reviewed).
- Distribution of Resources to Group –distributing a submitted resource to the members of a group based on a given pattern. Different distribution patterns could be used (for example, broadcasting a resource to all group members, or allocating each group member the resource of another group member).
- Submission of Reviews –group members submit reviews of given resources
- Feedback from Reviews –group members see the reviews of resources that they have submitted
- Retrieve Reviews – tutors retrieve the reviews that have been submitted by a given group member
The aim of this project is to develop lightweight resource distribution services that can be used with IMS Enterprise group data from the JoinIn services to fulfil the early stages of the Peer Review SUM. Together the services could be used to organise groups, and manage the flow of resources around group members (in particular, items for review, and the resulting reviews). These functions are important as they are generic services that could be reused within any SUM that deals with group work. They also have interesting characteristics that make a demonstrator desirable:
- They require low levels of security, and thus can be developed as lightweight REST services
- Lightweight services are more likely to be reused
- Will demonstrate REST services within a Reference Model and in the e-Framework
- Will demonstrate REST services developed using agile engineering techniques
- They have necessarily extensible parts (the resource distribution pattern)
- Demonstrates a common service interface used with different underlying logic
- Creates a context for later projects investigating novel peer review and assessment activities which may have more complex patterns
Project ManagementThe project will be led by the Learning Technologies Group at Southampton. Southampton will retain a senior researcher (Soton RF) to manage the project day to day, to organize the liaison with the wider community, to arrange the dissemination and to take responsibility for reporting. Southampton will also employ two programmers (RAs) to work on implementing the prototypes.
The project will begin with an initial project start-up face-to-face meeting with all those taking part in the project. Further team meetings will occur at monthly intervals to monitor progress against objectives. There will be a final, project closure meeting. Public versions of the minutes of these meetings will be published on the project Website. Each of the work packages will require formal review and sign-off meetings, and these are spaced a month intervals. There will be weekly technical meetings of the project staff. Financial reports will be supplied by ECS financial management, and a Final Report will be produced at the end of the Project.
1 JoinIn website: http://kn.open.ac.uk/public/index.cfm?wpid=6117
Work plan and DeliverablesThe project comprises four work packages (W1-W4) and seven deliverables (D1-D7). The deliverables are described in the work package entries below.
Workpackage 1: Refinement of Peer Review SUMFREMA has developed a Service Usage Model for Peer Review. In this early workpackage the services created in the JoinIn project will be mapped to this SUM, and new complimentary services for resource submission and distribution will be identified. At this stage the existing Service Expressions may also be refactored into a more granular design. The mapping and these refactored expressions will be recorded on the FREMA Community Wiki. Work will be commissioned from the JoinIn project to ensure accurate and effective exploitation of the JoinIn products within the PeerPigeon project, especially for deliverable 1, but including work related to subsequent deliverables 2 and 3.
Deliverable 1 (D1): Mapping of JoinIn Services to the FREMA Peer Review SUM
Deliverable 2 (D2): Refactored Service Expressions (SRCs and documentation) for the selected Resource Submission and Distribution Services
Workpackage 2: Development of Resource Distribution Service(s)The refinement of the Peer Review SUM will have identified service expressions that support Resource Submission and Distribution. This workpackage will realise these designs into working REST services by defining concrete operations and data models. At least two patterns of distribution will be implemented, based on the evidence for Peer Review located in FREMA. The code for these services will be made available on the project web site and documentation and descriptions will be added to the FREMA Community Wiki.
Deliverable 3 (D3): Code for Resource Submission and Distribution Service(s)
Deliverable 4 (D4): Documentation for Resource Submission and Distribution Service(s)
Workpackage 3: Simple Web-based DemonstratorThis workpackage will develop a simple web-based demonstrator that allows users to access the functionality of the services identified and developed in WP1 and WP2. This will also serve as a validation of the service design, and final test harness for the system as a whole. The demonstrator and its code will be made available on the project web site and documentation and descriptions will be added to the FREMA Community Wiki.
Deliverable 5 (D5): Simple Web-based demonstrator (both running demonstrator and code)
Deliverable 6 (D6): Documentation and User guide for demonstrator
Workpackage 4: Final reportThis workpackage will complete the report and entries in the FREMA Community Wiki.
Deliverable 7 (D7): Final report