Effectiveness in delivery (3 Short Papers 0189, 0099, 0271)


10:50 - 11:50 on Wednesday, 8 September 2010 in Room 1


99 Can the computerized peer-assessment of digital stories fully assess student subject knowledge?
Phil Davies


189 INVISQUE - INteractive VIsual Search and QUery Environment
Raymond Chen, William Wong


271 Using a graphical user interface for generating reports from a complex curriculum knowledgebase
Tim Cappelli


99 Can the computerized peer-assessment of digital stories fully assess student subject knowledge?
Phil Davies
This presentation details the development of a student-based Digital Story as an alternative to producing an essay in assessing their knowledge in a particular subject area (e.g. Accessible Web Page Design). It also reports upon the use of the CAPODS (Computerized Assessment by Peers of Digital Stories) tool as a means of developing and assessing student reflective higher order skills. For peer-assessment to be considered effective then students will be rewarded in a qualitative manner for showing consistency and critical awareness in providing both comments and marks to their peers (semi-automatic via the CAPODS system). The development of a personalized Digital Story provides the student with a way of expressing their understanding of a particular problem whilst including appropriate academic references to support their views. Digital Stories have in the past mainly been used in the area of social reflection (2 to 3 minutes in duration) where individuals tell their own stories making use of various media resources. The key to this study is that the Digital Story produced is intended via ‘technological role-play’ to place the student in the position of an individual and then tell the story as if it were their own (e.g. a student suffering from colour blindness). An example will be presented with the final mark allocated to the student being an amalgamation of their peer-generated mark, a grade for performing the peer marking/ commenting in a qualitative manner and finally a tutor generated grade for their final submission. The development of this personalized Digital Story requires the student to fully understand the needs of an individual placed in a given scenario and all of the associated problems and possibilities, which may not have been the case if they merely developed an essay. The use of the peer-assessment aspect of the CAPODS system enhances the reflective and evaluative skills of the student within the assessment process.


189 INVISQUE - INteractive VIsual Search and QUery Environment
Raymond Chen, William Wong
Users of most electronic resource discovery systems (ERDS) are often faced with difficulties in making sense of the massive amounts of bibliographic data. Currently most ERDS present the search results in lists. Users have to scroll through a lot of pages and click on a lot of links to find the results they want. At the end, they often come to the problem of ‘what was I looking for?’ or 'getting lost in navigation'. This is a common issue in both academic and Internet resource discovery systems. The INVISQUE project, (Wong, et al, 2010a) was conceived to develop an interactive visualisation user interface to facilitate sense-making, query formulation, and information search showing in visual representations to support information search and retrieval tasks in large, loosely-coupled data sets, enabling the users to find relevant scholarly resources and to see the search process visually. Studies show that undergraduates, postgraduates and researchers find academic ERDS too complicated and difficult to use (Wong, et al, 2009). Therefore, many turn to Internet search engines to find scholarly articles. While Internet systems can provide quick search results, they do not offer analytical representations of the relationships between the data and can lead users to unreliable results (Jaccs, 2009). We investigated the problems that most ERDS have (Wong, et al, 2009). Novel concepts and ideas such as interactive visualisation, proximity to show associations between data, information layers to segregate search results, and visual workflow support for collating results were developed in a prototype to assess their feasibility in addressing the shortcomings (Wong, et al, 2010b). This new and potentially more intuitive way of searching is expected to help users learn from the searching process and to allow librarians and academics to discuss searching as part of the information skills learning process. The interactive functions in the system will allow the users to learn from relationships and associations (Palmer, 1994), retrieving and exploring on information which they might not know before. Collaborative functions will also allow the users to collaborate with others to learn, comment, share information and to solve the issues they encounter when searching.


271 Using a graphical user interface for generating reports from a complex curriculum knowledgebase
Tim Cappelli
University programmes are complex bodies of information; curricula, and associated pieces of data that constitute a complete programme are characterised by many inter-related elements. Administration and maintenance of a programme is similarly difficult when successful management is often dependent on extracting single data points and extrapolating connections to other parts of the programme. To assist with this, the institution has created a curriculum knowledgebase that stores and connects curriculum elements based on a data model. However, difficulties arise in visualisation and navigation of the different elements since the complexity of interrelationships makes it impossible to view all the connected instances for any given element; for example, the interrelationship between learning outcomes, activities and assessment. This has led to the development of a graphical user interface to visualise and navigate the various elements. The creation of this Java interface was designed as a context-neutral application that could be used with any data model. Thus, the model has been used on a number of other knowledgebases and could theoretically be used with any model based dataset. Once implemented and applied to real data, it was realised that the interface had the potential to provide a user-friendly mechanism for report generation. Creating reports from complex datasets is dependent on the inclusions of pre-scripted queries being built into the application and/or providing users with a report-builder. However, report builders frequently require users to have a knowledge of the data elements and their legal connections or expose the user to SQL code or Expressions. The GUI we created offers a visual mechanism for report building that makes such requirements redundant. The interface allows a report to be created incrementally using a series of images and tick boxes, with users choosing to filter at each stage of the process until a very specific report is created. This tool has great potential in that allows naive users the ability to easily interrogate any complex set of data, as long as the basic data model exists. This makes information far more accessible to students, managers and the public and has applications in administration and learning scenarios.