Stage 1

Several studies will be undertaken concurrently under stage 1 of the project.

Study 1

Title:
International and national quality teaching and learning performance models currently in use .
Purpose:
To provide an overview of indicators and outcomes in use at the national and international levels on the quality of teaching and learning in Higher Education.
Outcomes:
This study will contribute an overview of national and international indicators and outcomes in current use. An analysis/critique can be made on types of the indicators in use and the relationship of these to teaching and learning quality.

Study 2

Title:
Asserted university teacher and teaching indicators and outcomes and their evidence base at institutional level and individual teacher level.
Purpose:
To provide an overview of indicators and outcomes at the institutional (including whole of institutional, faculty, school/department/program level) and individual teacher levels that claim to identify aspects of quality in teaching in HE. This will be primarily a literature review to look at what is proposed to be useful in identifying, understanding, developing, assessing quality teaching and learning in HE. Identifying the evidence available to show that what is proposed has a demonstrable effect on the quality of T&L will be an emphasis.
Outcomes:
This study will contribute an overview of indicators and outcomes that are asserted or are in use at the institutional and individual levels. Also documented will be the evidence base on which they are asserted. An analysis/critique can then be made on types of the indicators that are asserted, their evidence base and the claimed relationship of these to teaching and learning quality. Gaps in evidence/indicators can then be drawn out - this presumes a framework of quality teaching and learning.

Study 3

Title:
University teacher, teaching indicators and outcomes in use in Australian Universities (Survey of practice)
Purpose:
To provide an overview of indicators and outcomes in use at the institutional (including whole of institutional, faculty and program level) and individual teacher levels on the quality of teaching and learning in HE. This is primarily a survey of what is current practice in institutions. Sources of data will be policy, procedures, appointment and promotion criteria, performance management processes and outcomes related to teacher quality. Also of interest will be any student perception and learning indicators that are collected and used (Link with Study 6); Institutional/teaching quality indicators and assessment might include quality enhancement/assurance processes and procedures in relation to T & L; strategic planning and operationalisation strategies, curriculum review processes and use of that information, institutional systems for allocating funding on teaching quality internally using the operating budget, and also for allocating LTPF funding, reporting and evaluation strategies on the use and impact of the T & L funding.
Outcomes:
This study will contribute an overview of indicators and outcomes in use at the institutional and individual levels of Australian universities. An analysis/critique can then be made on types and extent of the indicators and measures that are currently in use at the teacher and teaching levels, and the extent of any commonality across these.

Study 4

Title:
Identify learning indicators and outcomes at the international, national, institutional and individual teacher levels
Purpose:
To provide an overview of learning indicators and outcomes that are used or asserted at the various levels. These are likely to include proxy measures. The aim is to look for evidence for asserting these as learning indicators, in use and proposed. Identify examples where the measures have implications for funding, rewards, recognition. Identify linkages between the levels eg do any of the learning indicators at the individual level scale up for use at the faculty level etc.
Outcomes:
This study will contribute an overview of learning indicators at the various levels, the evidence base, and the extent that they might be scalable, or assessable at the various levels.

Study 5

Title:
Student surveys on teaching and learning: Final Report. This report replaces the interim report published in October 2007.
Purpose:
To provide an overview of student perception surveys on T & L in use in Australian universities/HE and some commentary and evaluation of them. This project has been commissioned and is undertaken by a team led by Associate Professor Simon Barrie, University of Sydney.
The project will be undertaken in two stages:
Stage 1 will provide a preliminary descriptive framework to facilitate identification of commonalities in the use of student perception/feedback across Australian universities.
Stage 2 will improve the commonality in uses of student feedback and provide a sound rationale for the framework developed in stage 1.
Outcomes:
This study provides an environmental scan and initial analysis of within-university SET practices across 29 Australian universities. The report is based on data collected in the first half of 2007 and as such represents a 'snapshot' of institutional SET practice which may well have changed since. The report analyses the core items drawn from internal student evaluation of teaching surveys currently in use in these Australian universities using a framework derived from a key review of the major multi-section validity studies of SET. It considers patterns of use of SET items in Australian universities and suggests how the analytic framework might be developed for use in the Australian context. Based on the findings of the analysis the report provides some preliminary conclusions and suggestions to better enable institutions to use internal SET data for benchmarking and quality assurance. The report also identifies a range of validated survey scales which could be used to gather SET data in relation to the proposed TQI dimensions (Chalmers 2007) and identifies a network of SET experts with the potential to collaborate in developing and validating shared SET items, scales and procedures, for those levels of the proposed TQI dimensions where no suitable validated SET scales were identified.
This report does not provide prescriptive recommendations as its intention is to support discussion on the current SET use. However in the context of the overarching Teaching Quality Indicators project a provisional set of survey items will be identified for each of the proposed TQI dimensions for use at the level of individual subjects. The items will be posted on this website shortly and will provide the basis for further consultative development and validation by universities participating in the Teaching Quality Indicators project.
Contact details:
Comments on this report are welcomed. Please contact A/Prof Simon Barrie at The University of Sydney S.Barrie@usyd.edu.au .

Synthesis of outcomes of studies and development of framework

Six reports will be written - one per study.
Three further reports will be written

1. DEST commissioned report: "Review Australian and international performance indicators and measures of quality of teaching and learning in higher education".
2. Summary of the 6 reports.
3. Framework outline (draft)

Development of Framework

1. From the six studies, a framework will be developed that will encompass the institutional and individual levels. The institutional level may be broken down to include program, department/school, faculty as well as overall institution. Potential benchmarking and national indicators will be flagged.
2. The draft framework will be canvassed widely for discussion and comment, particularly the pilot institutions, the wider group of DCV/PVC(A) members of the AVCC and via them to their institutions, AUQA, DEST.
3. The framework will be revised in light of the comments and will then go to the pilot universities for their further consideration and detailed input.
4. Final revision of the framework will then take place prior to Stage 2 implementation.

Stage 1 Implementation and approach

1. A project team will be established by Denise Chalmers and Professor Judyth Sachs. Denise Chalmers will lead the Carrick Institute project and Stage 1. Professor Judyth Sachs will lead the pilot institutions.
2. The project will be coordinated and funded by the Carrick Institute.
3. The team will carry out stage 1 of the project and identify pilot universities for stage 2.
4. The AVCC group of DVC/PVC(A)s have agreed to the project in principle at their Nov '06 meeting, attended by Prof Judyth Sachs and Denise Chalmers. A letter confirming this support was provided by Prof Peter Booth, Chair of this group to Prof Richard Johnstone.
5. A Reference Group comprising members of the DVC/PVC(A), DEST, AUQA and key stakeholders has been established. The Reference Group will meet with the project team leaders at least twice yearly (see table for membership categories of the project team and reference group).
6. Principles that underpin the Framework will be developed through a process of wide consultation and with the agreement of the sector.
7. Evaluation and dissemination strategies will be embedded throughout each of the stages. Engaged dissemination strategies, as outlined in the Carrick Dissemination Framework, will be the main strategies utilised.
8. The framework will be developed to include shared indicators and outcomes that can be used as benchmarks across the system. In addition, flexibility will be a feature so that institutions can capture particular dimensions that might be common through established associations (eg Go8, ATN, regional) or individual institutional missions.
9. The project will be collaborative, with institutions involved as valued members of the project and the process.

Project and Reference Group membership