StudentsurveysasmeasuresofeffectivelearningandteachinginAustralianuniversities

《StudentsurveysasmeasuresofeffectivelearningandteachinginAustralianuniversities》由会员分享，可在线阅读，更多相关《StudentsurveysasmeasuresofeffectivelearningandteachinginAustralianuniversities（8页珍藏版）》请在装配图网上搜索。

1、Student surveys as measures of effective learning and teaching in Australian universitiesJULIE ARTHURHEAD, EVALUATION UNITTEACHING & EDUCATIONAL DEVELOPMENT INSTITUTETHE UNIVERSITY OF QUEENSLANDAUSTRALIAASSOC PROFESSOR MARIAN TULLOCHDIRECTORCENTRE FOR ENHANCING LEARNING & TEACHINGCHARLES STURT UNIVE

2、RSITYAUSTRALIAABSTRACTThe evaluation of the effectiveness of learning and teaching in higher education in universities in Australia has traditionally been an area fraught with tensions and recent macro systemic changes in higher education policy have served to increase these. The introduction of the

3、 Learning and Teaching Performance Fund (LTPF) by the Department of Education, Science and Training as part of the Australian Governments Our Universities: Backing Australias Future (2002) reform initiative was a catalyst for both change and angst at the institutional level. To be eligible for fundi

4、ng universities were required to provide public web access to strategies, practices and policies relating to learning and teaching. Additionally, the aggregated results of student evaluations of subjects were to be available on websites. Student surveys were introduced in Australian universities in

5、the early 1990s and until the introduction of the LTPF, information collected was, in most cases, only provided only to staff members with a view to improving the quality of teaching and of subjects or programs. Historically, attempts to use student evaluation data at an institutional level drew ind

6、ustrial action by academic staff. This paper reflects on the challenges of institutional changes introduced in one Australian multi-campus regional university with regard to student surveys as one measure of assurance and enhancement of quality in learning and teaching. Lessons have been learned at

7、both the corporate and academic staff levels including around the links and tensions between formal accountability processes and academic engagement with student feedback to foster effective learning environments. Furthermore the paper discusses the implications of the elevation and adoption of stud

8、ent data as a quality measure of corporate performance, given the diversity of the learning and teaching provision through both face to face and distance learning modes and with a significant offshore student cohort.BackgroundThe current configuration of the higher education sector in Australia had

9、its begins following reform in 1988 which aimed to end the binary system of post secondary education provision that separated research and training. As with subsequent reforms in the sector, the Dawkins reform (named for the then Federal Minister for Education), the first and most impactful, was dri

10、ven by Australian Federal Government policies to shape the economic and social prosperity of the nation. Increased participation in education and training were linked with aspirations to position Australia well in the emerging global economy. The Unified National System emerged. The next 10 years we

11、re ones when much change occurred in the sector as identified in the terms of reference to the second wave of reform with the West report in 1998:Increases in access and participation have completed its transformation from an elite to a mass higher education system. At the same time, there has been

12、pressure both to restrain expenditure and to address the issues arising from the rapid expansion of the system, such as the need for quality assurance processes. Associated with this transformation have been other changes, including the removal of the binary divide in higher education, increased use

13、 of targeted research funding, and the increased use of advanced communications technology by institutions. (West Report, 1998, p.177)Although the recommendations of the West Report were not implemented due to changes in the government, subsequent reforms of the late 1990s focused on changes to fund

14、ing models. The third wave of change in higher education began with the Higher Education at the Crossroads discussion paper and subsequent Backing Australias future policy reforms. This most recent review highlighted several problems from the commonwealth government perspective. Essentially there we

15、re concerns that universities were not meeting student needs as best they could with high attrition rates, excess unmet demand and insufficient attention to equity concerns all highlighted. University governance was called into question with the senate and council configurations not being seen to me

16、et the strictures placed on boards of directors in the corporate sector. Throughout all reforms there has been a focus on research and the research productivity and economic capacity of the nation to compete on the global stage. As a result Australian universities have had a spotlight on various asp

17、ects of their operations for the past 15 years. First was structural change with amalgamations then cultural change and focus on building research and most recently the emergence of learning and teaching as performance indicators.The Australian reforms sit against those in other parts of the world.

18、Reviewing higher education is a world wide phenomena. The UK government (2003) white paper, “The future of higher education” which addressed higher education primarily in England, has characterised the diversity and mission of higher education institutes as encapsulating lifelong learning, research,

19、 knowledge transfer, social inclusion and regional economic development. It goes on to add “it is unreasonable to expect all higher education institutions to sustain all of these activities simultaneously at global, and not just national, levels of excellence.” (UK, 2003,p.22). This is the challenge

20、 not only for the UK, but also for universities world wide as the OECD recently noted:Public research institutions are being asked to contribute to economic development but also to be more responsive to evolving societal concerns such as food safety, environmental degradation, and health issuesWhat

21、is put into question is both the sole focus on scientific excellence and the criteria for judging this excellence when evaluating public researchers and research institutions. (OECD 2002, p55). One of the effects of reform processes is that staff in universities have been expected to be active in te

22、aching, scholarship and research, although recent developments in Australia have mooted changes to institutional foci to be restricted to either research or teaching. This is in line with international trends. In the UK there has been movement toward teaching-only universities with research funds re

23、directed to a limited number of institutions. Across Europe, the Bologna Process seeks to “harmonise higher education systems across nation states”In the US, higher education debate and the role of universities was in part triggered by Boyers (1990) seminal text Scholarship Reconsidered: Priorities

24、of the Professoriate in which he introduced to the debate on the role of universities the term Scholarship of Teaching and Learning. Boyer (1990) contended that students have important role in the evaluation of teaching scholarship and drew attention to universities where student evaluation was an i

25、ntegral part of the tenure process for teaching staff. Changes to academic workAccelerating change and financial constraints have characterised the last two decades of Australian higher education. Excellence in teaching and research is, increasingly, expected and sought after. The national reform ag

26、enda has emphasised the need for greater diversity among our universities and the vital importance to Australias knowledge-based future of research in universities. With universities as a whole being asked to diversify their portfolios academic staff are also being asked for greater productivity and

27、 quality. Expecting staff to be all things academic research, teacher, provider of community service - has been the subject of debate. Australian academics now feel they operate within a publish or perish” research dominated framework, that values certain types of research and devalues applied resea

28、rch and teaching. Increasing personal research outputs is felt to be an impost on top of increased student numbers and teaching expectations.The catch cry is more with less. Marginson and Considine (2000) identified the impact changes have had on the management structures, institutional systems and

29、frameworks within which academic staff seek to balance research and the scholarship of teaching and learning. research is managed. They noted a trend towards the commercialisation of research and the increasingly competitive nature of funding, and identified the results of these as being: a tendency

30、 to reduce risk, the manipulation of publication in order to maximise funding impact, an emphasis on quantity rather than quality; and a focus on the short-term. Elsewhere Marginson (2002) noted that between 1975-76 and 1997-98 student load in higher education rose by 127.9 % while total public fund

31、ing of higher education rose by just 26.1%, such that by the late 1990s core public funding of higher education per student was less than half the level of two decades earlier (Marginson 2002, p114). Staff student ratios had also increased.Quality Assuring Learning and TeachingThe West Review (1998,

32、 p.50) concluded that the promotion of high quality learning experiences for students was one of the principles underpinning the committees proposed higher education financing and policy framework. They reinforced the point thus:In an increasingly competitive market for the provision of educational

33、services, universities will need to seek continually to improve the range and quality of their offerings.(West, 1998, p.51) In the 18 years or so since the establishment of the “unified national system”, which abolished the binary divide separating Australian universities from teachers colleges, ins

34、titutes of technology and colleges of advanced education, “academic development” has been an emergent and increasingly important area of the higher education context. Building capacity in that which defines what a university is requires a strategic focus on all components of academia: research, teac

35、hing and service. Of these, identifying excellence in teaching has been the most problematic. In Australia, it has been a contested area, perhaps in part due to the cultural hang-over of the binary system. Simplistically viewed, it could be argued that colleges had focused on teaching and universiti

36、es on research and the combination of the two cultures led to disparities in the conceptualisation of university work. The reform context changed the ways in which staff in universities operated. Reform however, focused the sector on quality assurance, with the early rounds of QA highlighting the ne

37、ed for quality indicators. Research performance is reasonably easily quantifiable, based on number of citations (A), number of publications (B), and citation impact (A/B), but teaching excellence often relies on qualitative “objective” measures of outputs such as self report, peer assessment and stu

38、dent ratings which are often viewed dubiously and can be confounded by the fact that students are most likely to describe “effective teaching” as that which they perceive to assist their chosen approach to learning (Entwistle and Tait, 1990). Vidovich (2002, p.404) argues that despite the rhetoric a

39、bout the importance of external paying customers in QA policy for Australian higher education of the early 1990s, it was the Government which positioned itself as the primary customer of the outputs of universities. Furthermore, QA was about Government control and reforming the management of the sec

40、tor with QA policy being a managerial device (ibid). An alternative perspective focuses on the student. The current student cohort is more aware, correctly so as identified by Catts et al (2002, p.40), of the components of their courses and the structure and content of their subjects. They are corre

41、spondingly demanding of the nature of their experiences and current student attitudes provide a challenge to the academic and university alike. In a world more aware of consumer rights, “best practice” and “established procedures” students want and need qualifications acceptable to employers (and pr

42、ofessional associations) in a global market place. Haughey (p.28) contends that the impact of the socio-political changes correspondent with globalisation are reflected in the tertiary sector in the growing demand for post secondary spaces which he believes can only be met through the adoption of al

43、ternative strategies to campus-based instruction. He calls for the reorganisation of the traditional disciplinary information into interdisciplinary structures that relate specifically to business partners needs. Universities used to receive their kudos on the basis of the work of the outstanding ac

44、ademics, rather than in the present situation where the market approach from a student perspective places the standing of the certification as paramount. The quality assurance of learning and teaching is important to students.The Learning and Teaching Performance Fund(LTPF) was announced in 2003 as

45、part of the Australian Governments Our Universities: Backing Australias Future initiative. The purpose of the fund is to reward universities that best demonstrate excellence in learning and teaching. The fund will provide $54.4 million in 2006, increasing to approximately $82 million in 2007 and $10

46、9 million in 2008. The LTPF was initially two stage process. The first stage targeted policies related to learning and teaching and Universities were required to comply with Commonwealth expectations, particularly in relation to the public availability of student evaluation data, albeit at an aggreg

47、ated level. At a Federal level the government insisted that as a requirement for L&T performance funding a university must demonstrate that student evaluations are considered as part of the probation and promotion policies of the university.Student evaluations sit in an uncomfortable space between i

48、nstitutional audit/ accountability and the enhancement of learning and teaching. In the former context, students are seen as consumers with rights and expectations and evaluation becomes a form of customer satisfaction survey. Whether externally through the LTPF or internally through hierarchical re

49、porting, student evaluation data is used as an indicator of teacher and institutional performance. By contrast, a continuous enhancement model focuses holistically on the learning community where the student voice is valued and they are empowered to engage collaboratively with academics to improve t

50、he learning environment. While these two perspectives are not mutually exclusive, they represent very different discourses and often generate different mindsets and emotional responses in staff.Case Study: Charles Sturt UniversityCSU was formally established under the Charles Sturt University Act (1

51、989) as a multi-campus institution and now principally operates from the inland regions of Albury-Wodonga, Bathurst, Dubbo and Wagga Wagga. The objects of the University are the promotion, within the limits of the Universitys resources, of scholarship, research, free inquiry, the interaction of rese

52、arch and teaching, and academic excellence.The main campuses provide access to higher education across central (Bathurst), western (Dubbo) and south-western NSW and north-eastern Victoria (Albury). Additional specific-purpose campuses have been established at Goulburn (Policing), Manly, Sydney (Poli

53、cing) and Canberra (Theology). CSU also has multi site delivery through collaboration with third parties. For example, Charles Sturt University collaborates with NSW Police to deliver the Associate Degree in Policing Practice, the entry-level program for policing in NSW. International enrolments als

54、o come through third parties in Sydney, Melbourne and offshore in a number of countries, including China, Singapore, Malaysia, the United Kingdom, Hong Kong and Canada.CSU is a non-sandstone university born of the conception of the Unified National System. The parent entities for CSU were teachers c

55、olleges, colleges of advanced education and agricultural colleges. The transition from College of Advanced Education to University was for many amalgamated institutions a tortuous route through cultural and organisational changes that saw discipline groups pitted against each other to claim the high

56、 ground of intellectual discipline. The holy grail was research and the search for the holy grail was every bit as convoluted and treacherous as history portrays.Institutional PerformanceIt is within this context and set of challenges that the implementation of CSUs new online evaluation system need

57、s to be understood. The immediate impacts of the system have been a major enhancement of internal, hierarchical communication and an institutional aggregation of evaluation data. Heads of School have immediate access, once the student window is closed, to responses to core items for every subject an

58、d to school aggregates separated by mode. The consolidated school data is displayed each session on the CSU Learning and Teaching website providing the public accountability required for eligibility for Learning and Teaching Performance funding.The impact on an organisation of the reporting of insti

59、tutional data was evident in the response of the Senior Executive Group to the first report of the Charles Sturt University Student Experience Questionnaire (CSUSEQ) which carefully contextualised and interpreted the data. In the early stages of planning a five-year University strategy, student eval

60、uation was identified as a potential source of performance indicators, and an answer to the criticism in a previous external audit that CSU lack of outcome measures of teaching performance. For the first time the University had access to aggregated feedback from its current students, with the possib

61、ility that areas of concern could be addressed and student satisfaction raised before graduation. The official Course Experience Questionnaire (CEQ) is administered to students following graduation. Data are gathered from all institutions and used for comparative analysis by the Federal government.

62、Data are heavily lagged, thus failing to provide up to date feedback on students views. As such it is a very blunt instrument for assessing the impact of change, although remaining important because of it use in LTPF calculations. The Key Performance Indicators established by the University in its c

63、urrent planning phase include continuous improvement in SEQ and OES indicators at University, Faculty and School level. In addition CSU aims to be placed in the top 10 out of 38 universities for learning and teaching by 2011(in the first round of LTPF it ranked 17th ). The willingness of the Vice-Ch

64、ancellor to link his own performance and remuneration to the outcomes he has set for the University demonstrates his personal accountability and commitment to the new internally generated indicators which then generates expectations of Deans of Faculties and Heads of Schools. The audit/accountabilit

65、y messages grow louder from a Federal Government demanding evidence of quality teaching and rewarding excellence, and in the scrutiny of academic organisational units and their individual staff members on the adequacy of teaching performance. As the stakes around results increase so do the questions

66、 about the validity of feedback. At a national level, extensive consultation took place after the first round of funding and some modifications have already been made with further more fundamental changes foreshadowed for subsequent years. At the level of individual staff and AOUs, questions about the reliability and va