Quality Assurance System

BACKGROUND OF THE QUALITY ASSURANCE SYSTEM
ELABORATION IN THE SPECTRA CENTRE

The main objectives of the QAS implementation in the SPECTRA Centre are following:

• to enhance awareness of the need for improving the quality of research and education
• to contribute to the quality of research and education and other activities of the SPECTRA Centre
• to develop quality assurance system linked to existing evaluation system
• to support experience transfer and European dimension to evaluation

The main principles followed by the elaboration of the QAS are as follows:

• autonomy and independence in terms of procedures and methods concerning evaluation both from government and from higher education institutions
• self evaluation
• external evaluation by a group of external experts (the external evaluation group) and site visits
• publication of the reports as the main tool of public control

The quality assurance system consists of four phases of the evaluation process:

• self evaluation
• external evaluation
• self-evaluation report
• external-evaluation report

SELF-EVALUATION

Main purposes and steps

1. • The self-evaluation is the first step in the process. It is undertaken by the staff of the SPECTRA Centre. Through the self-evaluation report, the external evaluation group consisting of the members of Council of Guarantors will get a description and analysis of the SPECTRA Centre performance in the research and education. The goal of the self-evaluation is to illuminate the strengths and weaknesses within the general context of constraints and opportunities, in which the SPECTRA Centre operates.
   • The basis for the self-evaluation of the SPECTRA Centre is the action plan, which will be discussed internally. This should encourage the staff of the SPECTRA Centre to think of ways to implement change and introduce efficient quality control mechanisms in the units of the Centre for ongoing improvement and strategic planning and development
   • The ultimate purpose of the self-evaluation is to encourage all members of the SPECTRA Centre to think that ach of them is responsible for he quality and for the improvements introduced.

As the first step in the evaluation process the self-evaluation has three major purposes:

• to present a succinct, but comprehensive statement of the SPECTRA Centre view of its research, educational and other activities within the institutional context of the FA STU, taking into account the impact of research activities on the spatial planning practice and educational process

• to analyse the strengths and weaknesses of the SPECTRA Centre and to propose an action plan

• to provide a framework against which the SPECTRA Centre will be assessed by the external evaluation groups

The self-evaluation will result in a report, one of the key documents along with the external evaluation report. The self-evaluation report is the main vehicle for the SPECTRA Centre to convey information about itself. At the same time, it is the place for a critical reflection of the SPECTRA Centre about the way it manages itself and it handles quality as a central value in SPECTRA Centre's strategic decision-making.

The self-evaluation report will be not only descriptive, but also evaluative and synthetic, that is, it will evaluate strengths and weaknesses in the context of the constraints and opportunities and show the interconnection of various elements of strategic planning and quality management. In addition, the analysis should take into account the changes that have taken place in the recent past as well those that are anticipated in the future.

The Self-Evaluation Team

To ensure the success of the self-evaluation, it is important that the self-evaluation team will represent a broad view of the SPECTRA Centre rather than the specific view of its management.

The following characteristics will be desirable:

• the members are in a good position to judge strengths and weaknesses
• the group will consist of 2 members of the Council of Guarantors of the SPECTRA Centre (with the exclusion of the executive management of the Centre), one representative of the students and one representative of the academic staff of the Centre,
• the group will select a chairperson and a secretary, who will write the report under the chairperson's responsibility.

The self-evaluation team will be led by a chairperson whose responsibilities include:

• plan and co-ordinate the work of the self-evaluation group: e.g. prepare the checklist, gather and analyse the data
• provide opportunities for a broad discussion of the self-evaluation within the unit to promote identification with the report.
• Act as a contact person with the external evaluation group, with the contacts established by the international project elaboration
• Clarify the responsibility of the self-evaluation team towards staff members who are not in the team, i.e., the self-evaluation team should not work in isolation but seek, through SPECTRA Centre-wide discussions, to present as broad a view as possible of the SPECTRA Centre
• Support and spur the process along by explaining its worth and allaying fears

Purpose and Handling of the Checklist

Every item of the self-evaluation will consist of both a description and analysis including comments on how the SPECTRA Centre plans to remedy the problems identified. Facts and evaluations will be verified by the external evaluation stage, will be the extent to which the SPECTRA Centre's mission and goals have been met.

Checklist

Institutional context

• brief presentation of the recent state of the SPECTRA Centre (size, number of working units, number of staff, professional and gender profile of the Centre)

• brief description of the current management structure of the Centre, role and function of the top management members, as well as the committees, councils and other bodies of the SPECTRA Centre

• total number of the staff of the SPECTRA Centre and composition of the staff (research, teaching, administrative, technical)

• breakdown of staff by the grade and full/part time staff unfilled appointment

• organizational structure and position of the SPECTRA Centre within the overall institutional structure of the STU, FA STU

• procedure and criteria for appointment of the staff to the Centre

• identify the extent to which the aims, objectives and priorities at the institutional level are in synergy with those at the University and Faculty level

Aims and objectives

• Description of the aims and objectives of the SPECTRA Centre with respect to research and education in the context of:

• EU objectives,

• National objectives,

• Institutional missions and goals.

Research quality

• Overview of the research activities

• Attendance of the conferences

• Published papers in the proceedings of the conferences

• Involvement in research peer review

• Publishing of professional books

• Chapters in professional books

• Published teaching materials

• Training courses and spred of know-how

• Expertise and other interventions in the planning practice

• Publishing in professional journals with current contents abroad

• Publishing in domestic professional journals with current contents

• Feasibility studies, designs, plans, development projects

• Participation and awards in competitions

• Exhibitions

• Quatations according to SCI, AHCI

• Other references on research activities

• Membership in professional communities and boards

• Organisation of conferences, colloquia and workshops

• Organisation of summer schools for PhD students

• PhD theses

• Applications for domestic research projects

• Applications for international research and educational projects

Educational activities

• Organization of the programme

• Study programs and curricula

• Consistency of the curricular structure with the aims and objectives of the SPECTRA Centre

• Educational requirements

• Strengths and weaknesses

• Proposed action plan to remedy weaknesses and to develop strengths further on

• Teaching and training practice

• Teaching and training methods applied

• Encouragement of independent and team learning

• International dimension of the curricula

• Use of foreign literature in translation and in the language of origin

• Courses taught in other than the national languages

• Assessment methods used in the teaching and training process

• Admission date and input-output balance = ratio of admitted to enrolled students and participants in the courses

• Strengths and weaknesses

• Proposed action plan to remedy weaknesses and to develop strengths further

External relationships

• Ties with industry, commerce, government agencies, professionals bodies, spatial planning practitioners, in the context of the SPECTRA Centre's mission

• Ties with similar unites nationally and internationally in the context of the SPECTRA Centre' mission

• Participation in the European exchange programs and programs of the research and educational co-operation

• Foreign students and researchers enrolled in the SPECTRA Centre, with a breakdown by country of origin

• Staff members and students of the SPECTRA Centre who studied/worked abroad with a breakdown by host county

EXTERNAL EVALUATION

• The major aim of the external evaluation is to enhance the positive effect of the self-evaluation. The external evaluation will be a part of the evaluation of the research and educational units at the Faculty of Architecture of the Slovak University of Technology. The external evaluation will be realized by the evaluation visits of the external evaluation groups

• The external evaluation group`s functions are as follows:

• To verify the content of the self-evaluation report

• To give recommendations of areas that could be further developed

• To provide an opportunity for dialogue between evaluators and evaluatees that will strengthen the self-knowledge developed during the self-evaluation process

• The external experts will be selected on the basis of their knowledge and experience in the spatial planning research and education in Europe . The basis for the external evaluation will be the external evaluators visit and the internal evaluation report.

• External evaluation group will include minimum one representative from the spatial planning practice, one expert from the spatial research from other EU country, one expert from the other university in Slovakia

The preparation for the site visit will include two steps:

• Analyzing the self-evaluation report as well as any other available evaluation report
• A pre-visit meeting

General questions to be answered by analyzing the self-evaluation report:

• Is the report sufficiently self-reflective and analytical?
• Are the aims and objectives clearly expressed and academically rigorous?
• To what extent are the aims and objectives met?
• Are the strengths and weaknesses clearly formulated and substantiated?
• Are the recommendations to correct weaknesses and to build on strengths clearly

presented and practical?

• Is there any information missing?
• How did the self-evaluation process go?

EVALUATION REPORT

The external evaluation group will write an evaluation report, which will be discussed with the SPECTRA Centre Executive Management to correct any factual errors. Once this is done, the report becomes a public document. The major function of the report is to provide the SPECTRA Centre and its major stakeholders with a written document that serves as a record of its present state and a foundation for future developments and improvements.

The standard report format:

• Introduction

• Description and analysis

• Staff and management of human resource (staffing, responsibility, management)

• Facilities

• Research

• Education (budget, sources, infrastructural support …)

• External relations (links with the practice, academic links, participation in European programmes, internationalization of the Centre)

• Quality management (internal quality evaluation, administrative efficiency, procedure for innovations)

• Capacity for qualitative and quantitative development (mission, institutional policies, mid and long term strategies…)

• Conclusions (main findings, recommendations)

The draft will be distributed to the external team members for input as well as for their personal observations on the evaluation process.

When it is finalized, it will be made available to the SPECTRA Centre Executive Management for comments or corrections of factual mistakes and misunderstandings. The report and its summary will be discussed in the Council of Guarantors and presented to the Supervisory Board of the SPECTRA Centre as well as to the staff of the SPECTRA Centre.

CONCLUSIONS

Elaboration and implementation of Quality Assurance System using the experience of ARL Hannover, IOeR Dresden and University Newcastle upon Tyne will be the core instrument in the management of research and personal management of the SPECTRA Centre, with the goal to support the improvment of the performance of the staff and efficacy in using their capacity. This system of internal evaluation has been implemented for the first time when evlauationg the year of 2003, as the first project year of the SPECTRA-PERSEUS 5FP project. A very important part of the QAS System – external audit, led by representative of ARL as the twinning parnter, was realized as the activity A.5.3. in the frame of the SPECTRA-PERSEUS Project. The measures of the executive management of the SPECTRA Centre reacting to the recommendations from the external audit and their efficiency will be assessed in the next regulary evaluation in the QAS system.