Proposal view
Proposal Type: Symposium 
Domain: Assessment and Evaluation 
SIG: Higher Education 
Scheduling category: Assessment of Competence 
Type Submitted Symposium 
Title Assessing vocational competence in higher education: Comparing different methods throughout Europe 
Abstract

Social scientists and governments throughout Europe argue to use the development of competence as an explicit goal of higher education. For example, the Bologna process, a political association of 45 European countries, aims to make education more vocationally oriented by stimulating the development of competences. Researchers are currently investigating how teaching and assessment can improve competence development. However, many different definitions of competence and methods to assess (elements of) competence development are used throughout Europe. Therefore, this symposium compares different methods to assess vocational competence in different countries, takes into account the different definitions of competence and investigates factors (such as teaching) contributing to competence development.

Each paper focuses on a different method of assessing competence. Stenstrom investigates the use of skills demonstration at the workplace as part of Finnish vocational programmes, in which the cooperation between education and working life plays a central role. Braun measures competence development by means of a self-report questionnaire and studies the relationship between teachers’ approaches and students’ self-reported competence. Bieri and Schuler focus on an assessment centre as part of a selection process for teacher education and study the relationship between self-ratings and ratings given by others. Finally, Baartman and De Bruijn present a literature study into the integration processes underlying competence development, distinguishing between integration between different types of knowledge, acquired in different learning situations such as school and work, and connected with skills and attitudes.   

 Contributions:

  1. Stenstrom: Assessment of competence through skills demonstrations at the workplace
  2. Braun: Assessing students’ competence by means of self-report questionnaire and the influence of teachers’ conceptions of learning
  3. Bieri and Schuler: Assessing future teachers’ competence in an Assessment Centre
  4. Baartman and De Bruijn: Assessing vocational competence aimed at the integration of knowledge, skills and attitudes
 
Equipment Computer and data projector / beamer
Keywords Assessment of Competence
Higher education
Vocational education 
Chairperson list
First Name Last Name/Surname Institution Country E-Mail EARLI Number
Liesbeth Baartman Utrecht University Netherlands l.k.j.baartman@uu.nl  
Organiser list
First Name Last Name/Surname Institution Country E-Mail EARLI Number
Liesbeth Baartman Utrecht University Netherlands l.k.j.baartman@uu.nl  
Edith Braun Freie Universitaet Berlin Germany edith.braun@fu-berlin.de  
Discussant list
First Name Last Name/Surname Institution Country E-Mail EARLI Number
David Gijbels University of Antwerp Belgium david.gijbels@ua.ac.be  
Paper Details
Paper type Empirical
Title Assessment of Competence through Skills Demonstrations at the Workplace
Abstract

In Finland, demonstrations of vocational skills have been developed to form part of vocational study programmes and to improve the relationship between education and working life. Vocational skills demonstrations represent a new form of student assessment, in which cooperation between education and working life plays a central role and which brings together representatives from working life and teachers (Stenström, 2001; 2005; Stenström, Laine, & Kurvonen, 2006).
The aim of this study is to describe assessment of competence through vocational skills demonstrations of Finnish vocational education and training carried out at workplace settings. This paper is based on a European study, the aim of which was to examine practice-oriented assessment as part of vocational education provision in authentic situations at the workplaces (Stenström & Laine, 2006). The Finnish context was at the experimental phase of introducing new vocational skills demonstrations; skills demonstrations were included in all initial vocational qualifications in autumn 2006. The data were collected by thematic interviewing of students in the fields of Social and Health Care and Construction (n=6), their teachers (n=8), and representatives of enterprises (n=6) operating in these fields in central and southern Finland. The interviews were analysed by means of content analysis. The results show that vocational skills demonstrations have contributed to the creation of closer links between education and work, closer collaboration between teachers and workplace instructors in the assessment process, and closer integration of theory and practice. However, there are challenges to developing assessments of vocational skills. The issues concern assessment criteria, targets, assessor training, assessment methods, resources and tasks of the skills demonstrations (Stenström 2008).

Summary

Aims
Vocational skills demonstrations represent a new form of student assessment, in which cooperation between education and working life plays a central role and which brings together representatives from working life and teachers (Stenström, Laine, & Kurvonen, 2006). The aim of this study is to describe assessment of competence through vocational skills demonstrations of Finnish vocational education and training carried out at workplace settings. The main research question is: What are the key dimensions of assessment of competence in vocational skills demonstrations? The sub-questions are as follows:
- What is the focus of assessment?
- Who are the assessors?
- How is competence assessed?
- Where is assessment of competence taking place?

Data and Methods
This paper is based on a European study, the aim of which was to examine students’ practice-oriented learning as part of vocational education provision in authentic situations at the workplace (Stenström & Laine, 2006). The context was the experimental phase of vocational skills demonstrations in Finland. Skills demonstrations were included in all initial vocational qualifications in autumn 2006. The data were collected by interviewing students in the fields of Social and Health care and Construction (n=6), their teachers (n=8), and representatives (n=6) of enterprises operating in these fields in central and southern Finland. The interviews were analysed by means of content analysis.

Findings
What is the focus of assessment?
In Finnish vocational education and training, the central competencies of each vocational study module are assessed by means of vocational skills demonstrations. The central competencies, as targets of assessment, are command of knowledge that forms the foundation of the work domain, command of work processes, command of working methods, tools and materials, command of occupational safety,  common emphases, and core skills common to all fields (Kinnunen, 2005, 70). The content of the Finnish vocational curriculum resembles Eraut’s typology (2004), although the concepts are different. In general, the targets of assessment of Finnish vocational education largely cover the vocational competencies including key skills. Therefore, the focus of assessment in vocational skills in Finland seems to be broader than in most other European countries (Stenström & Laine, 2006).
Who are assessors?
The most prominent principle guiding the implementation of vocational skills demonstrations in Finland is assessment by three actors: the student, the teacher and the workplace instructor (Stenström et al., 2006). Teachers have the main responsibility for the implementation of the assessment and they must ensure that the grades given are based on the agreed criteria. Workplace instructors are valued for their professional and workplace-specific expertise. Assessment by a professional is one of the main strengths in vocational skills demonstrations. The students’ most salient task within the assessment process is to demonstrate their skills. Alongside practical knowledge, students also show theoretical knowledge.
How is assessment carried out?
The most common methods in vocational skills demonstrations are observation, discussion and written assignments. The assessment discussion plays a central role in the vocational skills demonstration process, both as a whole and in relation to guiding the growth of students’ vocational competence (Stenström et al., 2006). The students have to prepare a written plan of their demonstration before the demonstration itself.
Where is assessed?
A workplace is a typical context of practice-oriented assessment, which can sometimes alternatively be carried out in a simulated environment. Teachers and workplace instructors alike emphasise the authenticity of vocational skills demonstrations. According to them, a genuine working situation is best able to bring out a student’s vocational skills.

Conclusion
The main results of assessment of competence through skills demonstrations are shown by the adapted assessment triangle (cf. NCR 2001) in Figure 1.

Figure 1. Assessment of competence in Finnish VET

Theoretical and Educational Significance
The paradigm of assessment has shifted from measurement to evaluation. Birenbaum (1996) distinguishes between two cultures of performance assessment: the culture of measurement and the culture of evaluation. The culture of evaluation is seen as an alternative to the culture of testing (Birenbaum & Dochy, 1996). This type of assessment is also called authentic assessment, performance assessment, alternative assessment or practice-oriented assessment as in our case (QUAL-PRAXIS project, Stenström & Laine, 2006). The new point of our study is that competence-based examinations have been used mostly in countries where vocational education is driven by working life (Eraut, Steadman, Trill, & Parker, 1996; Wolf, 1995). This study of skills demonstrations shows that the implementation of the demonstration of vocational skills has improved cooperation between education and working life (Stenström et al., 2006). However, there are challenges for developing assessment of vocational skills.

References

Birenbaum, M. (1996). Assessment 2000: Towards a pluralistic approach to assessment. Teoksessa M. Birenbaum & F. Dochy (Eds.), Alternatives in assessment of achievement, learning processes and prior learning. Boston: Kluwer, 3–30.

Birenbaum, M. & Dochy, F. (1996). Introduction. In M. Birenbaum & F. Dochy (Eds.), Alternatives in assessment of achievements, learning processes and prior learning. Boston: Kluwer, xiii–xv).

Eraut, M. (2004). Transfer of knowledge between education and workplace settings. In H. Rainbird, A. Fuller & A. Munro (Eds.), Workplace learning in context (pp. 201-221). London: Routledge.

Eraut, M., Steadman, S., Trill, J., & Parker, J. (1996). The assessment of NVQs. (Research Report No. 4). Brighton: University of Sussex, Institute of Education.

Kinnunen, E. (2005). Targets and criteria of assessment. In Kinnunen, E. & Halmevuo, T. (Eds.), Student assessment guide for vocational education and training (pp. 66-76). Helsinki: Finnish National Board of Education.

Stenström, M-L., & Laine, K. (Eds.) (2006). Quality and practice in assessment: New approaches in work-related learning. Jyväskylä: University of Jyväskylä, Institute for Educational Research.

Stenström, M-L., Laine, K. & Kurvonen, L. (2006). Practice-oriented assessment in Finnish VET - Towards quality assurance through vocational skills demonstrations. In M-L. Stenström & K. Laine (Eds.), Quality and practice in assessment: New approaches in work-related learning (pp. 89-120). Jyväskylä: University of Jyväskylä, Institute for Educational Research.

Wolf, A. (1995). Competence-based assessment. Bury St Edmunds, UK: St Edmundsbury Press.

Keywords Assessment of Competence
Collaboration
Vocational education
Appendices Symposium assessment_Stenstrom_Figure 1.jpg 
Authors
Name Surname Institution Country e-mail EARLI Number Presenting
Marja-Leena Marja-Leena Stenstrom University of Jyvaskyla Finland Marja-Leena.Stenstrom@ktl.jyu.fi   *  
Paper type Empirical
Title Assessing Students’ Competence by Means of Self-report Questionnaire and the Influence of Teachers’ Conceptions of Learning
Abstract

Supporting students’ gain in vocational competences will become a main task in higher education – against the background of the Bologna Process. Our research uses a self-report questionnaire, the HEsaCom (Braun & Leidner) to ask students about their subjective gain in vocational competence. We relate these outcomes to teaching variables, to identify variables that are responsible for students’ view of gain in competences. Therefore, 451 students were asked about their gain in vocational competences using the HEsaCom, and 68 lecturers were surveyed about their approaches to teaching (using Prosser and Trigwell’s ATI) and their applied teaching techniques. Our results show that students report higher gains in competence when their teachers’ conceptions of learning are more student-focused. Given that prerequisite, the students rate a higher increase in factual and vocational competences. Furthermore, specific teaching techniques are recognized as having a positive influence on students’ reported gain in competences. In academic courses in which interactive techniques were applied, students report more development in competences. Again, the use of techniques depends on lecturers’ approach to teaching. We conclude that teachers’ conceptions of learning are highly powerful in influencing students’ competence development.

Summary

Aims
Our thesis has been that more student-focused teachers support a higher gain in students’ competences. Moreover, we want to identify specific didactic techniques, which support students’ perceived gain in competences using a self-report questionnaire.
As prior research has shown, teachers who think of learning as knowledge transfer (teacher focused orientation) are going along with students who have a surface approach to learning. On the other hand, teachers who believe in learning as an active re-constructing process (students focused orientation), are found to support students to use a more deep learning approach (Trigwell, Prosser, Waterhouse, 1999). Additionally, students in a problem-based learning scenario show an approach more towards constructivism (Gijbels, van de Watering, Dochy, and van den Bossche, 2006). Generally speaking, there are findings confirming that learning environments and teachers’ approaches are connected to students’ learning processes.
The Bologna Process, the current higher education reform in Europe, emphasises education aimed towards vocationally orientated competences. Therefore, before we go on, competences need to be defined. Weinert’s (2001) definition is used, which encompasses more than expert knowledge about a given field; systematic aspects (e.g., problem-solving skills), personal (e.g., motivation) and social (e.g., cooperation skills) aspects are of importance for competent performance, too. In our studies we use students’ subjective reports of their acquisition of six domains of competences (Braun & Leidner).
1) Knowledge processing: students’ skills in applying their knowledge and skills in analyzing their applications of what they have learnt.
2) Systematic competence: an individual’s skill in effectively planning his or her work as well as the mastery of relevant techniques.
3) Presentational competence: students’ learning to speak in front of others, and consequently improve their oral presentation skills.
Social competence is defined as the achievement of a balance between the development of positive outcomes for oneself on the one hand and adherence to context-specific expectations for behaviour by others on the other (Wentzel, 1999). We differentiate social competence in:
4) Communication competence: the ability to verbally assert one’s own opinion and to productively discuss it with others.
5) Cooperation competence: being capable of working with others effectively.
6) Personal competence: an individual’s positive attitude towards learning and development of the self.

Students’ quantitative and qualitative gain in vocational competences seems to be dependent on their teachers’ approach on learning, and the use of specific teachings techniques is highly probable moderated by teachers’ approach. Therefore we want to investigate the complex interaction between students’ view on their gain in competences, teachers’ approach to learning and the used didactic techniques.

Data and Methods
This research was conducted within 68 academic courses at the Freie Universität Berlin. In our sample, 68 lecturers and their 451 students have been surveyed. The lecturers filled out a German Version of “Approaches to Teaching Inventory“ (ATI; Trigwell and Prosser, 2006). The German version contains eleven questions, five questions regarding a teacher focused orientation and six questions regarding a student focused orientation.
Additionally, the lecturers have been asked about used didactic techniques. Therefore, 14 specific techniques have been derived from literature of training in higher education teaching.
The students filled out the HEsaCom (higher education, self assessed competences, Braun & Leidner, in press) which asks for self-reported competence acquirements within academic courses. To analyze empirical coherences, multi-level analyses as well as bivariate correlations have been have been conducted.

Findings
Here, we will only report statistically significant results. As expected from our hypotheses, students of teachers with higher student-focused orientations report higher gains in all six competences. Moreover, teachers with higher student orientation use more and different teaching techniques.
Concerning teaching techniques we found a noticeable influence of specific teaching techniques: especially simulation of authentic problems correlates highly with students’ reported gain in competences. Also, if teachers more often critically enquire the dealt subject, students report higher gains in competences. Though, this does not apply for invitation of professionals nor the frequency of group work.

Theoretical and Educational Significance
Being consistent with studies conducted by other authors, our results show significant influences of the teacher’s approach on students’ learning. Students´ and teachers´ views, surveyed independently from each other, are related to the teachers’ approaches to learning. We conclude that teachers with higher student focused orientation use more and different teaching techniques, which has a positive outcome in terms of students’ view of gain in vocational competences. One main aspect to meet the new criteria in higher education is to develop a more student focused orientated teaching approach. Higher education will probably need to train lecturers towards a more student-focused orientation.

References

Braun, E., & Leidner, B. (in press). Theoretical and empirical distinctions between self-rated gain in competences and satisfaction with teaching behaviour within academic course evaluation. European Psychologist.

Coatsworth, J. D., & Masten, A. (1998). The Development of Competence in Favorable and Unfavorable Environments: Lessons from Research on Successful Children. American Psychologist, 53, 205-220.

Gijbels, D., Van de Watering, G., Dochy, F., & Van den Bossche, P. (2006). New learning environments and constructivism: The students' perspective. Instructional Science, 34, 213-226.

Prosser, M., & Trigwell, K. (2006). Confirmatory factor analysis of the Approaches to Teaching Inventory. British Journal of Educational Psychology, 76, 405-419.

Trigwell, K., Prosser, M., & Waterhouse, F. (1999). Relations between Teachers’ Approaches to Teaching and Students’ Approaches to Learning. Higher Education, 37, 57–70.

Weinert, F.E. (2001). Concept of competence: A conceptual clarification. In D. Rychen & L Salganik (Eds.), Defining and selecting key competencies (p. 17-31). Goettingen: Hogrefe.

Wentzel, K.R. (1999). Social?motivational processes and interpersonal relationships: Implications for understanding students' academic success. Journal of Educational Psychology, 91, 76-97. 

Keywords Assessment of Competence
Higher education
Instructional Strategies
Appendices
Authors
Name Surname Institution Country e-mail EARLI Number Presenting
Edith Braun Freie Universitaet Berlin Germany edith.braun@fu-berlin.de   *  
Paper type Empirical
Title Assessing Future Teachers’ Competence in an Assessment Centre
Abstract

There is an increasing interest in teachers’ cross-curricular competencies as a result of reforms in teacher training and the Bologna Process. Teachers need a variety of social skills in order to succeed in their professional career. At the University of Teacher Education in Zurich, future students who do not account for the appropriate formal qualifications, are required to pass several examinations and to undergo an assessment centre. The underlying model of competencies is currently being evaluated in a longitudinal study. The initial findings regarding assessment centre validity and self-other agreement are presented in this study. Furthermore, the implications for practice are discussed.

Summary

Aims
It is essential that teachers have a variety of social skills at their disposal in order to succeed in their professional career. As a consequence, competence models have been developed in recent years with the aim of assessing teachers and future teachers’ competencies. 
As Swiss student teachers gain practical experience in the classroom at a very early stage of their studies, they require high competencies right from the beginning of their studies. Several models to assess future teachers’ self-ratings such as the ‘CCT – Career Counselling for Teachers’ project (e.g., Mayr, 2001; Märki, 2006) have been developed. However, none of these models combines self-reports with ratings provided by others. Based on the theory of teachers’ professional competencies (e.g., Oser & Oelkers, 2001; Terhart, 2002; Herzog, 2002; Frey, 2004; Darling-Hammond & Bransford, 2002; 2005; Schaarschmidt, 2004) and a general job analysis, the following model to assess future students’ competencies, combined with self- and other-ratings, has been developed at the University of Teacher Education in Zurich. Future students who do not account for the appropriate formal qualifications are required to pass several examinations and to undergo an assessment centre. (In Switzerland, higher education is organized as a highly selective system. Only 20% of young adults achieve the formal qualifications to be admitted to University, and 64% of all adolescents go through vocational training.) The goal of the assessment centre, which is part of the selection process of the University, is to match students’ individual competencies and interests with the University’s requirements. A second objective is to assess the individual’s potential that can further be developed during the study. 
Assessing future teachers’ competence in an assessment centre provides the opportunity to combine self- and other ratings. Research has demonstrated that self-other agreement is related to individual outcomes such as performance. Self- and other ratings or perceptions that are in agreement are preferable because they indicate a degree of mutual understanding and accuracy. Accurate self-ratings correlate with self-awareness. In addition, self-awareness has been noted as an important element of emotional intelligence, which, in turn, has been associated with effective leadership (Atwater, Waldmann, Ostroff, Robie & Johnson, 2001, 2005). Effective leadership is one of the most important prerequisites for teachers’ classroom management. 
An assessment centre is a useful standardized method for measuring multiple competencies that are otherwise difficult to measure. Assessment centres are usually able to predict future success, they do not cause adverse impact, and they are seen as fair by participants. They provide detailed information on individuals to give them a feedback on their strengths and weaknesses (Thornton & Rupp, 2006). In summary, assessment centres show considerable validity when used for selection processes. Nevertheless, they suffer from low construct validity (Kleinmann, 1997).
The underlying model of competencies to assess future teachers’ competence is currently being evaluated in a longitudinal study, planned with several points in time. The research questions are as follows: 1) What groups of students are at a disadvantage? Do background factors such as gender, level of education and intelligence predict success? 2) How valid is the Assessment Centre? 3) To what extent are self-ratings associated with ratings provided by others?

Data and methods
A total number of 330 future students, participating in the assessment centre from 2005 to 2008, were examined. Candidates were examined twice: first during the assessment centre and second after their first examinations at university. At each point in time, personal and social skills were assessed, along with performance in achievement tests. Regression analysis was used to investigate the impact of background factors on success in the assessment centre. The multitrait-multimethod (MTMM) approach and nomological network were used to shed light on the relationship between self- and other agreement and on some aspects of construct validity.

Findings
Initial findings regarding assessment centre fairness show that none of the above mentioned groups is at a disadvantage. Results indicate that high performance candidates at the assessment centre are more likely to be successful during their first years of studying. However, construct validity of the assessment centre is rather low. In general, self-ratings are moderately associated with ratings given by assessors. Differences in self-other agreements between competencies have been found.

Theoretical and educational significance
This study contributes to the assessment of (future) teacher’s professional skills and the self-other agreement of competencies. Providing substantial feedback on candidates’ strengths and weaknesses as well as comparing self- and other-ratings at a very early stage of their career makes an essential contribution to their further professional development. Consequences for teacher education and professional training have to be discussed.

References

Atwater, L., Waldman, D., Ostroff, C, Robie, C. & Johnson, K. (2005). Self-other agreement: comparing its relationship with performance in the U.S. and Europe. International Journal of selection and assessment, 13 (1), 25-40.

Darling-Hammond, L. & Bransford, J. (2005). Preparing teachers for a changing world. What teachers should learn and be able to do. Jossey Bass: San Francisco.

Frey, A. (2004). Die Kompetenzstruktur von Studierenden des Lehrerberufs. Zeitschrift für Pädagogik, 50 (6), 903-925.

Herzog, W. (2002). Zeitgemässe Erziehung. Weilerswist: Velsbrück.

Kleinmann, M. (1997). Assessment Center. Stand der Forschung – Konsequenzen für die Praxis. Göttingen: Verlag für angewandte Psychologie.

Märki, A. (2006). Internetgestützte Laufbahnberatung bei angehenden Schweizer Lehrpersonen. Beiträge zur Lehrerbildung, 25, 132-134.

Mayr, J. (2001). Career Counselling for Teachers. Context - European Education Magazine. No. 25, p. 11.

Schaarschmidt, U. (2004). Halbtagsjobber? Psychische Gesundheit im Lehrberuf. Weinheim: Beltz.

Terhart, E. (2003). Reform der Lehrerbildung: Chancen und Risiken. In I. Gogolin & R. Tippelt (Hrsg.), Innovation durch Bildung (S. 163-180). Opladen: Leske und Budrich.

Thornton, G.C. & Rupp, D. (2006). Assessment Centers in human resource management. Strategies for prediction, diagnosis and development. London: Lawrence Earlbaum.

Keywords Assessment of Competence
Higher education
Initial Teacher Education (Pre service)
Appendices
Authors
Name Surname Institution Country e-mail EARLI Number Presenting
Christine Bieri Padagogische Hochschule Zurich Switzerland Christine.bieri@phzh.ch   *  
Patricia Schuler Padagogische Hochschule Zurich Switzerland patricia.schuler@phzh.ch    
Paper type Theoretical
Title Assessing Vocational Competence Aimed at the Integration of Knowledge, Skills and Attitudes
Abstract

Adequate assessments are needed to assess competence development, the integration processes of knowledge, skills and attitudes. However, it is not clear what these integration processes actually pertain and how they develop. Therefore, this paper conceptualises integration in three different processes: (1) integration between different types of knowledge (knowing that and how), (2) integration between knowledge acquired in different learning situations, and (3) integration between knowledge, skills and attitudes. Based on a literature study, the three integration processes are described, and different assessment methods are proposed to assess integration processes: a video task with reflective questions, concept maps, stimulated recall interviews and think aloud tasks, and performance assessments. Together, these assessment methods are thought to enable the elicitation of integration processes in vocational education students.

Summary

Aims
Vocational institutions in the Netherlands are required to base their programmes on competence profiles from 2010 on. Therefore, adequate assessments are needed. Competence is often defined as the integration of knowledge, skills and attitudes (e.g., Lizzio & Wilson, 2004), but no research exists on how these integration processes develop, rendering assessment of integration processes very difficult. Therefore, this study focuses on the conceptualisation of integration in vocational education. The paper presented in this symposium proposes three methods for assessing integration, based on a literature study.

Conceptualising integration
Integration is conceptualised in three different processes:
(1) Integration between different types of knowledge
Cognitive psychological literature distinguishes two types of knowledge: declarative (that and why) and procedural knowledge (how and when). Anderson and Schunn (2000) represent declarative knowledge by a set of primitive knowledge units and procedural knowledge by production rules that fire as a response to contextual cues and the strength of the production rule. Declarative and procedural knowledge are linked because declarative knowledge is retrieved from memory when a production rule fires to carry out a task. However, these cognitive theories only study relatively easy tasks in laboratory settings that are very different from the much more complex tasks in practical job situations. Therefore, we propose a more authentic video assignment, in which students are asked reflective questions to elicit integration.
(2) Integration between different contexts
In vocational education, students acquire knowledge in many learning situations, such as classrooms and workplace contexts. Students experience difficulties integrating this knowledge (Eraut, 1994). Important theories focusing on the integration between knowledge acquired in different contexts are activity theory and its ideas of boundary crossing (Engeström et al., 1995) and conceptual change theories (Vosniadou, 2007). To integrate knowledge, it is assumed that students need ‘boundary objects’, materials or representations that are shared by the two contexts. Also, students develop ‘naïve intuitive theories’ organised in theory-like structures and based on daily experiences. Through learning in different contexts, these structures change. Based on these theories, we propose a combination of assessment methods consisting of concept maps, stimulated recall interviews and think aloud methods (see also Meijer et al., 2002).
(3) Integration between knowledge, skills and attitudes
Professional competence not only involves different types of knowledge, but also skills and attitudes. Skills are interwoven with knowledge, and are defined as a considered approach for the solution of problems (Benjafield, 1992). However, assessment literature suggests that knowledge cannot always be inferred from the performance of skills and the two are thus not automatically integrated (Eraut, 1994). Attitudes involve knowing one’s personal characteristics and mobilising them in such a way that positive or useful aspects are applied, and negative on unuseful aspects are neutralised (Van der Sanden, 2004). To assess knowledge, skills and attitudes in an integrated way, many authors argue to use professional tasks in non-routine situations (e.g., Baartman et al., 2007). This non-routine character should guarantee that students need to retrieve declarative knowledge when carrying out the tasks, thus integrating knowledge and skills.  

Proposed assessment methods
Separate assessments are suggested for the three integration processes distinguished above. However, this is done mainly for conceptual and empirical clarification. In practice, the three integration process are assumed to happen simultaneously.

(1)   Based on Beardsley et al., (2007), students watch a video of a practitioner carrying out a task typical for their profession. They identify fragments that show examples of a theory in practice (= from knowing that to how) and explain why the professional carries out the task in that way ( = from knowing how to that).

(2)   Based on conceptual change and boundary crossing theories (Vosniadou, 2007; Engeström et al., 1995), a longitudinal assessment is proposed, using concept maps, stimulated recall interviews and think aloud. Data are collected once a month, depending on the amount of assessment needed to adequately follow students’ integration processes.

(3)   Based on literature on competence assessment (Baartman et al., 2007), an integrated performance assessment is proposed, offering a challenging professional task. Additionally, separate knowledge tests, skills demonstrations and attitude questionnaires are administered to study correlations between the different assessments.  

Theoretical and educational significance
This paper contributes to the development of adequate competence assessments, conceptualised as the integration between different types of knowledge, acquired in different learning situations, and connected with skills and attitudes. Such assessments are needed not only to award certificates and provide students with feedback, but also as effect measures of educational innovations in vocational education. Based on a literature study, three different assessment procedures are proposed, which can be further validated in future research.

References

Anderson, J. R., & Schunn, C.D. (2000). Implications of ACT-R learning theory: no magic bullets. In: R. Glaser (Ed.), Advances in Instructional Psychology. London: Lawrence Erlbaum.

Baartman, L.K.J., Bastiaens, T.J., Kirschner, P.A., & Van der Vleuten, C.P.M. (2007). Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks. Educational Research Review, 2, 114-129.

Beardsley, L., Cogan-Drew, D., & Olivero, F. (2007). VideoPaper: Bridging research and practice for pre-service and experienced teachers. Paper presented at the EARLI, Budapest, Hungary.

Benjafield, J.G. (1992). Cognition. New Jersey: Prentice-Hall.

Engeström, Y., Engestrom, R., & Karkkainen, M. (1995). Polycontextuality and boundary crossing in expert cognition: learning and problem solving in complex work activities. Learning and Instruction, 5, 319-336.

Eraut, M. (1994). Developing professional knowledge and competence. London: Routledge Falmer.

Lizzio, A., & Wilson, K. (2004). Action learning in higher education: an investigation of its potential to develop professional capability. Studies in Higher Education, 29, 469-488.

Meijer, P., Verloop, N., & Beijaard, D. (2002). Multi-method triagulation in a qualitative study on teachers’ practical knowledge: An attempt to increase internal validity. Quality & Quantity, 36, 145-167.

Van der Sanden, J.M.M. (2004). Ergens goed in worden. Naar leerzame loopbanen in het beroepsonderwijs. Oratie 12 maart 2004, Fontys Pedagogisch Technische Hogeschool, Eindhoven, The Netherlands.

Vosniadou, S. (2007). The cognitive-situative divide and the problem of conceptual change. Educational Psychologist, 42, 55-66.

Keywords Assessment of Competence
Learning theory
Vocational education
Appendices
Authors
Name Surname Institution Country e-mail EARLI Number Presenting
Liesbeth Baartman Utrecht University Netherlands l.k.j.baartman@uu.nl   *  
Elly de Bruijn Utrecht University Netherlands E.deBruijn@uu.n    
Visit NQcontent
© European Association for Research on Learning and Instruction, 2010 All rights reserved.