|
by Len Holmes, The Business School, University of North London (at time of presentation)
Keynote presentation at 'What is Competence?', the inaugural conference of The Competence Network, Centre for Labour Market Studies, University of Leicester, 1st December 1994.
Promises and problems
What is competence? This seemingly simple notion has risen to prominence, apparently irresistibly, over the past decade or so not only in the development work on vocational qualifications but more broadly within what is termed 'human resource management'. The idea does seem disarmingly simple whilst promising so much. A competent person performs their job well, to the standards required; competence is what enables them to do this. Base your recruitment and selection, and promotion systems on this simple notion and you can ensure that you recruit and promote people who can do the jobs required. Base your training and development strategies and programmes on competence and your staff will develop the abilities they need to perform the activities which the corporate strategy requires. Change the assessment systems for vocational qualifications so that these are awarded only when nationally agreed standards of competence are achieved and employers will be 'guaranteed' that an employee with such a qualification can do the job. What has everyone being doing before now that such a blindingly obvious and essentially simple approach was not the normal practice?
Unfortunately, examination of this 'simple' idea shows that it is not as simple as it first appears. There are different methods for the analysis of competence, with no way of deciding which is best. The method chosen for developing a national system of competence-based qualifications is a relatively untried and under-developed one, which has been applied differently in various occupational areas, and appears constantly to need modification and repair. The simple idea of competence turns out to require lengthy descriptions in what appears to many to be an arcane language. Employers don't rush forward to get involved. Some employers who get involved in the early stages later pull out. Embedding the approach turns out to give rise to major structural problems. What is it about the concept of 'competence' that it evades yielding the fruits it appears to promise?
What I aim to do in this session is the share with you parts of my examination of the notion of 'competence', particularly looking at inherent flaws in the approaches usually adopted for its analysis and application. I am adopting a principle which I take as generally accepted in any form of purposive human inquiry and activity: if you fail to get a satisfactory answer to a question and solution to a problem despite repeated attempts and considerable effort, then maybe you should reframe the question and problem. So rather than ask 'what is competence?', perhaps we should ask 'what are we doing when we say someone is competent?' In exploring this question I come to the conclusion that at the very heart of the answer are major issues to do with confidence.
Reframing the question
The most common definitions of competence usually tend to include some notion of it being the ability to perform to some desired or required standard. However, we can discern two separate types of approach which have been developed that claim to be 'competence-based'. One type of approach focuses on the notion of 'ability', treating this as a quality, characteristic or attribute of a person. So, for example, by examining in some way the qualities, characteristics and/or attributes of an organisation's currently high-performing managers we might, it is assumed, identify what qualities etc the organisation should seek in recruiting new managers, or should attempt to develop in other managers to raise their performance. A whole battery of aptitude and psychometric tests have been developed, forming the basis of the technology of testing.
The second type of 'competence-based' approach focuses not on these purported qualities etc, but upon the standard of performance required. By specifying in detail the performance required, some form of standard is obtained against which the actual performance may be compared. In essence, this is the type of approach which has been adopted for the development of the competence-based vocational qualifications system which the De Ville working party called for in 1986.
Both these approaches suffer, in my view, by an over-reliance on the technical specification of competence to the detriment of considering the social processes involved in the decisions and actions in which attributions of competence may play a part. They attempt to make use of the term 'competent' which is rather loosely used, along with terms like 'able', 'proficient', 'capable', and so on, within normal activities of vocational education and training, recruitment and selection, appraisal, etc, in the 'real world'. But in taking the term, the approaches specifically claiming to be 'competence-based' attempt to engage in technical specification which rob the term of a key aspect of its normal meaning-in-use. I want to explore these normal activities and the normal meaning-in-use, in order to assess the more technical specifications, particularly as in the NVQ approach.
'Competence' is future-oriented
The NVQ approach seeks to reduce the assessment of competence to the comparison of past performance (as claimed through a 'portfolio of evidence') with the performance standard set. This must be done for a number of 'elements of competence', using the performance criteria set, and covering a range of situations and circumstances. Of course, such performance evidence might be supplemented by some form of assessment of 'underpinning knowledge and understanding'. If someone produces satisfactory evidence that they have performed as specified, they are competent, according to this approach.
However, such a reductionist approach fails to take into account a key aspect of the judgement of competence, that is, that it is future-oriented. The whole point of making a judgement of competence is that in making it we are expressing a view, a belief, a judgement, about the future performance of the individual. By saying 'M is a competent manager' we express the belief, the confidence, that they will perform well as a manager. This confidence or belief is, purportedly, the basis on which we then make the decision about employing them, promoting them, awarding them a qualification or whatever.
This emphasis on the future orientation of any judgement of competence is critical. If we forget or neglect this point, we may fall into the trap of believing that competence can be reduced to some administratively straightforward system of describing past performance. No matter what detail of specification we devise, the collection of evidence of past performance cannot in itself form the complete assessment of competence. To be useful, competence inferred must be greater than performance observed. So, when we say someone is competent, we are expressing our confidence in the inference we have made from some information we have about the person, that they will perform as required in the relevant circumstances or situation.
The social context
By drawing attention to this issue of confidence in the inference of future performance, I am, of course, also pointing out that this is an area in which there is risk. The future is, by definition, uncertain; we cannot know how someone will perform in the future. Nor can we know how someone we did not select, did not promote, did not award a qualification to, might have performed.
Moreover, we do not make the judgement in a social vacuum. We do so within a social context in which there are various actors, various 'stakeholders', with different interests and accountabilities, different things they are trying to achieve and various ways in which others will hold them accountable. If we are selecting an employee we shall probably have to justify our decision to others in the organisation. In a panel interview we shall engage in discussion about the candidates with other members of the panel. They too must come to share our confidence. Equal opportunities policy, and anti-discrimination legislation, may impose a requirement on us to articulate clearly how our choice is based on assessment compatible with the policy and lawful. Those charged with ensuring compliance on this must also be confident that this is being achieved. Although we are doing the selection, it maybe others who will enjoy or suffer the consequences of our decision, as the immediate line manager of the person appointed, or as leader and members of a project team of which they become a member. Again, they will need to be confident in our decision. Or, to put it another way, we shall need to gain and maintain such confidence if we are to retain our involvement in these processes of selection.
This becomes more complicated when we move outside an organisation, looking at the relationship between prospective employers, prospective employees, bodies which award qualifications, those that assess candidates for qualifications, and providers of education and training. An NVQ is, it is claimed, a statement of competence; it is a claim that a job applicant with such a qualification can do the job. It is claimed. There's the rub, for a prospective employer cannot verify the claim except by taking on the applicant and seeing whether, over the short, medium and longer term, they really can. For the present, they must take the claim on trust, investing confidence in the claims made. Similarly, prospective employees will have earlier made decisions about what qualifications to seek. In doing so, they will need to have confidence that the qualification they choose to obtain will be acceptable to prospective employers, ie confidence that employers have confidence in the qualification.
The awarding body must gain and maintain such confidence if it is to continue to attract demand for its qualifications and the services it provides. Relevance becomes an obvious issue affecting the level of confidence; various procedures and systems will usually be established in order that the awarding body should be seen to be maintaining the relevance of the qualifications to the requirements of employers. But also of concern will be issues of quality assurance. How can the awarding body be confident that those who assess candidates and recommend them for a qualification are doing this according to the methods prescribed?
Competence and communication
This all points to the fact that confidence is socially produced; it is not merely some internal psychological state, a calculation or a feeling. 'Confidence' is the basis on which the various parties implicated in the decisions and actions taken within a competence system will seek to account to others for those decisions and actions. Whether stated explicitly or implicitly conveyed, to base a decision on the purported competence of an individual is to step into a network of actors and agencies such that the relationship between them is critically dependent on the maintenance of confidence. This requires the parties concerned to engage in some form of communicative practice, whereby some understanding is shared about what these various parties are doing in arriving at the decisions they take.
Such communication is based on what we might call conventions of assessment. Some common understanding is achieved by which a certain set of arrangements become socially accepted as the basis for linking different contexts. Contexts differ in particular in terms of time. So performance in the past is linked to future situations in which desired performance is anticipated. There are other ways in which contexts may differ, for example in terms of the types of performance required. So, it may become conventional to link the performance involved in translating ancient Latin texts with the performance required of government officials in British embassies. Or it may become an accepted convention to link the a self-constructed 'portfolio of evidence' on certain specified aspects of the performance of a manager in the marketing department of a fast moving consumer goods organisation with the performance required of a manager in a large hospital.
This linking of contexts will normally involve some model, some way of accounting for the claimed link. There are many such models in the vocational education and training field, based on an attempt to analyse what it is that enables someone to perform well in the situations being analysed. Much work was undertaken in the UK in the 1960s and 1970s, especially through the work of the industrial training boards. In north America there is a longer history and greater volume of work; in the late 1970s the Manpower Services Commission sent staff on a study tour to look at and report on the approaches being used, including DACUM and the Functional Job Analysis developed by Fine. More recently in this country we have had the Occupational Training Families approach, to be followed by Functional Analysis. The focus for most analyses is what I have described as a 'job' or 'technical' approach. However, there have been approaches which I would describe as focusing on social role, for example the rather short-lived MSC sponsored inter-ITB collaborative project on supervisory development, in the late 1970s, and the work of Pettigrew and colleagues on the development of training specialists. There are also what I would term 'person' or 'biographical' approaches, eg work done by Cranfield on managers in transition, and the work of Willis and Daisley in developing the award-winning Springboard programme.
Conventions of assessment
So there is a plethora of models, all with their success claims, many with a relatively long ancestry having, we might say, 'stood the test of time'. All have their underlying rationale. All claim to show how performance in one context links with performance in other contexts. So how do we make sense of such diversity?
We do so, I believe, by regarding the conventions of assessment, with their implicit or explicit models of performance, primarily as conventions of warrant. That is, they are used to seek acceptance that the actions and decisions taken are done so on accepted, rational grounds. The models of performance do have a place in experimental studies, ie in the evaluation of different approaches to education and training and to the assessment of their outcomes. They form the basis of articles in publications such as the Journal of Applied Behavioural Studies, or the reports of funded research programmes. But who reads these? Certainly not the vast majority of people involved in the assessment process, especially those who use the outcomes of assessment in recruitment and selection. They do not sit down to study the claims made in research reports. Even if they had the time to do so, they would need to have an appropriate background in the behavioural science field in order to have sufficient understanding to subject the claims to critical scrutiny.
For the most part, conventions of assessment operate not as rationally conceived and executed replications or applications of such 'scientific' research studies. Rather, they are socially accepted, and therefore taken as legitimate, ways in which the various parties engaged justify the decisions and actions they take. This relates to the distinction made in the social and human sciences between action and behaviour, rationality and causality. An employer recruits someone because they have a particular qualification. But the fact that the person possesses the qualification is not the cause of the decision; it is the reason for it, the basis on which the employer can claim to have acted rationally, and expect that claim to be respected. The awarding body granting the qualification does so because those who did the assessment recommended it. But the recommendation did not cause the decision, it was the reason for it. The assessors recommended the award because certain results were achieved when the candidate performed certain actions and activities. But the results achieved did not cause the recommendation; they were the reasons for it. And so on. Ask any party to the process why they acted as they did, and they will provide reasons; question those reasons and they will provide more reasons. The whole network can only operate if there is an assumption that, if pressed, the various members can provide acceptable reasons; that is, there has to be an assumption of rational action, confidence in the reasonable-ness of the others on whom they rely.
The conventions of assessment do, of course, emerge within practical circumstances which affect the degree to which the various parties are willing to rely on others. Confidence may be granted in varying degrees, circumscribed in various ways, subject to all sorts of limits and constraints. These include notions of the costs involved and the benefits anticipated. The benefits lie in the extent to which the assessment process enables discrimination to be made between individuals. After all, there is little benefit to an employer seeking to recruit three people to be presented with 100 all of whom are judged to be suitable. Although there has been much emphasis in the rhetoric of competence on criterion-referencing, we must never forget that recruitment and selection is essentially a norm-referenced activity; we want to choose 'the best candidate'.
Lean assessment
The greatest benefit arises from assessment which allows the greatest inference about future performance. But the greatest cost-benefit arises where maximum inference can be made for minimum cost. We might call this 'lean assessment'. A lean system of assessment is one where there is a high level of confidence within the network of parties involved that such maximum inference can indeed be made on the basis of minimal cost. Costs are incurred not only in the actual assessment of individuals, but also in the systems of scrutiny involved, eg in checking that those involved do actually do what they claim and are supposed to do. These would include systems of double marking, external examiners, moderators, and of course, internal and external verifiers.
I therefore wish now to state a principle that I believe to be central to any consideration of assessment systems, based on this recognition that these are conventions. All conventions of assessment will tend towards achieving the maximum state of lean-ness. This maximum lean state is the equilibrium point of perceived cost-benefit at which confidence is maximised. If an assessment system does not achieve this formally and explicitly, it will do so informally and implicitly; that is, if the formal system does not reach the point of maximum lean-ness, then informal understandings and arrangements will be established. Or to put it more crudely, people will 'work the system'.
Assessing NVQ assessment
Turning now to consideration of the current specifications for assessment for NVQs, we can see that they do not fulfil the requirements of a lean assessment system. The specifications are extended and elaborate, and even now the specification spiral seems be extending ever onward: evidence requirements, specification of what is called 'underpinning' knowledge and understanding, and so on. We even now have the prospect of an inspectorate to 'monitor standards'. Who will judge the judges? It is a very costly system bringing insufficient benefit in terms of the difference it makes in discriminating between individuals, for judgements about future performance to be made with confidence. The case for the NVQ system that has been established has been put forward eloquently; the technically-rational basis for it has been well presented, and continues to be well presented. Many are persuaded. But it clearly has not achieved the widespread adoption which would seem inevitable from the case made for it. That is understandable when we recognise that assessment involves a social system, one that can only be build on confidence.
The attribution of 'competence' (or 'ability', 'capability', 'proficiency' etc) under normal circumstances thus always involves issues of confidence. The 'trick' is to gain, maintain and sustain such confidence, shared by all parties to the process. As in all normal social processes, this requires conventions through which claims to be acting reasonably may be made and accepted. Conventions of assessment are most useful when they are perceived as 'lean', and will always tend towards this. Anyone engaged in developing a system of assessment, whether for qualifications or not, should always concentrate on maintaining the confidence of all parties involved, by seeking to keep it lean! The De Ville working party on the Review of Vocational Qualifications recommended the development of 'a system which is comprehensible, relevant, credible, accessible and cost-effective'. It seems we still have a long way to go!