ABSTRACT
This contribution aims to address the issue of measuring and verifying the levels of clinical competence possessed by professionals, as a primary resource enabling the provision of quality health services by investigating systems and tools for mapping and evaluating clinical competence. Specifically, this contribution aims to show the reality of the clinical competence mapping project applied in the Department of General Surgery of the Bologna Local Health Authority. In particular, the objectives of the project, the research methodology, a detailed description of the pathway followed by the working group, the future developments of the project and the concluding reflections are highlighted and explained.
Key words: mapping, clinical competence, evaluation, Bologna Local Health Authority
INTRODUCTION
The issue of knowledge management has always been at the centre of considerations and practices related to all types of hospitals. It is particularly relevant for hospitals where knowledge is overseen by the system of professions and disciplines, while knowledge is an asset that professionals tend to defend as they consider it their own.[1] At the same time, being particularly critical components in healthcare production processes, knowledge and expertise need to be coordinated and managed by the hospital to the extent that the hospital itself intends to govern the end results.
What was new compared with traditional models of the National Health Service was the focus on the issue of outcomes, results and the conditions necessary to sustain the challenge of their improvement.[2] In this context, the quality of clinical competence, i.e., the definition and management of what professionals know and what they should know, becomes important. The real challenge for hospitals is to create reasoned and prospective professional development pathways for their professionals that are consistent with hospital strategies and the health needs of the population. To achieve this, hospitals should define models and reference tools for analysing, mapping and evaluating the skills and knowledge possessed by professional.[3]
Getting to the heart of the matter, it is good to define the concept of clinical competence,[4] understood as a representation of the knowledge, skills and technical abilities, the professional, relational and operational qualities possessed by individual professionals operating and working in the healthcare setting necessary to ensure the effectiveness of the services offered in terms of performance. The mapping and measurement of clinical competence makes it possible to identify the strong points, which areas need improvement and what the most critical gaps to be bridged are, with the aim of adapting these competencies to the demands of the various stakeholders with whom hospitals interact on a daily basis (users, citizens, private-accredited, regional hospitals and universities, to name the main ones).[3,5]
Hospitals are called upon to decide which models and reference tools to adopt to analyse and assess the capacities of individual professionals so as to build professional growth pathways in line with hospital strategies and the health needs of the population. A competence-based system is an appropriate tool for better and correct management of human resources in the hospital.
This article intends to address the issue of measuring and verifying the levels of clinical competence possessed by professionals, as the main resource enabling the provision of quality healthcare services, by focusing on the clinical competence mapping project applied in the Department of General Surgery of the Bologna local health authority.
METHODS
The analytical method used for this study involved an initial review of the international academic and institutional literature on clinical governance related to the appreciation and measurement of the competencies possessed by professionals. In particular, three cases of public and private hospitals that are leaders in the introduction of clinical competence assessment logics and tools (the Provincial Agency for Health Services of Trento, the Humanitas Clinical Institute of Milan and the European Institute of Oncology of Milan) were taken as a reference point for the methodology applied in Bologna local health authority.[3,6]
We decided to implement the competence mapping system, on an experimental basis, in the Department of General Surgery, with the aim of testing a specific analysis framework with respect to hospital contingencies and gradually extending the application of this system to other Departments.
The hospital's Organisational Development structure took the lead in this process by defining a multiprofessional and multidisciplinary working group and accompanying the departmental and facility management involved towards the definition of service catalogues, i.e., the activities and clinical procedures that are carried out within the operational units. The catalogues have been organised into macro-areas, which in turn are divided into clinical activities for which a description of the activities is given. In the Department of General Surgery, a total of two service catalogues were produced: a service catalogue for the surgical operational units and a service catalogue for Gastroenterology and Interventional Endoscopy (Multidisciplinary Unit), as shown in Table 1. For the creation of both service catalogues, data from procedures performed in 2019 (outpatient specialist and inpatient waiting list procedures) were used as a starting point. The working group focused on the evaluation of surgical area skills.
| Catalogue of Services Department of General Surgery | Catalogue of Services Department of Surgery: Gastroenterology |
|
1. Appropriateness, fairness and sustainability
2. Technical competency in the operating theatre: election (a) Cutaneous and subcutaneous interventions that can be performed on an outpatient basis (class I) (b) Abdominal wall conditions (c) Non-cancerous abdominal disease (d) Proctology and pelvic floor (e) Oncological abdominal disease 3. Technical competency in the operating theatre: trauma and emergency (a) Emergency 1 (a) Emergency 2 (b) Trauma 4. Clinical and technical competency: hospitalisation 5. Multidisciplinary and relational competencies and multimodal approach to surgery 6. Self-evaluation and monitoring |
1. Appropriateness, fairness and sustainability
2. Endoscopic techniques competency: choice (non-US-guided manoeuvres) (a) Upper GI tract procedures (b) Lower GI tract procedures 3. Endoscopic techniques competency: elective (US-guided manoeuvres) (a) Upper GI tract procedures (b) Lower GI tract procedures (c) Biliopancreatic tract procedures 4. Endoscopic techniques competency: emergency (non-US-guided manoeuvres) 5. Endoscopic techniques competency: emergency (US-guided manoeuvres) 6.Clinical and technical competency: inpatients/outpatients 7. Multidisciplinary and relational competencies and multimodal approach to surgery 8. Self-evaluation and monitoring |
The working group then chose the evaluation scale to be used for assessing the individual professionals, taking as its starting point, and with some modifications requested by the trade union representatives, the specific requirements for the accreditation of General Surgery Facilities deriving from the specific resolution of the Emilia-Romagna Regional Council.[7,8] The professionals of the Department of General Surgery were assessed in relation to their ability to perform the specific activities, listed in the catalogue of services, on the following five-level scale:
-
Level N, “service not assigned”;
-
Level D, “observation”, the professional needs training to perform the specific task;
-
Level C, “shadowing”, the professional needs supervision when performing the specific task;
-
Level B, “autonomy”, the professional is competent to perform the specific task without supervision;
-
Level A, “supervision”, the professional is competent to train others to perform the specific task.
The assignment of the degree of autonomy and competence of the professionals was exclusively the responsibility of the Director of the operational unit who filled out the 'Individual Assignment Form' for each clinician in their team through an individual interview.
RESULTS
Once the data from the 'Individual Assignment Form' had been collected, the surgical activity data from the hospital discharge forms for the time period from 2018 to 2022 were collected. Since this was an experimental phase, it was decided to proceed with a cross-analysis of the data by means of a data extraction from current information flows in order to verify the quality of the mapping tool used and to reduce any subjectivity that could influence its expression. In fact, the analysis of surgical activity data conducted by the working group, serving as an objective reference for this analysis, helped to minimise the subjectivity in evaluations made by the Directors of the Operational Units within Multidisciplinary Departments for the respective medical teams involved in the project. The findings of the ‘Individual Assignment Form’ distributed for the various Operational Units of the General Surgery Department are listed below (Table 2). In the third column of Table 2, the catalogue performances that showed the result “N”, i.e. “performance not assigned”, have been excluded from the total count of skills, in order to better appreciate the skills actually possessed and exercised by the professionals.
| Operational units | Percentage | Percentage of excluding “N” values |
| Total “N” | 19% | |
| Total “D” | 15% | 19% |
| Total “C” | 19% | 23% |
| Total “B” | 42% | 52% |
| Total “A” | 5% | 6% |
| Total operational unit NO. | ||
| 1 | 100% | 100% |
| Total “N” | 5% | |
| Total “D” | 21% | 22% |
| Total “C” | 23% | 24% |
| Total “B” | 22% | 23% |
| Total “A” | 29% | 31% |
| Total operational unit NO. | ||
| 2 | 100% | 100% |
| Total “N” | 23% | |
| Total “D” | 0% | 0% |
| Total “C” | 9% | 11% |
| Total “B” | 31% | 41% |
| Total “A” | 37% | 48% |
| Total operational unit NO. | ||
| 3 | 100% | 100% |
| Total “N” | 2% | |
| Total “D” | 16% | 16% |
| Total “C” | 24% | 25% |
| Total “B” | 57% | 58% |
| Total “A” | 1% | 1% |
| Total operational unit NO. | ||
| 4 | 100% | 100% |
| Total “N” | 11% | |
| Total “D” | 1% | 1% |
| Total “C” | 11% | 12% |
| Total “B” | 29% | 33% |
| Total “A” | 48% | 54% |
| Total operational unit NO. | ||
| 5 | 100% | 100% |
In addition, as shown in Table 3, the assessment of the clinical competence of the professionals was enriched by means of a demographic analysis of the personnel involved in the pathway, in particular through the following indicators:
| Indicators | Number | Percentage |
| No. of doctors with seniority > 5 years | 21 | 78% |
| No. of doctors under 50 years of age | 9 | 33% |
| No. of doctors close to retirement (or already retired) | 3 | 11% |
| No. of female doctors under 45 years of age | 5 | 19% |
| Total No. of professionals | 27 | 100% |
• Number and % of professionals with more than 5 years' seniority to consolidate their competencies;
• Number and % of doctors under 50, in whom the hospital intends to invest by increasing competencies, including excellence through training;
• Number and % of professionals approaching retirement, who will have to transfer their competency and know-how to younger doctors through shadowing;
• Number and % of female professionals under 45, in order to ensure the replacement of their competencies for the periods necessary to manage possible pregnancies.
At the end of the project, single feedback was organised between health management, department directors and facility directors. These feedback meetings were also an opportunity to discuss the methodology adopted and to receive feedback on the tool used, precisely in the logic of experimentation.
DISCUSSION
The Bologna Local Health Authority believes that the management of a complex hospital network requires the cataloguing of clinical services, specific to the department, and the evaluation of the clinical competencies possessed by professionals in order to guarantee the optimal performance of activities also at all hospital sites. The hospital was responsible for managing the clinical competence mapping project in order to highlight and identify both areas of excellence and areas of critical or undiscovered competencies. With regard to the areas of excellence, the aim is to progressively upgrade professionals who possess the “supervision” level (Level A) as a reference for the enhancement of critical or uncovered areas of competence within the team/department.
The assessment of clinical competence made it possible, in the first instance, to identify for each of the individual activities and services how many doctors are autonomous (Level A-B), how many need to be shadowed (Level C), how many can observe (Level D) and how many do not have the competence to perform the service or it is not within their remit (Level N). If the surgical service were to be performed independently by a small percentage of professionals in the operational unit, one could envisage the possibility of distributing the case load between several professionals so that the appropriate training could be planned to strengthen the technical competency and ensure a greater number of autonomous doctors (shift towards Levels A and B).[3]
The cross-analysis of the results of the clinical competence mapping with the extraction of data from current information flows shows a good degree of compliance, for operational unit nos. 1, 2, 3 and 4 in the table above, between the assessment made by the director of the unit and the analysis of the ministerial data flows. In operational units 1 and 4, there is a prevalence of autonomy in competences (Level B), a lower percentage of professionals considered to be supervisors (Level A) and a balanced distribution between shadowing and observation (Levels C-D). In operational unit no. 2, an equal distribution of the competence assessment can be seen, with a prevalence of supervisor level (Level A), which is 30% lower than the total assessment of the operational unit. In operational unit no. 3, there is a prevalence of supervisor (Level A) and autonomy (Level B) owing to the presence of a senior professional. In operational unit no. 5, there is a high prevalence of supervisors (Level A) and autonomy (Level B), and in this case there is a partial correspondence between the results of the clinical competence mapping and the activity data from current information flows.
The demographic analysis of the Department of General Surgery shows that a significant proportion of medical professionals (78%) have already accrued at least five years of service with the hospital; therefore, for the aforementioned pathway, the intention is to continue consolidating their competencies. One third of the medical population of the department is aged ≤ 50 years. The hospital intends to invest in these professionals with a structured pathway of competency growth, including excellence through the structuring of targeted training courses. Two out of twenty-seven professionals are due to retire in 2025 and 2027, respectively, and therefore the departmental management has been asked to set up a pathway for transferring competency and know-how to younger doctors through shadowing activities.
Being an experimental project, this mapping and assessment system of clinical competences would allow the Bologna Local Health Authority to make reasoned and prospective three-year staff requirement plans with the definition of strategies that would aim to extend and implement the mapping of clinical competences in order to support organisational choices (specialisation of the supply network) and staff choices (acquisition and/or development). In addition, such a system would also allow the mapping of competencies also in the context of hospital care.
Besides the strategic choices that will be defined in the three-year staffing requirement plans, in the Hospital Management's vision, the development of the system could involve several actions, which are outlined below:
(1) The dissemination and application of the clinical competence assessment model and tool in the hospital's other facilities;
(2) A link with the measurement of clinical outcomes, activity volumes, effectiveness and efficiency of healthcare production processes, starting from the requirements of Ministerial Decree 70/2015;
(3) The definition of professional development paths both for professionals at the end of their careers and for physicians under 45/50; for the latter, the objective will be to create a motivational drive to remain in the hospital, also and above all through the preparation of and participation in training pathways of excellence;
(4) The enhancement of competence mapping tools also in the logic of creating spoke-hub networks among the hospitals distributed throughout the province.
The definition of the catalogue of activities and competence mapping of all the hospital's professionals would ensure greater clarity with regard to the skills present in each hospital in order to help set up professional and organisational growth pathways that guarantee safety, quality and effectiveness in the provision of services to the user.
In conclusion, many of the competitive advantages and critical success factors in hospitals are what medical professionals can do and their level of autonomy and specialisation. If the hospital wants to govern its development pathways while maintaining consistency between objectives and the conditions for pursuing them, it is essential that it knows how to assess and manage these competencies.
The mapping and assessment of clinical competencies defines a map of specialised knowledge, ensuring that patients are treated by highly specialised professionals and could serve as an aid to hospital strategic choices related to expected developments, thus representing an important lever of Hospital governance. Moreover, it is useful for obtaining possible international accreditations, a symbol of high standards, quality and safety of care in hospital and regional facilities (for example, Joint Commission International or Accreditation Canada, to name but a couple).[9]
Mapping plays a key role in defining the roles and responsibilities of professionals and is useful for ensuring effective selection, evaluation and development over time of the staff employed in the hospital. Clinical competence mapping helps in the creation of professional growth pathways consistent with hospital strategies and evolving health needs, enhancing individual and distinctive competencies through the involvement of hospital training.[10,11] The Hospital Management believes that the mapping of clinical competences can play a decisive role in driving organisational change and improvement, in synergy, among other things, with the important international accreditation process in which the Bologna Local Health Authority is currently involved. In addition to external stimuli, such as the aforementioned international accreditation pathways to quality and excellence, possible reorganisations and structural changes within the Hospital also lead the Hospital Management to reflect on the proper management and governance of competencies. Therefore, even for the strategic management decisions that Management is required to take, it is obvious and crucial to be aware of and know how to enhance the competence possessed by the professionals employed in the hospital in order to improve the management of professional development paths.[3,12]
As previously mentioned, on an experimental basis, the competence mapping system was implemented in the Department of General Surgery, with the aim of extending its application to the other departments of the Bologna Local Health Authority.
DECLARATIONS
Author contributions
PB have decided to start a mapping and evaluation process of clinical skills and individual performance by choosing the Department of General Surgery. AB have coordinated the whole project from the definition of the working group to the feedback phase. LS was responsible for the operational part of the project, working with the departmental management and the structures involved to define the performance catalogues. He was also responsible for collecting and analyzing the data and drafting this article. CM have contributed to the analysis of data and the drafting of this article.
Use of large language models, AI and machine learning tools
None declared.
Informed consent
Not applicable.
Ethical approval
Not applicable.
Conflicts of interest
There is no conflict of interest among the authors.
Data sharing statement
No additional data is available.
REFERENCES
- Saiani L, Palese A. Guida per pianificare lo sviluppo professionale e documentare le competenze. Assist Inferm Ric 2002; 21: 1-58.
- Decreto legislativo del 27 ottobre 2009, No. 150.
- Del Vecchio M, Lega F, Prestini A, Sartirana M. Mappatura e valutazione delle competenze dei professionisti nelle aziende sanitarie: sistemi e strumenti a confronto. Rapporto OASI 2018 CERGAS. 2018.
- Mathieu G, Greco A, Nardi R, Stornello M, Berti F, Colombo F, et al. La clinical competence in Medicina Interna. Ital J Med 2011; 5:17-29
- Lega F, Pirola F, Panella V, Sartirana M, Toparini F. La valutazione delle competenze manageriali dei professionisti con il sistema a 360°: la prima sperimentazione in una ASL. Rapporto OASI 2011 CERGAS. 2011.
- Joint Commission International. Joint Commission International Accreditation Standards for Hospitals 6th Edition. Joint Commission Resources. 2017 Luglio.
- Emilia-Romagna Regional Council Resolution No. 327 of 23 February 2004.
- Emilia-Romagna OIV-SSR Resolution of 15 May 2017, No. 5.
- Zucal M, Barelli P, Bertoni O, Pedrolli S, Recla M, Ventura L. Costruzione ed implementazione di un modello aziendale strutturato per la gestione delle competenze per il personale tecnico sanitario di radiologia medica. GIHTAD. 2019 Marzo; 12:1.
- Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;65:S63-S67
- Mintzberg H. Structure in fives: Designing effective organizations. Prentice/Hall International. 1983.
- Barbieri M, Girosante G, Valotti G. La valutazione dei medici: un modello attuativo. Rapporto OASI 2012 CERGAS. 2012.
This work is licensed under a