ABSTRACT
Background: Evolving innovative, feasible and cost effective technology related with teaching and learning in context of recent COVID-19 pandemic has led the medical educationists to think about its increased use in the assessment process. Need of the hour is to develop and establish a framework in our institutions for technology enhanced assessment (TEA) efficiently. European countries were using the e- assessment before the COVID-19 pandemic so their education system was less disturbed by it as compared to the Asian countries like Pakistan. TEA can be done with a number of tools and mobile phone technology, learning management systems (LMS) and social media have played remarkable role in this regard. The objective of this scoping review was to find best practice regarding the use of TEA in context of COVID-19 pandemic according to the Best Evidence Medical Education (BEME) principles. Methods: Databases which were searched were Google Scholar, Educational Resource Information System (ERIC), Cochrane library and ScienceDirect (Elsevier). The key words and phrases which we used for search were "TEA", "Online assessment", "assessment in context of COVID-19", "Role of TEA", "Advantages and disadvantages of TEA". 1252 articles were retrieved after screening total searched articles by Mendeley software. These articles were then assessed according to the eligibility criteria and 84 articles were selected out of which finally 12 articles were included in this scoping review after assessment of quality by Qualsyst appraisal tool. Data was extracted by using data extraction form which was synthesized to create the themes by axial coding. Results: After synthesis of data following themes were identified describing the TEA. What is TEA, role of TEA. Advantages and disadvantages of TEA, measurement and implementation issues in TEA. Prime benefit of TEA is its potential to foster learning by its ability to deliver vast variety of test formats in high fidelity virtual reality situations and by providing feedback after formative assessment using number of digital tools. Major issues related to TEA are cheating, validity, quality and security of test materials. Conclusion: E-assessment tools were already being used in European countries with the intent to improve the teaching and learning activities but after the COVID-19 pandemic it became the necessity. TEA has extreme potential for improving the learning and assessment process especially in context of current COVID-19 pandemic if the available technology is used efficiently. Implementation of TEA in place of conventional assessment should be tailored according to the local requirements and context. Finally the word of caution is to be prepared to address the issues of validity, quality, cheating and security of test materials connected with this technological change.
Key words: technology enhanced assessment, role, formative assessment, feedback
INTRODUCTION
Evolving innovative, feasible and cost effective technology related with teaching and learning in context of recent COVID-19 pandemic has led the medical educationists to think about its increased use in the assessment process.[1,2] It can be used to deliver and manage both formative and summative assessments. Need of the hour is to develop and establish a framework in our institutions for technology enhanced assessment (TEA) efficiently.[3,4] For this purpose we have to accept the challenge to develop the guidelines and arrange for the resources. TEA can be done with a number of tools e.g. mind maps, self-assessment, discussions with the help of e-communication tools, peer assessment, viva, Item response based tests and e-portfolios. Modern integrated curricula use problem based learning (PBL) as a common instructional strategy and TEA can be efficiently used to assess the students in a virtual environment which closely matches the real problem situation in which these students will work during their professional life.[5,6]
European countries were using the e-assessment before the COVID-19 pandemic so their education system was less disturbed by it as compared to the Asian countries like Pakistan.[7,8] A number of challenges were faced by our country due to the COVID-19 pandemic and one of these major challenges was threat to our education system particularly medical education. Assessment being an integral part of learning process needs to be enhanced under the special circumstances of COVID-19 pandemic with the help of available technology. Scoping reviews are best tool for mapping the literature and finding the gaps in the knowledge regarding a new topic. The objective of this scoping review was to find best practice regarding the use of TEA in context of COVID-19 pandemic according to the Best Evidence Medical Education (BEME) principles.
METHODOLOGY
A team of two reviewers was formed which after discussion developed a research question and decided a scoping review search strategy for this scoping review. Ethical approval was applicable to this review study. We used Arksey O’Malley 2005 frame work for this scoping review.
Research question
The research question of this scoping review is: What is the role of TEA in context of COVID-19 pandemic in health professions education?
Study design
Search for this scoping review was not limited to a specific design. Mix design studies were included in this review according to the inclusion and exclusion criteria.
Eligibility Criteria
The two member reviewer team decided the inclusion and exclusion criteria with discussion and consensus. We included both quantitative and qualitative studies and surveys. Grey literature and systematic reviews were included in this scoping review. Only full original articles were included. Chapter from books and all others were excluded (Table 1).
| No. | Number of retrieved articles | Key words and phrases used | Filters used |
| I. Cochrane library | |||
| 1. | 75 | Technology enhanced assessment in medical and dental education | All text |
| 2. | 104 | online assessment in medical and dental education | All text |
| II. Google scholar | |||
| 1. | 240 | assessment in medical and dental education "technology enhanced" | 2017—2021 |
| 2. | 6 | Role of technology enhanced assessment in medical and dental education technology OR enhanced OR assessment OR dental OR education "technology enhanced assessment" | Any time |
| III. Eric | |||
| 1. | 229 | Technology enhanced assessment in medical and dental education | None |
| 2. | 320 | Role of technology enhanced assessment in medical and dental education | None |
| IV. ScienceDirect (Elsevier) | |||
| 1. | 341 | Technology enhanced assessment in dental and medical education | Year 2020 |
| 2. | 395 | Technology enhanced assessment and role and in dental and medical education | Year 2020/21/22 Review and Research articles |
Search strategy
Two member team first of all decided the core concept regarding the defined research question. We decided scoping review strategy for this review by consensus.
Electronic database sources, key words and phrases
Databases which were searched were Google Scholar, Educational Resource Information System (ERIC), Cochrane library and ScienceDirect (Elsevier). The key words and phrases which we used for search were "technology enhanced assessment", "Online assessment", "assessment in context of COVID-19", "Role of technology enhanced assessment", "Advantages and disadvantages of technology enhanced assessment".
Study selection
Articles identified through database searches were screened according to the eligibility criteria to select the final full text articles to be included in this scoping review. All this process has been shown in PRISMA flow diagram (Figure 1). All the searched articles were imported in Mendeley software and then duplicates were removed from these articles. 1252 articles were retrieved after screening total searched articles by Mendeley software. These articles were then assessed according to the eligibility criteria and 84 articles were selected, out of which finally 12 articles were included in this scoping review. Quality of these studies to be included in this review was checked by using Qualsyst appraisal tool and minimum score of the article for inclusion in the review was set at 35% after mutual consensus of the two reviewers (Annexure III). Qualsyst tool consists of 14 item criteria for quantitative studies and 10 item criteria for qualitative studies. Each item in the tool is scored on a three point scale (NA: 0; Partial present: 1; Present: 2). Maximum total score for quantitative study is 28 and for qualitative study it is 20. Formula for calculating the score is given in annexure III. Final score of >75% was considered as high, 60%-74% as good, 35%-59% as sufficient and below 35% as limited quality in this scoping review. Value of 0.80 for Cohen's Kappa coefficient was considered excellent for inter rater reliability.
Figure 1. PRISMA Flowchart diagram.
Publication year
As COVID-19 pandemic started a year back this further highlighted the importance of TEA. Although European countries were already using the technology in the process of assessment but emphasis of research on TEA came after the COVID-19 pandemic. So we did not set any filter by time to include studies in this review but the finally selected studies came out to be within last ten years.
Outcome measures
Prime objective of this scoping review was to collect references from evidence based medicine to inform the practice of TEA and ascertain its role in dental and medical education.
Screening
The studies retrieved by literature search were carefully reviewed by the two reviewers. Firstly the titles and abstracts were reviewed and fulfilling the eligibility criteria to some extent even were screened out. Secondly full text articles and systematic reviews from these screened out studies were assessed by the reviewers according to the inclusion and exclusion criteria (Table 2). Conflict of opinion on two papers was resolved by consensus between the two reviewers after which Qualsyst score improved.
| No | Inclusion criteria | Exclusion criteria |
| 1 | Systematic reviews | Editorials |
| 2 | Short communications | Book chapters |
| 3 | Manual search of grey literature | News papers |
| 4 | Articles in English language only | Dissertations |
| 5 | Original research articles | Abstracts only |
| 6 | Research articles published at any time. | Articles not accessible |
| 7 | Articles related only to medical and dental education. | Government reports |
Data extraction
A data extraction form was developed and pilot tested to calibrate the assessment scoring of the two reviewers. Main entries of the data extraction form were country and year of publication, study design, objectives, main outcome measures, full citation, target population, strengths, limitations, bias, results and conclusion.
Assessment of risk of bias
It was tried to assess the risk of bias in the included studies by at the used questionnaires, interview structure, study population and author affiliation. To check for inter rater reliability of the two reviewers Kappa coefficient was calculated and value more than 0.8 was considered excellent.
Analysis/synthesis of data
Studies were grouped according to the study design, objectives main outcome measures and other characteristics and put in tabulated form for data analysis and synthesis.
RESULTS
The articles were read in detail by the two reviewers and they carried out thematic analysis of the data extracted. Then open coding leading to axial coding was done to develop sub-categories and categories for explanation of themes and sub-themes. We identified the following themes describing the role of TEA in dental and medical education.
What is TEA
Researchers have defined the term TEA in their articles as use of any type of technology in any aspect of assessment. a number of technologies like computers, cell phones, personal digital assistants (PDA), interactive voice response equipment (ivr) video recording, video teleconferencing equipment can be used. Similarly range of test formats like mcqs and structured interviews can be used which can be evaluated by computers instead of human evaluators.
Role of TEA
"Use of technology in assessment has the potential to improve the potential of students' performance" is the final word of many papers on the topic of TEA. others have declared it as way forward in the recent challenging situation of COVID-19. technology can be used to make the assessments more reliable. by use of technology assessments for learning and assessment as learning can be more productive. Similarly assessment of learning can be made more transparent by using technology appropriately.
Advantages of TEA
Different articles have discussed advantages and disadvantages of TEA which come under the following sub-themes: (a) Effects on quality of testing: It enhances the quality of assessment by providing variety of formats in high fidelity situations. (b) Candidate expectations and reactions to TEA: Expectations and reactions of candidates should be taken into account while planning a TEA. For this purpose proper assessment plan and its specifications should be developed. It should be pilot tested before full implementation. (c) Consistency of administration of tests: TEAs offer consistency of tests thus increasing their validity. (d) Consistency of scoring of tests: Scoring of tests by the computers is more consistent than the human evaluators. (e) Administrative ease and flexibility of tests: TEA offers relative ease and flexibility regarding administration of tests.
Disadvantages of TEA
Some of the papers have discussed the disadvantages of TEA which can be listed as under: (a) Costs: While deciding to introduce technology enhanced the cost of the process should be considered first as it may add considerably to the budget of the institution. (b) Security of test materials: The test items and results may be attacked by the hackers to give pre information to the to the test takers leading to cheating. There are number of measures that can be taken to keep these test materials safe e. g. firewall application. (c) Issue of cheating: It is a serious issue in TEA (unproctored internet based tests) in cognitive testing. Cheating can be inferred in these situations three ways: (1) Statistical detection; (2) Score comparison and verification; (3) Technological detection. It can be discouraged by web petrol monitoring, key stroke monitoring, key stroke analytics, web cams, warnings and threats. To make cheating difficult test design features and characteristics like multiple test forms, speeded tests and computerized adaptive tests can be used. (d) Response distortion: This is important in non-cognitive testing. It means that the test taker when asked about the certain content pertaining to him (self-reports) in a survey he gives willfully incorrect answer.
Measurement issues in TEA
TEA has great potential for measurement and prediction of behavior of students. it can offer situations simulating real life conditions in which the students can be assessed. for valid measurements proper assessment plan with specification should be developed.
Implementation issues in TEA
If implemented efficiently TEA has number of beneficial effects like facility of data collection of students taking the test and excellent decision decisions can be taken on the basis of valid test measurements. different frameworks have been proposed for the implementation of TEA (Figure 2 and Figure 3).
Figure 2. Number of studies with their countries of origin.
Figure 3. Bubble plot of publication year of studies.
DISCUSSION
Use of TEA is need of the hour, keeping in mind this important aspect in mind we have carried out a scoping review to collect the evidence from all over the world regarding it. All the studies included in the review were screened for methodological rigor, rater and study selection bias. Overview of these studies is given below.
Overview of studies included in present scoping review
Mostly the studies included in this review state that TEA is beneficial for the learning of the students if used efficiently. A number of technologies are available for administration of tests in variety of formats and their scoring. Although the role of technology in process of assessment cannot be denied in terms of its advantages but disadvantages and challenges in terms of costs, issues of cheating and response distortion, student expectations and reactions and validity issues should also be considered seriously.
Sweeny et al. [9] in their study discuss the role of TEA in higher education contexts. They conducted a systematic review and found out that at the moment TEA is being used mostly for formative assessment to give feedback to the students. They stress upon the need to develop the good assessment feedback according to the principles described by them. They talked about the use of technological pedagogical and content knowledge (TPACK) framework and substitution, augmentation, modification, redefinition (SAMR) model.[10,11] Devedzic defined the term TEA as "a broad term which encompasses diverse methods by which technology can be used to support the management and delivery of assessment in educational institutions in workplace, lifelong learning and professional training and development".[12,13] Whitelock et al.[14] conducted a systematic review to collect from the literature about the use of technology in assessment. They found that majority of literature focused on that technology enhancement can enable assessment and feedback to meet their goals and the ways assessment can be designed to best use of technology. Moccozet et al.[15] in their study worked on online summative assessments in a blended learning context at the level of higher education. They mention some of the advantages of e-assessments most important of which is that scoring in this system is quick and this assessment can be done anywhere any time. The results of e-assessment are reusable and students can revisit their answers any time.[16] Ferreira et al. [17] stressed the need for awareness of students about their academic performance so that they can self-regulate them and by doing so teaching process will also be improved. In e-assessment and learning (open student models) intelligent interfaces which can be visualized can make learning interactive, fostering and easier.
Zickar et al.[18] highlighted some important aspects while changing the system to TEA i.e. ethical, scientific and practical issues. They also talked about the brain scanning and imaging techniques for selection of personnel. They also evaluated the potential of virtual reality assessment.[19] Rasman et al. [20] and Csapó et al. [21] compiled papers from 20th international conference on assessment and worth mentioning is the paper describing the pathways to successful online assessment. Hettiarachchi et al. [22] in their study declare that TEA can be used for both skill and knowledge learning and these e-learning systems make students more motivated and engaged for efficient learning. They also described the characteristics of e-assessment system as one which supports flexible and adaptive learning offers personalization of assessment activities, reduces potential for cheating, monitors question quality and statistical analysis, provides automatic marking and immediate feedback and supports variety of question formats for tests.[23] Huertas et al. [24] and Tomas et al.[25] described the use of technology in both online and blended educational contexts and stressed upon the need of more interactive tools for engaging and personalized assessment experiences for replacing traditional assessment practices.
A wide range of tools are available for TEA but issue of security of test materials is a serious one. Scott et al. [26] discussed this important issue in their book and also described different tools that can be used to deliver different test formats. Hense et al. [27] and Gaytan et al. [28] presented a case study of assessment centers and documented use of technology in these centers. Assessment is an important part of learning process. Cognitive, attitude as well as psychomotor domain needs to be tested. TEA offers wonderful opportunities for all of these. Capella et al.[29] and Stödberg et al. [30] in their study compared traditional assessment with TEA and recommended the use of technology in assessment. Burke et al.[31] and Petrisor et al. [32] in their study also recommended use of technology in assessment process. Kavanagh et al. [33] and Sim et al. [34] in their study tested the use of technology for enhancing laboratory sessions in health science and described student experience and perceptions regarding use of digital technology in assessment.
Recommendations
Internet connectivity has affected almost every aspect of our lives by facilitating our working but it has also raised some new issues and problems as well. The discipline of health professions has also been affected by it more so positively. Technology has been used widely in teaching, learning and assessment because of easily available and affordable tools. Use of technology in assessment process has been largely welcomed by all the stakeholders because of the ease and flexibility it offers.
On the basis of this review we recommend the use of TEA in our institutions because it fosters student learning, engages them and provides opportunities for self-assessment especially in the context of recent COVID-19 pandemic. Almost all the studies and reviews included in this scoping review support the use of TEA for efficiently using the result data in student centered curricula. We recommend to carefully select the assessment tool according to the local requirements and context. Feasibility of the assessment tool selected is also important in context of its cost. While replacing the conventional system of assessment with technology enhanced system of assessment issues of its validity, test material security, quality of assessment process, issue of cheating and reaction of stakeholders should be considered seriously and one should be ready to address these problems beforehand.
We recommend future systematic reviews and studies in this topic area to further inform the practice of TEA in dental and medical education as evidence for some aspects of TEA is still equivocal.
Strengths and limitations of this scoping review
We used Arksey and O’Malley’s framework 2005 and Qualsyst tool for this scoping review and tried to make the whole process rigorous and transparent. We used standard search strategy and searched almost all common databases for articles and PRISMA flow diagram to assess eligibility, screen and finally include the articles in this scoping review. Thematic analysis and axial coding was used for data collection and synthesis.
Limitations of our study are that we did not search all databases so there is possibility of missing some of the quality studies. Also we did not consult the experts and researchers to include some additional studies in this scoping review. Also the studies included in this review were reviewed and assessed by only two reviewers which may subject this study to reviewer bias although we calculated kappa coefficient to check for inter rater reliability. Lastly time constraint was also a factor which could affect the quality of this review.
CONCLUSION
Medical educationists have been thinking of the ways to continue and foster the learning of the students since the start of COVID-19 pandemic in a safe and conducive environment. Although e- assessment tools were already being used in European countries with the intent to improve the teaching and learning activities but after the COVID-19 pandemic it became the necessity. Affordable and easily available assessment e-tools have made it even more tempting for medical educationists to use them for engaging their students in the learning process. E-assessment is mostly being used for formative assessment and giving feedback to the students. Mobile phone technology and learning management systems of different types are freely being used for this purpose. Cheap and widely available internet connectivity has made this task even easier. Different of frameworks have been developed TEA implementation in the higher education institutions.
On the basis of this scoping review it is evident that TEA has extreme potential for improving the learning and assessment process especially in context of current COVID-19 pandemic if the available technology is used efficiently. Implementation of TEA in place of conventional assessment should be tailored according to the local requirements and context. Finally the word of caution is to be prepared to address the issues of validity, quality, cheating and security of test material connected with this technological change.
Declaration
Author contributions
Rashid S concepted and designed the study; Rashid S collected and analyzed the data, and drafted the manuscript.
Source of funding
This research received no external funding.
Conflict of interest
The authors declare no competing interest.
Data availability statement
Data supporting the findings of this study are available upon reasonable request from the corresponding authors.
REFERENCES
- Bree R, Healy E, Maguire M, Faller D, Harding N, Mulvihill A, et al. Introducing technology enhanced assessment methods (TEAM) to health and science practical settings; bringing digital skills to laboratory and clinical skill sessions. Accessed October 24, 2021. https://eprints.teachingandlearning.ie/id/eprint/6589/.
- Alhonkoski M, Salminen L, Pakarinen A, Veermans M. 3D technology to support teaching and learning in health care education–A scoping review. Int J Educ Res. 2021;105:101699.
- Webb M, Gibson D. Technology enhanced assessment in complex collaborative settings. Educ Inf Technol. 2015;20(4):675-695.
- UNESCO. Policy makers re-examine implications and effectiveness of ICT use in Education. Accessed October 24, 2021. https://www.unesco.org/en/digital-education.
- Khalaf K, El-Kishawi M, Mustafa S, Al Kawas S. Effectiveness of technology-enhanced teaching and assessment methods of undergraduate preclinical dental skills: a systematic review of randomized controlled clinical trials. BMC Med Educ. 2020;20(1):1-3.
- Pelgrum WJ, Anderson R. ICT and the Emerging Paradigm for Life Long Learning. Amsterdam: IEA, 1999.
- Khan RA, Jawaid M. Technology enhanced assessment (TEA) in COVID 19 pandemic. Pak J Med Sci. 2020;36(COVID19-S4):S108.
- Siragusa L, Dixon K. Theory of planned behaviour: Higher education students' attitudes towards ICT-based learning interactions. Accessed October 12, 2009. https://www.semanticscholar.org/paper/Theory-of-planned-behaviour%3A-Higher-education-Siragusa-Dixon/e6925f8d3d64bd72b483805e537ac8c2e4507f1a.
- Sweeney T, West D, Groessler A, Haynie A, Higgs BM, Macaulay J, et al. Where's the transformation? Unlocking the potential of technology-enhanced assessment. Teach Learn Inq. 2017;5(1):1-6.
- Tippins NT, Adler S. Technology-enhanced assessment of talent. Accessed October 12,2021.https://books.google.com.pk/books?hl=en&lr=&id=TbVgLAXYTewC&oi=fnd&pg=PR13&dq=Tippins+NT,+Adler+S.+Technology-enhanced+assessment+of+talent.+John+Wiley+%26+Sons%3B+2011+Mar+29.&ots=9T5_n71MfT&sig=z0YQuRGP4Fr-Bl3oL9PwsH8Gcq8&redir_esc=y#v=onepage&q&f=false.
- Kozma RB. Comparative analysis of policies for ICT in education. In: Voogt J, Knezek G, eds. International Handbook of Information Technology in Primary and Secondary Education. Springer; 2008:1083–1096.
- Devedzic V, Devedzic M. Technology-Enhanced Assessment at universities and in schools: An initiative. Int J Learn Teach. 2019;11(3):89-98.
- Cook J, Jenkins V. Getting Started with e-Assessment, January 2010. Accessed May 31, 2016. http://opus.bath.ac.uk/17712/.
- Whitelock D, Gilbert L, Gale V. Technology Enhanced Assessment and Feedback: How is evidence-based literature informing practice? Accessed November 15, 2021. https://oro.open.ac.uk/29081/.
- Moccozet L, Benkacem O, Burgi PY. Towards a technology-enhanced assessment service in higher education. Accessed Sep 21, 2016. https://link.springer.com/chapter/10.1007/978-3-319-50340-0_40.
- Bree R, Healy E, Maguire M, Faller D, Harding N, Mulvihill A, et al. Technology Enhanced Assessment Methods (TEAM) in Science and Health Practical sessions: a progress report on an Irish multi-institutional initiative. Accessed November 11 2017. http://teamshp.ie/wp-content/uploads/2017/09/TEAM-AHE-poster.pdf.
- Ferreira H, de Oliveira GP, Araújo R, Dorça F, Cattelan R. Technology-enhanced assessment visualization for smart learning environments. Smart Learn Environ. 2019;6(1):1-7.
- Zickar MJ, Lake CJ. Practice Agenda: Innovative Uses of Technology-Enhanced Assessment. In: Tippins NT, Adler S, Kraut AI, eds. Technology-Enhanced Assessment of Talent. 2011:394-417.
- Sadid-Zadeh R, D'Angelo EH, Gambacorta J. Comparing feedback from faculty interactions and virtual assessment software in the development of psychomotor skills in preclinical fixed prosthodontics. Clin Exp Dent Res. 2018;4(5):189–95.
- Rusman E, Nadolski R, Ackermans K. Students’ and Teachers’ perceptions of the usability and usefulness of the first viewbrics-prototype: A methodology and online tool to formatively assess complex generic skills with video-enhanced rubrics (VER) in dutch secondary education. Accessed October 19, 2021. https://link.springer.com/chapter/10.1007/978-3-030-25264-9_3.
- Csapó B, Ainley J, Bennett RE, et al. Law, Technological Issues for Computer-Based Assessment. In: Assessment and Teaching of 21st Century Skills. Springer; 2012:143–230.
- Hettiarachchi E, Mor E, Huertas MA, Rodriguez ME. A Technology Enhanced Assessment System for Skill and Knowledge Learning. Accessed October 9, 2021. https://www.scitepress.org/Papers/2014/48456/48456.pdf.
- Maki PL. Developing an assessment plan to learn about student learning. J Acad Librariansh. 2002;28(1):8–13.
- Huertas MA, Mor E. Technology-Enhanced Assessment (TEA) in online education: Challenges and opportunities. Accessed October 9, 2021. https://ieeexplore.ieee.org/abstract/document/7083866.
- Tomas C, Borg M, McNeil J. E-assessment: institutional development strategies and the assessment life cycle. Br J Educ Technol. 2015;46(3):588–596.
- Scott JC, Bartram D, Reynolds DH, eds. Next Generation Technology-Enhanced Assessment: Global Perspectives on Occupational and Workplace Testing. Cambridge University Press; 2017.
- Hense R, Janovics J. Case study of technology-enhanced assessment centers. Technology- enhanced assessment of talent. In: Tippins NT, Adler S, Kraut AI, eds. Technology-Enhanced Assessment of Talent. 2011;30:324.
- Gaytan J, McEwen BC. Effective online instructional and assessment strategies. Am J Distance Educ. 2007;21(3):117-132.
- Capella JV, Ors R. Technology enhanced assessment model in higher education e-Learning. E-Learning. Accessed October 3, 2021. https://elearning.metropolitan.ac.rs/files/pdf/2013/26-juan-v-capella-rafael-ors-technology-enhanced-assessment-model-in-higher-education-e-learning.pdf.
- Stödberg U. A research review of e-assessment. Assess Eval High Educ. 2012;37(5):591–604.
- Burke E, Mahoney-Phillips J, Bowler W, Downey K. Going Online with Assessment: Putting the Science of Assessment to the Test of Client Need and 21st Century Technologies. In: Tippins NT, Adler S, Kraut AI, eds. Technology-Enhanced Assessment of Talent. 2010;30:356.
- Petrisor M, Marusteri M, Simpalean D, Carasca E, Ghiga D. Medical Students. Acceptance of Online Assessment Systems. Acta Medica Marisiensis. 2016;62(1):30–32.
- Kavanagh Y, Brazil D, Noonan G, Dowling D, Bree R, Healy E, et al. Studying Student Experience of Technology Enhanced Assessment Methods (TEAM) in Science and Health in Ireland. MRS Adv. 2018;3(12):631-636.
- Sim G, Holifield P, Brown M. Implementation of computer assisted assessment: lessons from the literature. ALT-J. Res Learn Technol. 2004;12(3):215–229.
