Posted by in Articles | 0 comments

 Abstract

Background: Effective communication is crucial to the dentist-patient relationship. Part of the challenge that arises in integration of communication skills within dental education is in assessing attainment of these skills. This paper discusses the current evidence base surrounding assessment of communication skills among dental students.

Methods: A literature review was performed using OVID with the key search terms (Dental) AND (Education OR Learning OR Teaching OR Instruction) AND (Communication) AND (Assessment OR Evaluation). The reference lists of relevant journals and “similar articles” search function were also used. An additional search of journals, reports and book material was made via the World Wide Web using the same key words.

Results: The evidence base for communication skills assessment is sparse in the field of dentistry. Tools have been adapted from other health professions, however it is not accurate to assume that demonstrated validity and reliability is transferred directly to a dental context. Despite promising developments, no assessment tools have demonstrated favourable psychometric properties in dental applications.

Conclusion: Assessment of communication in dentistry is a relatively young field of research. There is currently inadequate evidence to define effective assessment practices. Intra/post-operative interactions and multiple source assessment are areas in which further research is required. Sharing of psychometric data is important to afford efficient advancements in this field.

 

Introduction

The Australian Dental Council is responsible for setting the educational standards of the profession in Australia. These standards are listed in a formal ADC publication entitled “Professional Attributes and Competencies of the Newly Qualified Dentist.”(1) Among these listed qualities, communication plays a central role, featuring in 4 of the 6 domains of competency and earning its own explicit domain listing.

The inclusion of communication benchmarks in such prominent policy documents mandate the inclusion of communication education within the undergraduate dental curriculum.(2) In the competency-based education paradigm in which Western dental education now functions, such mandates should also be subject to measures of their attainment. It is likely that this is an expectation of the community since there is evidence that good dentist communication skills can result in improved patient outcomes and are highly valued by patients.(3) There are also clear benefits in assuring communication competency from a practitioner perspective, since evidence exists to suggest that patients infer a clinician’s technical competency and knowledge based on communication abilities.(4) Effective communication skills have also been found to translate to improved patient loyalty(5) and reduction of litigious claims.(6) It would therefore appear in both the public and profession’s interests to adopt a similar level of measurement standard as that expected of other clinical proficiencies in dentistry before attributing ‘competence’ to graduates.(2, 7)

This paper seeks to explore the current evidence that surrounds assessment strategies for communication skill evaluation in dental education. Desirable properties of assessment tools and areas in which there are opportunities for further research will also be identified.

Methods

A search for papers published between 1990 and 2012 was conducted in October 2012 using the electronic journal database search platform OVID. The key search terms that were used were (Dental) AND (Education OR Learning OR Teaching OR Instruction) AND (Communication) AND (Assessment OR Evaluation). The reference lists of useful journals were used to identify other relevant articles and historical content. The “similar articles” search function within OVID was also used to access related articles across health professional fields. A secondary search of the Journal of Dental Education Archives was performed by hand. An additional search of journals, reports and book material was made via the World Wide Web using the same key words. These resources were manually reviewed by the author to determine research quality and relevance to this narrative review paper.

 

 

What evidence exists to guide communication skill assessments in dentistry?

A key challenge posed by communication assessment is that it does not conform to many of the traditional assessment strategies used in medical and dental education to test if students can remember facts.(7) At its core, communication is an applied behavioral science, yet it is still viewed with suspicion as a ‘dark art’ by a profession rooted in ‘hard science’, technical solutions and quantification. It has been observed that communication has historically been taught in dentistry through observed interactions of a more experienced clinician such as a tutor. Assessment of communication abilities has followed suit in subjective valuations, often by the same faculty member providing the role modeling. Such assessment has typically been woven obscurely into an overall impression of student ability in items such as sessional or daily grading. As a consequence, this area has been subject to widely varied expressions of the ‘hidden curriculum’ and broad bias towards qualities valued by the individual assessor.(7) This has been the experience of the author, despite only recent involvement as an undergraduate. The overall result is a lack of clarity and consistency in the communication skills of dental graduates.(7)

Pioneering educators have attempted to improve the standards of dental education in the field of communication, particularly over the past decade. Much of the work in this area has been undertaken with close reference to the strategies adopted across other health professions, especially medicine. Checklists of desirable communication qualities originally designed for physicians such as the MACY model,(8) have frequently been retrofitted to the dental context.(9) Such models however, fail to acknowledge some of the communication skills that are unique to the dental context. The high anxiety often associated with the dental environment, technical basis of the profession and inability for the patient to speak for extended periods during treatment pose unique communication challenges that must be addressed by dentists.(4, 9) Hybridised systems of assessment using existing methods validated in their application in medical education and adjoined by some dental additions tend to ensue. This is problematic when validity and reliability demonstrated in a medical context is assumed to be transferable to its amended dental application.(2)

A systematic review published by Carey et al.(2) in 2010 revealed 11 ‘quality’ studies in the field of communication in dental education. The assessment techniques employed in these studies are summarised in table 1.

 

 

 

 

Table 1. Assessment tools used as part of communication education interventions in review by Carey et al.(2)

Author(s) Assessment tools used as part of intervention
Broder & Janal (10) 2006 USA Arizona Clinical Interview Rating

Student evaluation questionnaire

Cannick et al.(11) 2007 USA Interpersonal communication checklist

Tobacco cessation counseling checklist

Videotape reviewing

Croft et al.(12) 2005 UK Student evaluation questionnaire
Gorter & Eijkman(13) 1997, Netherlands Course evaluation questionnaire
Hannah et al.(14) 2004, New Zealand Patient response form by simulated patient

Marking by tutor

Student evaluation questionnaire

Hiler (15) 2001, USA Global rating based on rating scale
Hottel & Hardigan(16) 2005, USA Behavioural observation form
Ratzmann et al.(17) 2007, Germany Course evaluation questionnaire
Theaker et al(7). 2000, UK Dental Consultation Communications Checklist (DCCC) pilot
Van der Molen et al.(18) 2004, USA Knowledge MCQ

Behaviour role play test

Video-taped assessment using DCCC

Learner Log

Wagner et al.(19) 2007, USA Interview checklist

 

 

Prior to 2010, the only research to focus on developing a communication assessment tool specific to the dental context was research by Theaker et al.(7) The instrument created by this group is the Dental Consultation Communication Checklist (DCCC). Use of a Likert scale within this assessment instrument is purported to impart sensitivity to its findings, a feature that is likely to improve its ability to provide feedback and ‘fine-tune’ skills.(7) This potential benefit must be balanced with a converse consideration that dichotomous ‘checklist’ assessments have been shown to provide clearer behavioural definitions and increased reliability for less experienced assessors in medical applications.(20)

A key strength of the DCCC is that it was formed based on observed interactions between clinicians and patients in an oral medicine clinic over time. This affords exposure to a wide variety of patient responses and communication demands in an authentic (albeit highly specific) context, thus providing content validity for this particular purpose. Although not explicitly noted by the authors of the paper, it is important to acknowledge that the context of an oral medicine clinic in which this assessment tool was developed varies markedly from restorative and other specialist dental clinics in terms of the nature and frequency with which invasive procedures are performed. To draw an illustrative medical comparison, this would be akin to using the same tool to assess communication strategies employed in an oncology ward to those required in an emergency department. Although certain features are shared, the specific context of an oral medicine clinic may intuitively affect the relative importance of anxiety reduction strategies and procedural descriptions, instead placing increased emphasis on patient counseling. Such a finding compromises the generalizability of conclusions from this study across broader clinical settings until they are assessed more broadly.

The DCCC was used in a subsequent study by Van der Molen et al.(18) on the basis of it being “reliable (high inter-observer agreement) and appropriate for the assessment of communication skills of dentistry students.” The latter half of this statement is not supported by the findings of Theaker et al.(7) and disregards their warning that “further work is needed to confirm the instrument’s construct validity, intra-observer reliability…and sensitivity to education training interventions.”(7) On this basis, care is warranted in applying the DCCC before a psychometric suite is able to demonstrate its effects across clinical applications. Despite this shortcoming, Van der Molen et al.(18) are credited for their ‘holistic’ approach towards communication assessment, including a multiple-choice questionnaire, behavioural role-play test, video-taped assessment and learner log. Unfortunately, the increased validity purported to be associated with such an approach has not been confirmed through real-world application analyses to date.

Despite a promising beginning to 21st century communication assessment through the formation of a dentally specific tool,(7) problems with research design and lack of psychometric analysis has ensued.(2, 9) Key criticisms forwarded by Carey et al.(2) towards the existing research in this field are that some studies rely solely on the use of student satisfaction questionnaires,(12, 13, 17) use medical methods that have not been validated in dentally-specific contexts,(10, 11, 14, 16, 19) or use non-specific global-rating scores.(15) A further major flaw identified in assessment tools prior to 2010 is that they failed to acknowledge the patient’s perspectives as part of the evaluation, or even as a part of validation of the tools used.(2)

Although research by Hannah et al.(14) in 2004 went some way to addressing this fault, the use of simulated patients in this intervention may have hindered the authenticity of the patient response, since actors may become socialized into relating to students.(21) This is a key general criticism of using an OSCE format for communication assessment, since this typically requires use of a standardized simulated patient. Compromised authenticity and unrealistic resource requirements have been criticisms associated with such strategies. (2, 21)

Given that patients make up half of the dentist-patient relationship, presumably patient perspectives are of considerable worth in designing communication assessment tools and even in the assessment itself.(9) Schirmer et al.(21) state that such efforts within a broader medical context increase validity and enhance the psychometric properties of an assessment item. The notion of patient-derived assessment information has only been a recently acknowledged prospect in the field of dentistry.(9)

Research by Wener et al.(9) published in 2011 has attempted to integrate dental patient perceptions into the design of a communication assessment tool as well as the assessment process itself. Consideration of all stakeholder perspectives in assessment design is certainly a progressive approach, however more research is needed to validate the assessment methods forwarded. Schirmer et al.(21) warn that in medical applications of such strategies, patient evaluations can be subject to the ‘halo effect’ of relationship continuity as well as bias introduced through the perception of the extent to which their reason for seeking care has been satisfied. Wener et al.(9) appear aware of risks associated with overdependence on patient ratings, hypothesizing that tutor evaluations through live or video assessments as well as simulated patient interactions should be used to complement patient-derived assessments. Whilst this multi-modal approach would seem to be supported anecdotally in both medical and dental research to date, conclusions cannot be drawn until the applied psychometric properties and pragmatic realities of these tools are known.(2, 21)

Based on the evidence that is available through the methods employed by this paper, there is currently no communication assessment tool for application in dentistry that is well supported by the literature. This presents both a challenge and an opportunity for dental educators. It is important that in the effort to advance this field, lessons from the past are heeded and development of tools is approached in a systematic way.

Key lessons and future directions in communication assessment design in dental education

Although there is not significant evidence to support one particular method of communication skill assessment, some guidance can be gained from the existing evidence base. The following recommendations are based on the published research and expert commentary in this field. Quality research into an appropriate assessment tool for the evaluation of communication skills among dental students is likely to:

  • Be designed specifically for application in a dental context.(2, 9)
  • Be subject to psychometric analysis in a dentally-specific context (rather than assuming it if applied in other health professions).(2)
  • Include both verbal and non-verbal communication skills as part of the assessment.
  • Use a control group and randomization where possible.(2)
  • Strive for authenticity in assessment. This may include considerations such as only having the student and patient present (e.g. using video-based and/or patient-derived assessment, real patients in preference to actors and selection of random interactions rather than a test event.)(2, 21)
  •  Use standard-setting to determine the relative importance of each element of the assessment.(19)
  • Be applicable across a broad range of clinical contexts and institutions to facilitate collaboration and comparisons across cohorts and longitudinally.(2)
  • Consider patient satisfaction/ratings as part of the assessment.(2, 9, 21)
  • Involve liaison with all key stakeholders as part of the assessment design process.(9, 21)
  • Avoid sole reliance upon student satisfaction scores.(2)
  • Include video review to facilitate standardisation of assessment and training of staff.(2, 21)
  • Clearly define target skills to direct learning and assessment. This is critical in facilitating observer training to maximize reliability. (2, 21)

A key area of communication skill assessment warranting further exploration is the interactions that occur during the intra and post-operative period. To date, all published dental communication programs have sought to examine only pre-operative consultation skills.(2, 9) The finding that most patient anxiety is associated with events occurring during treatment makes it imperative that assessment of communication during these other very important times are included in the determination of professional competency.(15)

It must be acknowledged that a weakness of this paper is that the author is not affiliated closely with a dental teaching institution. This context means that only communication assessment strategies that are published have been able to be included in this paper. It may be that assessment methods validated by psychometric data are currently being applied in communication assessment in dental education, but have not yet been exposed to a public forum.  This narrative has been limited by the narrow scope of research that exists in this area. As a result, this research paper has evolved to include a significant portion dedicated to commentary on where research could be usefully directed.

This paper has therefore not provided a conclusive answer to the initial question of ‘how can communication skills among dental students be assessed?’ Educators cannot simply sit on their hands until techniques are proven, since it is innovation (and sometimes adversity) that drives progress in any field. In practical terms, documents such as the ADC’s “Professional Attributes and Competencies of the Newly Qualified Dentist” provide a picture of the desired outcomes. Patients may provide another valuable source of information about important qualities to assess. Many tools already exist to apply these principles. The next critical step for any researcher in this field is to go beyond the architecture of an instrument or swag of instruments, to the daunting yet vital stage of psychometric evaluation. It is an unfortunate finding of this paper that no research to date in dentistry has ventured to provide sound post-application analysis. It is the author’s opinion that it is only through such systematic and self-critical measures that communication is ever likely to be considered a worthy associate to technical skills and clinical sciences by the broader academic dental community.

Conclusion

Communication is now a mandated component of dental graduate competence and is highly valued by patients. There is currently no widely accepted standard tool for assessing communication skills in dental or medical education.(2, 21) Existing studies in the dental context rely heavily on student-satisfaction evaluations, which do not provide an objective measure of the skills obtained, or the impact of the educational strategies applied. Many of the assessment tools that have been applied in dentistry are borrowed from medical fields and amended to suit the dental context. There is no published evidence that the tools currently applied in dental education to assess communication skills are psychometrically sound.

Since it is often assessment that drives student priorities for learning, there is a pressing demand to establish reliable and valid methods to assess competency in communication among dental students. This represents an important opportunity to clarify communication standards within the dental profession, so that these can be integrated systematically into dental education at appropriate levels. Research efforts should focus on multiple-source assessments with input from key stakeholders as part of the assessment design process. Sound psychometric principles must be applied in analyzing assessment tools in order to identify strengths and weaknesses of current and prospective methods so that developments in this field can receive an evidence-based acceleration to meet existing demands.

 

 

 

 

References

 

1. Professional Attributes and Competencies of the Newly Qualified Dentist. Australian Dental Council; 2010 [cited 2012 June 15th]. Available from: http://www.adc.org.au/adcmajoractivities.html.

2. Carey J, Madill A, Manogue M. Communication skills in dental education: a systematic research review. European Journal of Dental Education. 2010;14:69-78.

3. Newsome P, Wright G. A review of patient satisfaction. 2: Dental patient satisfaction: an appraisal of the literature. British Dental Journal. 1999;186:166-70.

4. Corah N, O’Shea R. Development of a patient measure of satisfaction with the dentist: The dental visit satisfaction scale. Journal of Behavioural Medicine. 1984;7:367-73.

5. Holt V, McHugh K. Factors influencing patient loyalty to dentist and dental practice. British Dental Journal. 1997;183:365-70.

6. Mellor A, Milgrora P. Dental attitudes toward frustrating patient visits: relationship to satisfaction and malpractice complaints. Community Dentistry and Oral Epidemiology. 1995;23:15-9.

7. Theaker E, Kay E, Gill S. Development and preliminary evaluation of an instrument designed to assess dental students’ communication skills. British Dental Journal. 2000;188(1):40-4.

8. Janicik R. The Macy model: structure and sequence of effective doctor-patient communication. New York: New York University, Macy Initiative in Health Communication; 1999.

9. Wener M, Schonwetter D, Mazurat N. Developing New Dental Communication Skills Assessment Tools by Including Patients and Other Stakeholders. Journal of Dental Education. 2011;75(12):1527-41.

10. Broder H, Jabal M. Promoting interpersonal skills and cultural sensitivity among dental students. Journal of Dental Education. 2006;70:143-7.

11. Cannick G, Horowitz A, Garr D, Reed S, Neville B, Day T, et al. Use of the OSCE to evaluate brief communication skills training for dental students. Journal of Dental Education. 2007;71:1203-9.

12. Croft P, White D, Wiskin C, Allan T. Evaluation by dental students of a communication skills course using professional role-players in a UK school of dentistry. European Journal of Dental Education. 2005;9:2-9.

13. Gorter RC, Eijkman MA. Communication skills training courses in dental education. European journal of dental education : official journal of the Association for Dental Education in Europe. 1997 Aug;1(3):143-7.

14. Hannah A, Millichamp CJ, Ayers KM. A communication skills course for undergraduate dental students. J Dent Educ. 2004 Sep;68(9):970-7.

15. Hiler C. Psychological consultation in dental training. Journal of Psychotherapy Independent Practice. 2001;2:91-104.

16. Hottel TL, Hardigan PC. Improvement in the interpersonal communication skills of dental students. J Dent Educ. 2005 Feb;69(2):281-4.

17. Ratzmann A, Wiesmann U, Gedrange T, Kordass B. Early patient contact in undergraduate dental education in Germany–‘The Greifswald Model’. European journal of dental education : official journal of the Association for Dental Education in Europe. 2007 May;11(2):93-8.

18. Van der Molen H, Klaver A, Duyx M. Effectiveness of a communication skills training programme for the management of dental anxiety. British Dental Journal. 2004;196(2):101-7.

19. Wagner J, Arteaga S, D’Ambrosio J, Hodge CE, Ioannidou E, Pfeiffer CA, et al. A patient-instructor program to promote dental students’ communication skills with diverse patients. J Dent Educ. 2007 Dec;71(12):1554-60.

20. Regehr G, MacRae H, Reznick R, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Academic Medicine. 1998;73(9):993-7.

21. Schirmer J, Mauksch L, Lang F, Marvel M, Zoppi K, Epstein R, et al. Assessing Communication Competence: A Review of Current Tools. Family Medicine. 2005;37(3):184-92.