STANDARDIZATION OF EVALUATION TOOL BY COMPARING SPOT, UNSTRUCTURED VIVA WITH OBJECTIVE STRUCTURED PRACTICAL EXAMINATION IN MICROANATOMY

SABNIS A.S.¹*, BHOSALE Y.^2
1Department of Anatomy, K.J. Somaiya Medical College, Mumbai- 400022, MS, India.
²Department of Anatomy, Seth G.S. Medical College, Mumbai- 400012, MS, India.
* Corresponding Author : dranjus2003@yahoo.com

Received : 16-10-2012     Accepted : 25-10-2012     Published : 29-10-2012
Volume : 3     Issue : 7       Pages : 225 - 228
Int J Med Clin Res 3.7 (2012):225-228
DOI : http://dx.doi.org/10.9735/0976-5530.3.7.225-228

Conflict of Interest : None declared
Acknowledgements/Funding : Our sincere thanks to Dr. P.S. Bhuyian, Dr. Avinash Supe for their valuable guidance

Evaluation of students is a vital component of any educational process and it is necessary to assess their performance. It also gets a feedback of teaching process so as to improve the performance of teacher and student both. It is also of paramount importance to evaluate the tool used to assess the students so that the competencies aimed to achieve in the learner, can be measured. To fulfill the above goal, a study was undertaken for 1st M.B.B.S students at Seth G.S.Medical College, Mumbai. Students were exposed to two methods of evaluation viz a viz routine evaluation method in microanatomy (spots with unstructured viva and OSPE-objective structured practical examination).Feedback was taken in the form of a questionnaire and statistical analysis was done. The result shows that objective structured practical examination increases the objectivity and reduces subjectivity compared to conventional viva.

OSPE, Structured viva, Evaluation, Assessment, Microanatomy.

Conventional practical histology examination as per Maharashtra University of Health Sciences is of 10 marks. 6 marks are allotted for spots identification of slides and 4 marks are allotted for viva (slide identification 2 marks and viva 2 marks). The marks achieved totally depends on the slides which do not evaluate the true performance of the student. The subjectivity involved in this examination affects the co-relation between the marks given by the same examiner to the different students on different slides. General performance of the student is evaluated with this type of examination and not the individual competency. There is no objectivity in the conventional examination and so the outcome is not proper. To overcome this, objectivity can be applied to the method of examination to get valid and reliable results.
OSPE is being increasingly used not only in developed countries but also in developing countries like India and Nepal due to its objectivity and reliability [1] . OSPE is an assessment tool in which the competence of the student is evaluated [2] .
In the department of Anatomy of Seth G.S. Medical College, Mumbai the study was undertaken to compare traditional examination and Objective Structured Practical Examination.
The aim of the study was to evaluate attitude of medical students towards OSPE by their feedback and to standardize evaluation tool by comparing two methods.

The study was conducted at Seth G.S. Medical College in the department of Anatomy in the subject of microanatomy which involved 1st M.B.B.S. students. 40 students responded voluntarily to participate in the study. Proper care was taken so that the students did not interact with each other.
Project consists of 2 methods of evaluation for comparison.

This method is the conventional one followed as per Maharashtra University of Health Sciences pattern. Total marks for this method were 10. Students were made to identify 6 spots. One mark was allotted for each spot (Total 6 marks). The time duration given for the identification was one minute per slide. After they finished spots, one slide was given for slide viva. Time duration of 10 minutes was given for identification and drawing (Total 4 marks).

Students were exposed to 5 slides at 5 stations on which structured viva was taken. On each slide same 4 questions were asked having ½ mark each (2 marks per slide). The time duration given for the structured viva was 5 minutes per station. Checklist was given to the examiners. At the station particular structure was asked to identify in one minute. The teacher evaluates the student silently (Total 10 marks).
Comparison of performance of Type I and Type II was done.
Questionnaire was given to the students, data was collected [Table-1] & [Table-2] .
Difficulty and discrimination index of each question asked was calculated.

The results were examined and marks were given on the basis of performance. The following observations were made:
• 29 students out of 40 scored more marks in structured viva. 07 students scored more marks in unstructured viva while 4 students scored same marks in both the methods used [Fig-1] .
• The difference between marks obtained in Type I & Type II method was found statistically significant [Fig-2] , [Table-3] .
• According to 28 students Type II is a better method of evaluation. 4 students felt that Type I is a better method and 12 students were not able to tell which method is better [Fig-3] .
• In the item analysis, 07 questions had difficulty index between 50-70% which is acceptable and 11 questions have between 30-50% & in 2 questions difficulty index was less than 30% which means that the questions were difficult to answer. All 20 questions have discrimination index between 0-1 [Fig-4] , [Fig-5] .

New and interesting ideas are being used in the field of medical education to improve teaching learning methods, assessment & evaluation. Assessment of students being initial step acts as backbone to achieve effective learning. Thus assessment has a significant impact on teaching learning process [3] . Evaluation of the methods is equally important as student’s assessment as we can crosscheck the use of method, its effectiveness & find out areas of weakness or strength. Thus evaluation carries high weightage & importance in such new methods. Evaluation should reflect the justice given to the knowledge of the student. The tool which we use should reduce the subjectivity by increasing its validity, reliability & authenticity. This we can achieve by standardizing the evaluation tool to get effective results.
In conventional practical examination student variability is seen often. Some students have confidence and examination pressure can not affect their performance but for others who have examiner phobia, this may markedly alter their performances. Bias creates gross variations in marking which further depresses studious students. Moreover, marks given by the same examiner for similar competence may vary. Such subjectivity makes evaluation tool unreliable and biased. This does not assess the students properly. In an attempt to overcome problems associated with the traditional examination, Harden et al introduced use of structured examination and found that OSPE is reliable method [4] .
To verify the same, we conducted a study based on OSPE and compared two types of examinations (Traditional Examination with OSPE). The term OSPE is derived from objective structured clinical evaluation (OSCE) in 1975 which was later extended to practical examination & was modified by Harden & Gleeson [4,5] . OSPE is considered as the gold standard of clinical assessment for medical students [6] . OSCE is a tool that evaluates 3rd stage of Miller’s Pyramid (What student is able to do) [3] . In Miller’s Pyramid assessment moves from the know stage via knows how and shows how to the final does level [7] OSPE was largely tested on the basis of the success of OSCE used in medical schools. The experts are now recommending OSPE for both educational and assessment purposes ever for other faculties as well [2] .
During this evaluation students pass through a number of stations and answer the questions and the teacher evaluates the students. Because of objectivity in the method, the reliability, validity and authenticity of the evaluation is increased. The students face same questions which remove the variability of questions asked on different slides. The main benefit of OSPE is that the method of examination and students both are evaluated also large number of students can be assessed in short time.
The questions which were asked to the students were analyzed to find out their difficulty (p) and discrimination (d) index. The difficulty index is determined to find out whether the question is difficult. If the ‘p’ value is between 50-70% it is considered acceptable [8] . In our study 7 questions out of 20 were in the range of 50-70%.11 questions were between 30-50% and the difficulty index in 2 questions was less than 30% which means the questions were difficult. The discrimination index measures the ability of an item to discriminate between the students. If the discrimination index greater than 0.35 then it is considered to have excellent discrimination power, if it is between 0.25 and 0.35 it is considered as good, if between 0.2 and 0.25 then it is acceptable [9] In our study 12 out of 20 questions have discrimination index greater than 0.35.
OSPE provides good integration of teaching and evaluation and also has a good capacity of discriminating between different categories of students [1] . Feedback forms suggested that the students are in favor of OSPE compared to traditional examination. We can judge the students and get the idea for improving their performance by modifying method. Modifications in the method can be done on availability of sources.
Assessment has a big role in the improvement and development of institution [3] . Really speaking because of assessment, teaching and learning process has got its own significance and value. Thus it becomes the foundation for OSPE to make it more interesting and challenging.
The result also showed that OSPE is superior, to routine practical examination. It is reliable assessment tool and it is acceptable and generated wide appreciation. This same study reveals the importance of role of students in developing new assessment tool. OSPE is practical, reliable, valid alternative for conventional examination [9] .
In the present study students preferred OSPE over traditional practical examination. The mean marks in Type II are 7.14 ±1.5 where as in Type are 5.91 ±1.6. This difference was found statistically significant. So OSPE is much better from students view as well as it has proved to be standardized evaluation tool. Following disadvantages of OSPE like preparation for examination is time consuming & requirement of trained and qualified staff members are required to conduct the examination smoothly may affect the expected result. Once we overcome these problems we can conduct ideal examination to achieve effective results. So it should be administered in the university practical examination to get unbiased, authentic and reliable results.

29 students scoring more in Type II indicates that they had in depth knowledge of microanatomy which was tested by four questions asked on 5 slides. These students scoring less marks in Type I method indicates the flaws in the Type I method, as the method failed to test all the aspects of their knowledge of microanatomy.
• 4 students scored approximately double marks as compared to Type I method, which indicated that their performance improved when the method was standardized.
• 7 students scoring more in Type I that Type II implies that when in-depth questions were asked, they were not able to answer.
• As Type II method tested the in-depth knowledge, 28 students felt that it was a better method of evaluation. They felt that this method evaluated their knowledge in an objective manner.

Our sincere thanks to Dr. P.S. Bhuyian, Dr. Avinash Supe for their valuable guidance.

[1] Nayar U., Malik S.L., Bijlani R.L. (1986) Med. Educ., 20, 204-209.  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

[2] Muhammad Abdul Azeem (2007) Pak. J. Physiol., 3(2).  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

[3] Tamara Gamboa-Salcedo, et al (2011) Bol. Med. Hosp. Infant. Mex., 68(3), 169-176.  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

[4] Harden R.M., Gleeson F.A. (1979) ASME Medical Education Booklet, 8(64), 123-5.  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

[5] Harden R.M., Stevenson M., Downie W.W., Wilson G.M. (1975) British Medical Journal, 2, 447-51.  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

[6] Steven Marwaha (2011) B.M.J. Psychiatry, 85, 11-85.  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

[7] Miller G.E. (1990) Acad. Med., 65(9), 63-7.  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

[8] Rege N.N., Bhuiyan P.S., Supe A.N. (2002) The art of Teaching Medical Students, 2nd ed, Medical Education Technology Cell, 239-254.  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

[9] Harden Ronald (1980) Assessment of Practical Skills: The OSPE Routledge, 5, 187-196.  
» CrossRef   » Google Scholar   » PubMed   » DOAJ   » CAS   » Scopus  

	Fig. 1- Comparison between marks obtained in Type I and Type II
	Fig. 2- Mean of Marks in Type I and Type II
	Fig. 3- Response of the students
	Fig. 4- Difficulty index of questions asked
	Fig. 5- Discrimination Index of questions asked
	Table 1- Student’s Response Pertaining to Type I Method 1-Strongly agree, 2- Agree, 3- Undecided, 4- Disagree, 5- Strongly Disagree
	Table 2- Student’s Response Pertaining to Type II Method 1-Strongly agree, 2- Agree, 3- Undecided, 4- Disagree, 5- Strongly Disagree
	Table 3- Mean Marks of Type I and Type II Viva (Mann Whitney U test) (P < 0.05 – Significant) The mean marks in type II is 7.14 +_1.5 where as in Type I 5.91+_1.6 This difference was found statistically significant (P=0.001).