Improving the Credibility of QMS Certification QCI conducted a path-breaking study on QMS certification (May 2005-July 2007). The last of this two-year study was conducted through May-July 2007. The results of the study were widely reported in international journals and the QCI was appreciated for its role in taking up the first such study of its kind in the country. QCI Secretary General Girdhar J Gyani discusses the survey and its findings. Introduction The purpose of the ISO 9001:2000 standard is to provide a means to ensure that suppliers provide products and services that satisfy specified requirements. The certification bodies issue certificates after physically verifying that the organization (supplier of products) has complied with the requirements of ISO 9001:2000. The competency of the certification bodies is ensured through the process of accreditation wherein a designated accreditation body grants accreditation to the certification body based on ISO Guide 62, which has now been revised to ISO 17021. By this process of accreditation, the products of certified organizations get global acceptance. The International Accreditation Forum (IAF), which is the association of accreditation bodies, has established a mechanism of mutual recognition of accredited certification world over. This, in a way, acts as a guarantee for the ultimate consumer that products from certified organization comply with the specified requirements. This guarantee, of late, has eroded the confidence of ultimate users and hence there is a need to reexamine the process of certification. By way of physical assessment of 105 certified organiza-tions, the analysis of data reveals that there is a lack of consistency in the certification process between different certification bodies, in spite of the fact that they were accredited and were expected to follow uniform practices. BY WAY OF PHYSICAL ASSESSMENT OF 105 CERTIFIED ORGANIZATIONS, ANALYSIS REVEALS THAT THERE IS LACK OF CONSISTENCY IN CERTIFICATION PROCESS BETWEEN CERTIFICATION BODIES, IN SPITE OF THE FACT THAT THEY WERE ACCREDITED. The objective of study There have been concerns, more so in the developing countries, that products from ISO 9001:2000 certified organizations were not conforming to the specified requirements. A study carried out by AC Nielsen ORG-MARG on the effectiveness of ISO 9000 Table 1: Certification Bodies in Homogenous Groups Group First Group Second Group Third Group Fourth Group Fifth Group Range of Average Response 3.89 3.53-3.34 3.25 - 3.05 2.97- 2.84 2.44 -2.25 Certification Bodies  CB -1  CB-2  CB-3  CB-4  CB-5  CB-6  CB-7  CB-8  CB-9  CB-10  CB-11 certification in India in 2005, brings out that ISO 9000 certification did not result in the improvement in general about the quality of the final products. The study covered six corporate clients with respect to quality of products received from their 778 certified vendors. The study has shown that in 84 per cent cases of vendors there was no change or improvement in the quality of products after the vendors were certified to ISO 9001:2000 standard. The present study was undertaken by way of actual physical verification to: (a) verify whether certification process has been carried out as per applicable standards and guidelines, and, (b) measure the effectiveness of the certification process by assessing the status of QMS within the certified organizations. The other objective of thestudy was to provide a feedback to the Certification Bodies on areas for improvement so as to enhance the effectiveness of the certification process. The methodology The details about the number of certificates issued during July 2006 to September 2006 by various certification bodies were called for. A total of 11 CBs responded by providing necessary information. Of the 1464 organizations certified during this period, 105 organizations were picked up on a random basis. To measure the effectiveness of QMS Status, a checklist with 15 parameters — largely based on the ISO 9001:2000 requirements — was prepared, and data collected to ascertain the extent to which QMS complied with the requirements of Standard. This has been referred as QMS Status. Findings and analysis Effectiveness of QMS status has been evaluated through compilation and evaluation of data. Performance evaluation with respect to different parameters of QMS status has been tabulated by taking the statistical average. Variation or dispersion in performance is measured by variance or standard deviation. Lower value indicates lower variation which means results are more consistent. Response data gathered through field study for different certification bodies ( 11 in number) were analyzed for mean (average) and variance (dispersion) separately for QMS status (15 characteristics). Performance evaluation of QMS status Average analysis Performance of certification bodies vary widely from 2.25 to 3.89 and it has been presented as Bar Chart in Figure 1. Test of significance has been used to find out the homogenous groups in the case of certification bodies. Test of significance indicated that the Certification Bodies can be grouped into five distinct groups. Details are given in Table 1. THE PRESENT STUDY WAS UNDERTAKEN BY WAY OF ACTUAL PHYSICAL VERIFICATION TO : (A) VERIFY WHETHER CERTIFICATION PROCESS HAS BEEN CARRIED OUT AS PER APPLICABLE STANDARDS AND GUIDELINES, AND (B) MEASURE THE EFFECTIVENESS OF CERTIFICATION PROCESS BY ASSESSING THE STATUS OF QMS WITHIN THE CERTIFIED ORGANIZATIONS. OTHER OBJECTIVE OF THE STUDY WAS TO PROVIDE FEEDBACK TO CERTIFICATION BODIES ON AREAS FOR IMPROVEMENTS SO AS TO ENHANCE EFFECTIVENESS. Performance evaluation combining mean and variance Performance of certification bodies were evaluated for average and variance separately. Some certification bodies' averages were high but variances were not small and similarly there were certification bodies with low average and low variance. These were not very good performance levels. Good performance was one where average performance was high and variability was small. This aspect could be examined easily by the quantity known as coefficient of variation where ratio of standard deviation to mean was considered. Comparison of certification bodies with respect to mean, variance, and coefficient of variance (cv) is given in Table 2. Performance of CB-1 and CB-2 were best among various certification bodies. They had a high average and low variability which was also seen from cv value (0.21 and 0.22). Next best were CB-4, CB-3 & CB-6. For all the remaining CBs there was need to improve the average performance and remain consistent in the high level of performance. Learning from the study i) The effectiveness index computed for QMS Status (which reflect QMS compliance on ground) in respect of 11- CBs, varies from 2.25 to 3.89 on a scale of 5.0. Consideringthat all CBs were accredited and were supposed to be complying with ISO Guide - 62/ ISO 17021 (Revised Standard), such a wide variation reflected poorly on their performance. ii) The behaviour of CBs had been corroborated by applying test of homogeneity. Here 11-CBs fell into 5 non-homogenous groups. This indicated that CBs had wide inconsistency between them in respect of grant of certification to the organizations. PERFORMANCE OF CERTIFICATION BODIES WERE EVALUATED FOR AVERAGE AND VARIANCE SEPARATELY. SOME CERTIFICATION BODIES' AVERAGES WERE HIGH BUT VARIANCES WERE NOT SMALL. iii) There were large variations in the practices adapted, even within the same CB. For example, based on the data collected from sample number of organizations certified by same CB, the variance ranged from 0.61 one CB to 1.57 to another CB. This showed that CBs did not have adequate control on their internal processes. Table 2: Comparative performance of certification bodies Certification Body Average Variance Coefficient of Variation CB-1 3.89 0.65 0.21 CB-2 3.53 0.61 0.22 CB-3 3.44 0.91 0.28 CB-4 3.34 0.85 0.28 CB-5 3.05 0.77 0.35 CB-6 3.25 1.33 0.27 CB-7 3.17 1.16 0.36 CB-8 2.97 0.79 0.30 CB-9 2.84 1.00 0.35 CB-10 2.44 1.09 0.43 CB-11 2.25 1.57 0.56 iv) The study had combined effectiveness index with variance computed coefficient of variation, which was a fair representation of rating of CBs. This coefficient once again ranged from 0.21 for best CB of lot to 0.56 being worst CB of the lot. v) The large variation in the QMS Status undermined the effectiveness of certification and even credibility of ISO 9001:2000. To improve the effectiveness in certification process, the accreditation bodies may regularly carry out such validation surveys on certified organizations, which have been certified by the accredited CBs. By sharing of the outcome with CBs, the effectiveness could be improved to a large extent, bringing all-round benefit to the ultimate consumer who is user of product/ services from certified organizations. Conformity Assessment in Quality Governance | Improving the Credibility of QMS Certification | Quest for safe kitchens | We are taking one step at a time | Get well soon – in India | Railways chug to Quality mode | Lead is hampering the Quality of our lives | Technical Education | QCI initiates “Practical Benchmarking” | content

SOURCE