VOLUNTARY SPLIT-SAMPLE PROGRAM FOR ENVIRONMENTAL LABORATORIES IMPROVES BACTERIOLOGICAL DATA (Cont.)
Bias
Some labs seemed to consistently report results that were higher or lower than the group average. A simple ranking procedure was used to verify bias in the data set. For each study, lab data for each bacterial parameter were ranked from highest to lowest with the highest count assigned a ranking of 1.0. When more than one lab reported the same bacterial count, then an average of the ranks (as if the counts had been marginally different) was assigned to each lab. (6)
Lab ranks assigned for all nine studies were plotted (Graph 2. Lab Ranking for C. perfringens, Graph 3. Lab Ranking for Fecal Coliform, and Graph 4. Lab Ranking for Enterococcus). In general, Lab 2 consistently ranked lower than other labs for all three bacterial tests, while Lab 12 consistently ranked higher. Bacterial counts for Lab 2 were biased high, while Lab 12 results were biased low. These two labs were prompted to closely examine procedures for systematic error.
Sample Holding and Data Bias
The compiled data were evaluated to determine if a correlation existed between sample handling conditions and data bias. Low counts due to bacterial die-off would be expected for samples held at higher temperature and with long holding times before analysis. Higher bacterial counts would be expected for samples with the shortest holding times.
Sufficient data was available to produce a series of plots of bacterial counts against sample holding time (but data was insufficient on holding temperature). For a particular study, each enterococci count reported by a lab was divided by the enterococci group average. A ratio greater than 1 was calculated for a bacterial count that was high compared to the average and less than 1 for a low count. The ratios were plotted against the recorded holding time for the corresponding sample. Ratios were similarly plotted for all PE Studies and a trend line (least squared) was inserted. Data for fecal coliform and C. perfringens were handled in a similar manner.
The plot of enterococci data produced a trend line with a negative slope for both enterococci methods. Enterococci counts decreased as samples were held for longer periods of time before processing. This finding could explain why enterococci counts are consistently biased lower for some outer island labs e.g. Lab 12 that routinely receives a sample after a long transport time. Labs near the distribution site e.g. Lab 2 have the shortest holding time with results that are biased high. The rather poor correlation as indicated by low coefficient of determination values (r2 = 0.09 and 0.17) may be attributable, at least partly, to large variability between labs in sample handling conditions and efficiency of bacterial recovery. Refer to Graph 7 Effect of Sample Holding Time on Recovery of Enterococcus Method 1106.1, and Graph 8 Effect of Sample Holding Time on Recovery of Enterococcus Method 1600.
In contrast, fecal coliform and C. perfringens counts were minimally affected by sample holding time. The trend lines were nearly flat indicating no correlation between bacterial counts and sample holding time (r2=0.01 and 0.001). Fecal coliform counts did not decrease in samples held for up to 8 hours before processing; this is an unexpected finding since enterococci in the same sample was diminished. C. perfringens counts were only slightly lower when samples were analyzed nearly 8 hours after the time of collection. Under aerobic conditions, only spores of C. perfringens are recovered that are highly resistant to injury and would not be affected by long holding times. High and low bias for labs reporting fecal coliform and C. perfringens is not correlated to sample holding time and is probably due to other factors. Refer to Graph 5 Effect of Sample Holding Time on Recovery of C. perfringens, and Graph 6 Effect of Sample Holding Time on Recovery of Fecal Coliform,
CONCLUSION
This self-help program has enhanced the quality of bacteriological data and opened lines of communication between participating laboratories. The information from each study provided feedback, permitting laboratories to make adjustments and improvements to analytical processes. Laboratory personnel were key to the improvement process as many were willing to seek advice and training from experienced microbiologists working in other labs participating in the program. Over the course of nine performance evaluation studies, overall quality of enterococci, fecal coliform and C. perfringens data from all labs has measurably improved.
(6) Wernimont, G.T., Use of Statistics to Develop and Evaluate Analytical Methods, Association of Official Analytical Chemists, 1987.
