Drop Date: April 2019

In This Drop:
Rigor And Reproducibility (Part 1 Supplement): What Makes A Saliva Lab High-Quality?

Rigor and Reproducibility is a three-part series on the steps researchers can take to improve the quality of their salivary bioscience data.

High Quality Labs: High Quality Results

In Part 1, How Good Are Your Lab Results? of our Rigor and Reproducibility series, we shared how controlling for sources of variability leads to higher data consistency, which is essential in establishing precise results and drawing reliable conclusions from generated data. Part 1 specifically examined the findings from the 2017 study: “Measurement of cortisol in saliva: a comparison of measurement error within and between international academic-research laboratories”, from Calvi et al. The study results were impressive – in high-quality labs, the average intra-assay (same lab, same plate) CV was only 6.2%; the average inter-assay (same lab, different plate) CV was only 6.36%; the difference attributed to the labs was only 7.93%; and just 1.76% of the difference could be attributed to the Salimetrics Cortisol Assay. This is a highly relevant conclusion: high-quality laboratories combined with better assays produce better results that are reliable and precise for the scientific community.

Part 1, was well received by the salivary bioscience research community (as was Part 2, How Good are Your Saliva Samples?), with the overwhelming follow up question being: “What makes a saliva lab high-quality?” This Salivary Bioscience Bulletin, which we are calling “Part 1 Supplement”, is dedicated to providing key attributes of a high-quality salivary bioscience laboratory and why they are important, so that researchers can make the best choice and have the best chance at getting consistent, high-quality salivary bioscience data.

7 Attributes Of A High-Quality Salivary Bioscience Laboratory

#1   Performs routine calibration of lab equipment (Rigor)

Making sure lab equipment is calibrated is the same as making sure a car is in alignment. Over time, or as a result of being bumped or mishandled, equipment measurements can drift in one direction and lead to bias in the results generated. High-quality labs avoid this type of systematic error by having all their lab equipment in a scheduled, routine calibration program. A high-quality salivary bioscience lab will calibrate equipment such as pipettes every six months and monitor hard working equipment such as plate readers on a monthly basis. Calibration of lab equipment allows for reproducible measurements; conversely, uncalibrated equipment is one of the principle sources of lab error and almost always guarantees that the same samples tested across different labs will have a high inter-lab assay variability.

#2   Follows rigorous quality acceptance criteria for assay results (Rigor)

While assay manufacturers differ in their acceptance criteria, a good assay manufacturer will provide assays that meet rigorous quality acceptance criteria, and then provide protocols and guidance that ensures each laboratory can achieve great results. It’s a simple fact: the tighter the acceptance criteria, the less variability in results; the wider the acceptance criteria, the greater the variability in results. Salimetrics’ assays are manufactured and quality tested to rigorous acceptance criteria; any high-quality laboratory using Salimetrics’ assays should achieve the following quality acceptance criteria (don’t worry if you do not understand these terms – just ask Salimetrics for help. If the lab doesn’t understand these requirements, find another lab.):

• The standard curve, using all standards, shall have an R² curve fit ≥ 0.99
• For multiple plate runs, the B/B₀ standard curves shall overlay
• The high and low control concentrations shall each be within the expected range provided with the assay batch documentation
• For multiple plate runs from the same batch, the high and low control concentrations shall not deviate across all plates by more than 10%
• Quality Acceptance Data shall be available to be provided with results

Acceptance criteria are like guide posts; they let you know you are going in the right direction and can trust the results. High-quality labs understand the importance of meeting all acceptance criteria; they do not follow some and overlook adherence to others in order to provide results – results that will likely contain errors. For example, a less qualified lab may check that the controls are within the expected range but may not check the R² curve fit. Accuracy of the curve fit confirms that sample concentrations across the range of the standards are reading with precision. If a researcher is looking for pre and post intervention data, a good curve fit means the difference between the pre and post samples will be precise. Alternatively, if a less qualified lab checks that the R² curve fit is ≥0.99, but the control values are out of range, the results will likely have a bias. In other words, while the difference between two samples is precise, the actual value of each of the samples may be too high or too low depending on the bias, and therefore not true. High-quality labs understand these requirements, consistently meet the criteria, and make their Quality Acceptance Data available so that you can trust the results.

#3   Follows a documented CV% acceptance/repeat criteria (Rigor)

High-quality labs can control for many possible sources of lab error, but saliva samples are unique in that they may be viscous and heterogenous, and, if the saliva samples are not processed by the lab correctly, duplicate measurements of the same sample can have large deviations between the two measures. Consider duplicate measurements of the same sample having a CV% of 20%; this means that the two measurements are actually 40% apart, and the mean value is 20% apart from each of the two values. More often than not, only one of these measures is incorrect, so choosing the mean value suggests that the value intrinsically has a 20% error. Further, if the average CV% of the entire data set is 20%, then any conclusions made by the researcher would be unsupported if the conclusion did not rely on a >20% difference between the means.

In addition to following good protocols, a high-quality lab will avoid sample processing and handling errors by having well-documented criteria that only permits the acceptance of the duplicate mean based on a maximum allowable percentage of coefficient of variation between duplicate values, or when the absolute difference of the duplicate values is small and does not have biological significance. When the criteria are not met on a sample, the sample is rerun or excluded from the final data analysis. In addition, for quality control, the lab should record the sample as having been repeated.

#4   Allows only laboratory personnel trained in salivary testing to run the assays (Rigor)

High-quality labs know there are subtleties to lab work that are minimized through proper training and experience. Understanding how to work with viscous samples to avoid the introduction of bubbles and tiny volume discrepancies when pipetting, how to prevent bias across the plate, and even how to read and interpret the aforementioned quality control data, can make a difference in the integrity of the results. As with many things, it is not difficult; however, it requires the rigor which comes from good training. Both inter- and intra-assay CVs are improved when adequately trained personnel run assays. A high-quality lab will incorporate exercises such as pipetting proficiency into the training of personnel, and document the results, before actual assays are run.

#5   Routinely processes saliva samples and performs saliva testing (Repeatability)

Practice makes perfect. A proven lab has more expertise and experience at running assays compared to labs that run assays infrequently; that’s why all Salimetrics’ certified Center of Excellence labs must run a minimum of 5,000 salivary tests per year. It’s not that most labs are unable to get acceptable results from some of their salivary testing. It’s that most labs do not know how to get the best results from all of their saliva testing, and that’s simply because they have not tackled the full breadth of issues a proven, high-quality lab has experienced. A high-quality lab knows how to troubleshoot and determine where potential issues may lie, and how to correct to get the best results. For example, most studies have samples which vary in terms of volume, clarity and viscosity. A high-quality lab has the experience and know-how to get the best data from a low-volume, viscous sample and that means errors are minimized, or as is often the case, a high-quality lab will be able to test the sample and get results whereas the less qualified lab may not be able to test the sample, and that may lower the power of a researcher’s study.

#6   Participates in a proficiency testing program and sample exchanges (Repeatability)

Participating in a proficiency testing program is like getting a graded report card for the lab. It’s conducted after the lab meets all of the basic requirements for rigor, including training, and thereafter at minimum on an annual basis. For diagnostic labs, it is a regulatory requirement that they demonstrate proficiency in sample handling and processing by providing sample measurements on unknown, “controlled” samples. Once values are reported, the lab gets a proficiency report for each of the analytes tested and is deemed proficient to provide testing services if their results are within the expected range for the “controlled” sample. While this might not seem relevant to research, it is! A key finding of the Calvi study was that a researcher can trust that the difference in results between high-quality labs running the same assays will be small and the results repeatable. Participation in a proficiency testing program is preferred, but it is also possible to demonstrate proficiency through sample exchanges with another high-quality lab. In this instance, aliquots of samples are sent to an independent, qualified lab that is able to run the samples on the same kit/methodology. The results are demonstrated in the same way they would be demonstrated in a proficiency testing program.

#7   Uses and recommends validated saliva collection methods (Repeatability)

High-quality labs know that how a sample is collected and handled prior to testing can significantly impact the measurements obtained from that sample. They understand that data from poorly handled samples will likely harm the data set when developing scientific associations and conclusions. For example, not correcting for flow rate when measuring an analyte that is impacted by flow, using an incorrect method to stimulate flow, using samples subjected to multiple freeze-thaws, or using an unvalidated swab for the collection of saliva without fully understanding the bias/interference created by the swab material (See Part 2, How Good are Your Saliva Samples?) are all errors that may affect scientific conclusions. Conversely, following validated methods reduces variability: from collection; from transport; from storage; from freezing; from the bench – all of which increase the repeatability of the results. A high-quality lab knows what the validated saliva collection methods are, and like Salimetrics, they know how to make the best saliva collection recommendation for a study. These labs keep up with evolving best practices and are a great resource for newer researchers.

Summary

The high-quality Salimetrics SalivaLab has been an integral part of Salimetrics since the company was founded and has tested millions of saliva samples. While testing with the Salimetrics SalivaLab is always a great option for a researcher, we do recognize that at times researchers may need to select a different lab. To help make the selection of a high-quality lab better for researchers, Salimetrics developed its Center of Excellence (COE) Program which identifies and certifies high-quality saliva labs. Inclusion in the COE program is optional, and by no means does the lack of inclusion in the COE program signify that the non-participating lab is unable to achieve high-quality results. However, a researcher can be assured that in addition to the Salimetrics SalivaLab, all labs in the Salimetrics COE program meet the attributes of a high-quality lab. For researchers who intend to run assays in their own lab, following the key attributes we present here will provide the best chance of assuring that their data will be precise.

We conclude with a simple statement: salivary bioscience research is stronger when it is dedicated to the methods that bring rigor and reproducibility.