In addition, the ScoreCard software calculates quality measurements that enable the user to validate the performance of the overall microarray system. Such measurements include variation of signal for replicate spots from pen to pen (% pen variation) and from spot set to spot set within a slide (% spot set variation). These measurements are displayed upon the user’s request at the System Validation window shown in Figure 3. Quality measurements with values outside thresholds are displayed in red.
Fig 3. ScoreCard GUI: System Validation window.
Lucidea Microarray ScoreCard normalization method
The primary challenge in microarray data analysis is to determine which genes are differentially expressed and by how much. This is complicated by the fact that there are both systematic and random errors associated with microarray measurements that cause signal variation from slide to slide. Such variations make comparison of differential expression ratios within and across experiments almost impossible.
Systematic errors that can bias microarray measurements involve characteristics of the fluorescent nucleotides used in labelling. The reverse transcriptases used do not incorporate Cy3 and Cy5 fluorescent nucleotides with the same efficiency. Furthermore, Cy3 and Cy5 each have distinct fluorescence efficiencies and detection characteristics, so that observed signal intensities are not uniform between the same quantity of Cy3 labelled probes relative to Cy5 labelled probes. Due to these sorts of errors and variations, the raw ratios do not cluster around a value of one and are not distributed normally. In addition, ratios appear to be less accurate and less precise at signal values approaching the detection limits of the system. These observations suggest that a new normalization procedure is required to transform raw signals or ratios into normalized ratios, so that a ratio of one correctly represents non-differentially expressed genes.
Normalization is traditionally conducted within a channel, using such correction factors as the average or median signal for all spots on an array, or the mean signal for housekeeping genes or positive controls. Such methods assume that normalization is constant. However, experiments in which a single mRNA sample is split and labelled with the two different dyes (so all genes should show no differential expression) still show distorted ratios. Furthermore, such experiments suggest that the differential expression ratios vary with Cy5 signal, and the amount of distortion in ratio varies from slide to slide. An exponential curve adequately describes the relationship between the log ratios and the Cy5 signal. As a result, a non-constant normalization method that takes this into account is the most effective method for correcting the raw ratios.
In view of the above observations and extensive experimentation, Lucidea Microarray ScoreCard software utilizes a proprietary normalization method based on a regression analysis that includes data from all the spots on the slide. The normalization in Lucidea Microarray ScoreCard is a two-colour procedure that corrects for the following artefacts:
1. A difference in ratio caused by systematically reduced signal in one channel relative to the other.
2. A distortion in the average ratio at low signal intensities, an effect whose severity varies from experiment to experiment.
This normalization method has been tested and proven to work well with different slide types and spotting chemistries. Experimental data suggest that it improves both the accuracy and the precision of the microarray measurements (Fig 4 and Fig 5).
Fig 4. Comparison of the dynamic range control log ratios before and after normalization. Note the arrows indicating the zero position; a log ratio of 0 corresponds to a linear ratio of 1. All the dynamic range ratios are ~ 0 after normalization, as expected, indicating that the normalization procedure improved the accuracy of the log ratios. ULR = uncorrected log ratios; NLR = normalized log ratios.
Fig 5. Precision of dynamic range control log ratios before and after normalization. The standard deviation for the uncorrected log ratios (plotted in red) of the dynamic range controls (pg mRNA in spike) is compared with the corresponding normalized values (plotted in blue). After normalization, both the absolute standard deviation values and their confidence intervals (error bars) are decreased, indicating that the normalization procedure improved the precision of the log ratios. SDULR = standard deviation for the uncorrected log ratios; SDNLR = standard deviation for the normalized log ratios.
The ScoreCard software reports both normalized and observed ratios for the dynamic range and ratio controls, allowing easy comparison of the normalized values with the target values. These comparisons enable the user to evaluate if the normalization procedure improves the accuracy of the data.
Interpreting data quality using Lucidea Microarray ScoreCard
We have used Lucidea Microarray ScoreCard to evaluate the quality of various microarray experiments. In some cases, we have deliberately manipulated the experimental design to determine if ScoreCard would indicate the problem, and if the output values could help the investigator in troubleshooting the problem. In Figure 6, we have compared the normalized ratios calculated by ScoreCard for a typical good-quality experiment with those from an experiment with unusually high Cy3 background. In the latter experiment, the ratio values for the dynamic and ratio controls are outside the threshold range (1.5 fold difference from the target values) even after normalization. As a result, they are highlighted in red, indicating that the experiment should be rejected as invalid.
Fig 6. Comparison of the normalized ratios calculated by ScoreCard for a typical good-quality hybridization (panel A) and from a hybridization with high Cy3 background (panel B). Note the values for the normalized log ratios in the tables below each image. For experiment A, all normalized ratios are very close to the expected value (within the thresholds); however, in experiment B, all ratios are outside the thresholds and therefore highlighted in red.
In Figure 7, we have compared various ScoreCard quality measures from a successful hybridization (panel A) with those from a hybridization with poor hybridization uniformity (panel B). The ScoreCard measurements indicating spot-to-spot reproducibility and pen-to-pen variation are clearly outside the acceptable range and consequently highlighted in red.
Fig 7. Comparison of ScoreCard quality measures from a successful hybridization (panel A) and from a hybridization with poor hybridization uniformity (panel B). The housekeeping gene performance and system validation quality measurements are shown. All measurements with values outside the appropriate thresholds are highlighted in red.
In Figure 8, we have presented ScoreCard data from an experiment in which we evaluated the effects of probe concentration on hybridization. In this experiment, we compared hybridizations using a typical probe consisting of 25 pmol of dye equivalents with hybridizations using probes of only 3 pmol of dye equivalents. For the latter hybridizations, the software detection limits measurements are outside the acceptable values and highlighted in red. This indicates that with probe dilution, the sensitivity of detection and the dynamic range of the system decrease.
Fig 8. Effect of probe concentration on detection limit measurements. Both the housekeeping gene ratio and the dynamic range measurements have values outside the acceptable range (highlighted in red) when the hybridizing probe includes only 3 pmol of dye equivalents.
The above experimental cases demonstrate that Lucidea Microarray ScoreCard can be successfully used to identify a variety of problems in experimental data.
Conclusion
Lucidea Microarray ScoreCard system is the first integrated analysis tool for evaluating data quality across microarray experiments. With Lucidea Microarray ScoreCard, the user can compare and troubleshoot microarray experiments and assess the performance of the microarray system by including predefined genetic targets and spikes on individual slides.
By combining a system of control elements with analysis software, Lucidea Microarray ScoreCard quickly and easily provides the user with:
• QC for all aspects of hybridization, including probe labelling, slide quality, and system performance
• a guide for ratio and detection limit analyses and dynamic range evaluation
• a standardized report for each experiment
• data normalization before summarization and mining
• data preparation for visualization and mining
ScoreCard can identify various problems with data quality, some of which are not easily detected by simple visual examination of microarray images. Consequently, it can improve and streamline the process of microarray data analysis by allowing only normalized data of acceptable quality to proceed from data extraction to data visualization and mining.
References
1. Bowtell, D. D. L., Nature Genetics 21, 25-32 (1999). [PubMed abstract]
2. Brown, P. O. and D. Botstein, Nature Genetics 21, 33-37 (1999). [PubMed abstract]
3. Barker, D. et al., Systems Approach to Fabricating and Analyzing DNA Microarrays. In Microarray Biochip Technology, published by BioTechniques (Schena, M., volume ed., ISBN 1-881299-37-6), Natick, MA, pp. 65-86 (2000). | 
