To demonstrate that Invitae's next-generation sequencing (NGS) analysis provides the high-quality results you are accustomed to, Invitae has validated our analytic results and clinical interpretations.
A systematic comparison of traditional and multi-gene panel testing for hereditary breast and ovarian cancer genes in more than 1000 patients
A study comparing Invitae’s panel test to traditional BRCA1 and BRCA2 tests in more than 1000 patients was undertaken in collaboration with the Stanford University School of Medicine and Massachusetts General Hospital. The study demonstrated 100% analytic sensitivity and specificity for Invitae’s panel compared to traditional genetic test results for both sequence alterations and deletions/duplications. Variant classifications were also highly (99.8%) concordant.
Multi-gene panels for hereditary breast and ovarian cancer risk assessment are gaining acceptance, not only as additions to but also as replacements for traditional BRCA1/2 testing. To help determine which tests are appropriate for any given patient, it is important to understand the analytic and clinical performance of these tests by comparison with traditional testing.
A total of 1105 individuals were tested using an Invitae 29-gene hereditary cancer panel. Sequence alterations and copy number deletions/duplications were determined by next-generation sequencing (NGS) using Invitae’s custom biochemical and bioinformatics methodologies. For these 1105 individuals, high-quality reference and confirmatory data were available for direct comparison. Variants were classified using a framework (Sherloc) based on the American College of Medical Genetics and Genomics 2015 guidelines using only publicly available and not proprietary data resources. Classifications were compared for 975 individuals for whom traditional BRCA1/2 test results from Myriad Genetics were available.
Table 1: Analytic concordance
Figure 1: Types of pathogenic variants observed
Table 2: Interpretation concordance for BRCA1/2
Invitae’s NGS panel test can provide analytic and clinical results highly comparable to those of traditional BRCA1/2 testing. For both sequence and deletion/duplication variants across many genes, 100% sensitivity and specificity was observed, as well as high interpretation concordance (99.8%). Panel tests can also uncover potentially actionable findings that may be otherwise missed. A detailed study of the clinical actionability of non-BRCA1/2 variants observed in these and other patients is reported separately.
This study is published in the Journal of Molecular Diagnostics, the official journal of the Association for Molecular Pathology.
Stephen E Lincoln, Yuya Kobayashi, Michael J Anderson, Shan Yang, Andrea J Desmond, Meredith A Mills, Geoffrey B Nilsen, Kevin B Jacobs, Federico A Monzon, Allison W Kurian, James M Ford, Leif W Ellisen, A systematic comparison of traditional and multi-gene panel testing for hereditary breast and ovarian cancer genes in more than 1000 patients. J Mol Diagn. 2015.
Download the Invitae hereditary cancer analytic validation one-page PDF of this information.
Current clinical approach
For decades, Sanger sequencing has been accepted by clinicians as a gold-standard in genetic testing. However, its low throughput is not ideal for assessing the growing number of genes in a modern clinical gene panel. When clinical testing began to employ next-generation sequencing (NGS), a two-step approach was adopted whereby many genes were assayed by NGS sequencing and reportable variants were verified by Sanger sequencing. This continues to be the standard as set out by the ACMG guidelines: “... it is recommended that all disease-focused and/or diagnostic testing include confirmation of the final result using a companion technology.”1
It has since been reported that Sanger sequencing may introduce more errors than it actually prevents, and may be unnecessary for high quality variants.2,3,4 Here we examine the results of this two-step confirmatory approach as it has been implemented in our laboratory.
Quality in confirmation
For reasons both of efficiency and accuracy, Invitae this year validated Pacific Biosciences (PacBio) sequencing technology for clinical use. It is an alternate confirmatory sequencing method that, in our hands, provides an accurate and higher throughput method that is orthogonal to NGS sequencing. PacBio provides a more quantitative measurement of variants and additional QC metrics compared to Sanger sequencing. We continue to use both PacBio and Sanger to confirm variants.
Confident NGS variant calls are highly specific
Using this hybrid method of NGS followed by either Sanger or PacBio to deliver tens of thousands of clinical reports, Invitae has built a large dataset of variants detected by all three technologies. This has allowed us to complete a direct comparison of NGS with the two orthogonal technologies (see table below). Standard orthogonal confirmation assays rely on PCR-based targeting of the variant loci, which is susceptible to allele dropout and amplification failure. To guard against a false negative result due to these types of failures, we run multiple overlapping assays to redundantly target each variant.
In a cohort of approximately 70,000 individuals who have undergone genetic testing at Invitae, we identified nearly 6,800 high quality variant calls that because of our reporting policies required orthogonal confirmation. Our data show that among 6,788 variants (57% SNVs and 43% indels) detected with high confidence according to our QC parameters, 6,752 (99.5%) were confirmed by Sanger or PacBio sequencing. We have never observed a discordant result during the orthogonal confirmation process of high confidence variant calls. For the remaining 36 (0.5%) high confidence variants detected by NGS, the orthogonal assay generated no useable data after several attempts and the variant was reported with a corresponding limitation regarding the failure to confirm according to ACMG guidelines. These events tend to be in regions known to be difficult for these assays, such as areas of high GC content or within repetitive sequences.
Table: Next-generation sequencing and orthogonal methods show perfect concordance for high quality variant calls
False positive rate and sensitivity
In order to maintain high sensitivity, a wide net must be cast when initially calling variants. Our validated, custom computational pipeline identifies variants and assesses their quality based on a large number of sequencing attributes (including strand bias, read depth, Phred-scaled p-value, and many others). These quality attributes are used to separate variants into those that are high-quality and others that may be false positives and are “flagged” for further evaluation and mandatory confirmatory testing.
In the same testing period in which the 6,788 high quality variants were found, 508 low-quality, flagged variants were also detected and, among these, 411 (81%) confirmed as true positives and 97 (19%) were found to be false positives by an orthogonal method. Thus, out of 7,296 total variants, 1.3% (97 of 7,296) were NGS false positives that were correctly identified during confirmatory testing.
Our data show that confirmatory sequencing has not improved the accuracy of the final report for high quality NGS variant calls. It does however continue to be important when the quality metrics for the variant do not exceed empirically determined thresholds set in our lab.
N.B.: Although it is outside the scope of this white paper, note that Invitae confirms any reported CNV event and has performed more than 1,000 such confirmations.
Next-generation sequencing, as implemented at Invitae, is a high-quality, clinical grade technology. Our team understands that the stakes for clinical genetic testing are high. Results can lead to irreversible action and emotional distress for both the patient and their family. We are committed to maintaining the highest quality, while continually improving our processes in a responsible and data driven manner.
1. Rehm et al. ACMG clinical laboratory standards for next-generation sequencing. Genet. Med. 2013
2. Beck et al. Systematic evaluation of Sanger validation of next-generation sequencing variants. Clin. Chem. 2016
3. Baudhuin et al. Confirming variants in next-generation sequencing panel testing by Sanger sequencing. JMD 2015
4. Mu et al. Sanger confirmation is required to achieve optimal sensitivity and specificity in next-generation sequencing panel testing. JMD 2016
Download the Invitae confirmation for clinical genetic testing one-page PDF of this white paper.
Lynch syndrome, also known as hereditary non-polyposis colorectal cancer (HNPCC), is characterized by familial predisposition to cancers of the colon, endometrium, ovary, stomach, and urinary tract.1 Most cases of Lynch syndrome are caused by variants in MLH1, MSH2, and MSH6, but 4–11 percent of cases are caused by variants in PMS2.2-4
Testing for inherited variants in PMS2 is hampered by the presence of a pseudogene, PMS2CL, which has nearly identical homology to PMS2 in the final four exons of the gene (exons 12–15). Thus, sequence reads derived from hybridization capture in next-generation sequencing (NGS) methods cannot be unambiguously aligned to PMS2 or PMS2CL. Gene conversion between exons 12 and 15 of PMS2 and PMS2CL further complicates this issue.5
Why develop an NGS method?
Invitae is committed to making high-quality genetic testing affordable and accessible. Most laboratories perform multiplex ligation-dependent probe amplification (MLPA) to identify deletion/duplication variants, and use long-range PCR (LR-PCR) before sequencing to identify read-through variants and avoid interference from the PMS2CL pseudogene. This is a highly customized and resource-intensive approach to the analysis of a single gene in every sample. Having developed an approach that maximizes the use of our established workflows and capabilities, we are able to offer sequencing of this difficult but important region of PMS2 while maintaining our commitment to affordability.
Invitae's approach to PMS2
Invitae’s approach to the evaluation of exons 12–15 of PMS2 is a two-step process for read-through variants and a three-step process for deletions and duplications (Figure 1). The first step for both types of variants is a bioinformatics screen in which sequence reads derived from both PMS2 and the paralogous PMS2CL gene are analyzed for the presence of variants using PMS2 as the reference sequence. For read-through variants, non-benign variants identified in the screen are definitively assigned to PMS2 or PMS2CL using Sanger sequencing of LR-PCR products of PMS2 (exons 12–15) and PMS2CL (exons 3–6). For deletion/duplication variants, the second step is to confirm the bioinformatics screen call with MLPA, and to account for the possibility of gene conversion, a final step with LR-PCR is used to disambiguate the location of the variant.6
This approach was validated with samples known to have specific variants in these exons for both genes (reference set). For validation of the read-through method, we analyzed 32 unique samples carrying 205 true positive and 34,876 true negative variants in PMS2 or PMS2CL and demonstrated an accuracy, reproducibility, and analytical sensitivity and specificity of 100% (Table 1). For validation of the deletion/duplication method, we analyzed 28 unique samples carrying 90 true positive and 50 true negative individual exon variants in PMS2 or PMS2CL and demonstrated an accuracy, reproducibility, and analytical sensitivity and specificity of 100% (Table 2).
1. Lynch, HT, et al. Review of the Lynch syndrome: history, molecular genetics, screening, differential diagnosis, and medicolegal ramifications. Clinical Genetics. 2009; 76(1):1-18. PMID: 19659756
2. Gill, S, et al. Isolated loss of PMS2 expression in colorectal cancers: frequency, patient age, and familial aggregation. Clinical Cancer Research. 2005; 11:6466-6471. PMID: 16166421
3. Halvarsson, B, et al. The added value of PMS2 immunostaining in the diagnosis of hereditary nonpolyposis colorectal cancer. Familial Cancer. 2006; 5:353-358. PMID: 16817031
4. Truninger, K, et al. Immunohistochemical analysis reveals high frequency of PMS2 defects in colorectal cancer. Gastroenterology. 2005;128:1160-1171. PMID: 15887099
5. Hayward, BE, et al. Extensive gene conversion at the PMS2 DNA mismatch repair locus. Human Mutation. 2007; 28(5):424-30. PMID: 17253626
6. Vaughn CP, et al. Avoidance of pseudogene interference in the detection of 3’ deletions in PMS2. Human Mutation. 2011; 32(9):1063-71. PMID: 21618646
To learn more, please read our PMS2 sequencing and deletion/duplication validation statement.
To demonstrate the value of multi-gene panels in hereditary cancer risk assessment, Invitae collaborated with Stanford University researchers, James Ford, M.D. and Allison W. Kurian, MD, MSc. The results of this research, recently published in the Journal of Clinical Oncology, show that that multi-gene hereditary cancer panels can offer comparable performance to traditional BRCA1/2 genetic testing and can provide additional clinical benefit to doctors and patients seeking cancer risk assessment.
To learn more about this publication, visit our Clinical Studies page.
We are happy to share more details on any of our validation studies with you. Please contact Client Services to request additional information.