- Technical advance
- Open Access
- Open Peer Review
HPV testing for cervical cancer screening: technical improvement of laboratory logistics and good clinical performance of the cobas 6800 in comparison to the 4800 system
BMC Women's Healthvolume 19, Article number: 47 (2019)
European guidelines for cervical cancer screening now recommend the use of clinically validated assays for high-risk HPV-DNA sequences as primary test in women older than 30 years, performed in centralized laboratories, and run on systems providing automated solutions for all steps.
We conducted a comparison study, according to the international guidelines, nested within the organized population-based cervical screening program, between the cobas 4800 and 6800 systems (Roche Diagnostics), to evaluate accuracy and reproducibility of HPV test results and laboratory workflow. In Italy implementation of HPV cervical screening is under way on a regional basis; in Veneto it started in June 2015, following a piloting phase; the assay in use in the three centralized laboratories is the cobas 4800 HPV test, run on the cobas 4800 system. Comparison of HPV results with a new version of the assay (cobas 6800/8800 HPV) run on the cobas 6800 system, and intra- and inter-reproducibility analyses have been conducted in samples collected in PreservCyt medium (Hologic) from women without and with a subsequent diagnosis of high-grade lesion.
Samples from women older than 30 years attending organized cervical cancer screening were used. Clinical sensitivity and specificity were evaluated on 60 cases and 925 controls, respectively; intra-laboratory reproducibility and inter-laboratory agreement by the 6800 system were evaluated on 593 and 460 specimens, respectively. Our results showed a very high agreement (> 98%) for overall qualitative results between the two systems; clinical sensitivity and specificity of the HPV assay run on 6800 were non-inferior to those of the HPV assay run on 4800 (p = 0,0157 and p = 0,0056, respectively, at the recommended thresholds of 90 and 98%); kappa values of 0.967 and 0.969 were obtained for intra- and inter-laboratory reproducibility analyses in the 6800 system. The 6800 platform displayed several technological improvements over the 4800 system, with higher throughput and laboratory productivity, and lower operator’s hands-on time.
The new cobas 6800/8800 HPV assay run on the 6800 instrument is suitable for use in large centralized laboratories included within population-based cervical cancer screening programs.
DNA testing for HPV oncogenic types  as primary test in organized cervical cancer screening is more effective than cytology in women above 30 years of age . European  and Italian  guidelines recommend the use of clinically validated HPV assays , to be performed in centralized laboratories.
In order to allow testing in large centralized laboratories and to ensure diagnostic accuracy, the assays must be run on systems providing automated solutions for both pre-analytical and analytical phases, starting from specimen primary tubes. Operator interaction, test throughput, workflow and system maintenance requirements are important determinants and substantial differences among some available automated systems have been shown ; for most of the validated assays, at present, two or more diagnostic systems are necessary to guarantee the requested workload of a centralized laboratory. Technological improvements of the instrumentations in use with some of the validated assays have been released and/or are on the way; their clinical performance, consistency of results and operational characteristics are best evaluated by real-world studies.
In Italy, the Ministry of Health has introduced the implementation of HPV testing for cervical screening in the National Preventive Plan 2014–2018; cancer screenings are managed at regional level, and all regions are expected to comply by the year 2018. In the Veneto region, the use of DNA HPV testing in primary cervical screening was piloted in five organized programs from April 2009 to May 2015 [7,8,9], and fully implemented in all programs since June 2015. The HPV assay actually in use is the clinically validated cobas 4800 HPV test (Roche Diagnostics) [10, 11], run on the cobas 4800 system. More recently, a new version of the assay (cobas 6800/8800 HPV) for use on the cobas 6800 and cobas 8800 systems  has been developed and CE/IVD (Conformité Européenne/in vitro diagnostics) labeled.
The aim of our study is the evaluation of the cobas HPV assay performed on the new 6800 platform in comparison to the 4800 for use in cervical cancer screening.
Aim, design and setting of the study
The study was nested within the organized population-based cervical cancer screening, and carried out in one of the three centralized HPV laboratories of the Veneto region (Italy), serving five programs. The HPV screening protocol is applied to women older than 30 years and includes HPV testing and cytology triage of HPV-positive samples, followed by immediate colposcopy in case of ASC-US (Atypical Squamous Cells of Undetermined Significance) or more severe diagnosis, or HPV retesting at 1-yr recall in case of negative cytology . Women testing HPV-negative will be re-invited at 5-yrs interval.
The study design aimed at evaluating clinical sensitivity and specificity and assay reproducibility by the cobas 6800 HPV assay in comparison to the cobas 4800 assay, according to the Meijer’s criteria , as well as technical performance and laboratory workflow of the two platforms.
The instrumentation for the study (p 480 v2 and cobas 6800) was temporarily provided by Roche Diagnostics. The inter-laboratory analyses were performed at the Microbiology and Virology Unit of the General Hospital of Treviso, where a cobas 6800 system is in use for other assays, and the HPV dedicated software was implemented.
Clinical samples and HPV testing by cobas 4800
All samples were previously tested by the cobas 4800 HPV assay, based on real-time polymerase chain reaction (PCR) technology, by use of the 4800 system, that provides full sample preparation (cobas × 480) and HPV detection (cobas z 480 analyzer); the two instruments are physically separated and require the manual transfer of the reaction plate. The method detects 14 HPV types (the 12 designed as high-risk by the IARC, plus types 68 and 66), provides individual HPV genotype results for HPV16 and HPV18, while detecting the other 12 types as a pool (other HR), and includes an internal quality control (beta-globin) for each sample (in the screening report only the result for all hrHPV types as a pool is included). The primary vial is mixed and decapped/recapped by the p480 v1 instrument.
Residual material of cervical samples collected in PreservCyt medium (Hologic, Bedford, MA, USA) from women older than 30 years, attending cervical cancer screening was used. Consecutive specimens were collected during February 1 through May 31, 2017 and during December 1, 2017 through February 15, 2018, and stored for no longer than 6 months before testing by 6800; selected samples were also included, in order to increase the number of cases (25 samples obtained from women with high-grade lesions and stored for up to 3 years at 4 °C before testing for the present study) and of HPV-positive samples (to provide a 25–30% prevalence of high-risk HPV for the reproducibility analyses). The 25 samples stored longer than 6 months were re-tested by cobas 4800 to verify amplificability and consistency of results; all samples gave a valid result (1 was invalid only for the HPV channels previously negative) with overlapping qualitative results and Ct (cycle threshold) values, and were deemed suitable for inclusion in the study.
For ethical reasons, all samples were anonymized before HPV analysis. The study has been approved by the local Ethical Committee (EC code 2017–07 plus EM 193/2017).
Sample processing and cobas 6800 HPV assay
Primary vials were initially processed by the p 480 v2 (with software 2.1.1) instrument, that performed spin-mixing, cap removal, barcode alignment and transfer of 2-ml sample aliquots to secondary vials, and recapping with the original primary vial cap.
The secondary vials were then manually transferred to the cobas 6800 system, a fully automated unit that provides full sample preparation and HPV detection without further intervention by the operator.
Both cobas HPV real-time PCR assays detect the same 14 types of HPV, use the same primers and probes, and provide partial genotyping for HPV16 and HPV18. The two assays do however differ in their Thermal Cycling (CT) profile (the cobas 6800/8800 runs a universal thermal cycling profile to allow for mixed batching of different PCR tests), as well as in the elution sample volume amplified (50 μl on 6800/8800 vs 150 μl on 4800 out of a 400 μl aliquot of extracted nucleic acids). Both HPV assays were performed according to the manufacturer’s instructions. Each plate, besides the 2 assay’s controls, contained 92 clinical specimens and 2 additional samples (selected from 10 internal and 5 external quality controls, and 2 clinical samples previously found to be invalid by the cobas 4800 HPV test).
Clinical sensitivity and clinical specificity were evaluated on samples from women with a histologically confirmed high-grade lesion (CIN2+) and on samples from women with no or low-grade lesions, respectively; the results with the 6800 assay were compared to those with the 4800 by a non-inferiority score test, according to the thresholds (relative sensitivity of at least 90% and relative specificity > 98%) recommended by Meijer et al. . The HPV results obtained by the two systems were assessed using overall percentage agreement of qualitative (positive/negative) results and type (HPV16, HPV18, other HPV) agreement; complete and partial type concordance were evaluated also by hierarchical categorization (i.e., HPV16 alone and with any other type; HPV18 alone and with any non-16 other type; non-16/non-18 other HPV types only). For discordant results, the cycle threshold (Ct) values were also analyzed. The intra-laboratory reproducibility and the inter-laboratory agreement were evaluated by calculating the kappa values and 95% confidence intervals (95%CI) by bootstrap analysis (replications = 1000; bias corrected); we also report the squared correlation coefficient R2 (variation in Y explained by X/variation in Y) between the Ct values.
The cases’ group comprised 62 samples from women (mean age 41 yrs.; median age 40 yrs.; range 30–59 yrs) with a histologically confirmed high-grade lesion (32 CIN2, 27 CIN3, 2 squamous cell carcinomas, 1 adenocarcinoma) diagnosed either at baseline (42 with HPV-positive/cytology positive results, and 3 with persistent HPV positivity at 1-yr recall) or during follow-up (17 cases). The controls’ group comprised 925 women (mean age 46 yrs.; median age 45 yrs.; range 30–68 yrs) with no or low-grade lesions. By the cobas 4800 HPV assay, all the cases and 67/925 (7.2%) controls were HPV-positive.
A valid cobas 6800 HPV test result has been obtained for all but two samples (both from cases, comprising the partially invalid at 4800 re-testing, excluded from the study); 59/60 (98.4%) cases and 73/925 (7.9%) controls gave a positive result. A concordant result was recorded in 59/60 (98.4%) cases and in 915/925 (98.9%) controls. In Table 1 the qualitative results (positive/negative), as well as the HPV type distribution by hierarchical categorization, of all the samples are reported.
Among the 60 women with a CIN2+ diagnosis and a cobas 6800 valid result, the HPV test was positive in all but one sample (diagnosed as CIN2 and 4800-positive with a 39,7 Ct value). Of the 10 discordant samples among the controls, 2 were 4800-positive/6800-negative and 8 were 4800-negative/6800-positive (Table 1). Reactivity in only one of the three HPV channels was recorded in all discordant specimens, with a median Ct value of 38,05 (range 37,7-38,4) for the 4800-positive/6800-negative samples, and of 36,11 (range 31,46-37,47) for the 4800-negative/6800-positive ones (Table 2, upper panel).
Overall, HPV type analysis of the 124 samples positive by both systems (cases plus controls) showed complete concordance in 118 (95.2%) specimens and partial concordance in the remaining 6. The results of the partially discordant specimens are detailed in Table 2, lower panel. Reactivity to a single channel was detected by the 4800 assay in all cases; this was confirmed by the 6800 assay, which additionally displayed reactivity with another HPV channel. Of note, on the 6800 platform the Ct values for the concordant results were always lower than those for the additional positivity, indicative of a higher viral load of the concordantly detected type.
Clinical sensitivity and specificity values of the cobas 6800 HPV assay were compared to those of the 4800 assay, according to the recommendations for the clinical validation of new HPV-DNA assays ; the score values of the 6800 for both sensitivity (98% at 0.90 threshold) and specificity (99% at 0.98 threshold) were non-inferior to those of the 4800 (p = 0,0157 and p = 0,0056, respectively).
The intra-laboratory reproducibility of the 6800 assay was evaluated on 593 samples, 178 (30%) resulted HPV-positive at first testing. Overall qualitative agreement was recorded for 587/593 (99%), with a kappa value of 0.967 (95%CI 0.942–0.985). All 6 discordant samples (positivity for HPV16 in 2 and for other HPV in 4; median Ct values 34) were negative on the second testing. Among the samples HPV-positive in both runs, type-specific agreement was observed in 169/172 (98.3%) (Table 3).
The inter-laboratory reproducibility was evaluated on 460 samples (Table 4). Overall qualitative agreement was recorded for 456/460 (99.1%), with a kappa value of 0.969 (95%CI 0.941–0.990). All discordant samples displayed single reaction to one HPV channel (1 for HPV16 and 3 for other HPV). Type-specific agreement was observed in 117/119 (98.3%) samples HPV-positive in both runs.
Scatter plots of the Ct values of the HPV-positive samples are reported in Fig. 1; a linear correlation was found for all the data; coefficients for cobas 6800 intra- and interlaboratory data were higher (R2 = 0,92) than those for 4800 vs 6800 comparison (R2 = 0,78).
In each plate, two control samples were included. The 10 (8 HPV-positive, 2 HPV-negative) internal quality controls (IQC), each tested 2–6 times, were always concordant but twice for the same HPV18-positive sample that on two-out-of-three repetitions was judged as negative; additional in-depth evaluation of its analytical data disclosed that for one of the two negative results the Ct for the HPV18 channel was very high and that the mean fluorescence intensity was very weak. The 5 (4 HPV-positive, 1 HPV-negative) external quality controls (EQC) were concordant in 3 HPV-positive cases and invalid in the other 2 (one due to insufficient material, and one to a negative beta-globin result). The 2 samples resulted invalid on two different runs with the 4800 assay, both gave a valid result on the 6800 assay.
Comparison of the workflow by the two systems has shown higher laboratory operational performances of the 6800 over the 4800, and of the p 480 v2 in relation to the p 480 v1, respectively. The most important differences are summarized in Table 5. Of note, the 6800 unit contains an on-board refrigerator for reagent storage and additional storage space for consumables, thus improving both loading and long-term use of reagents and consumables. An additional improvement of the 6800 system is the automatic reporting of the Ct values for the HPV results and the internal control for the samples and the kit positive control on the assay report; the Ct values for the three HPV channels of the kit control were more uniform (median ~ 35 for all) than what recording with the 4800 system (median ranges comprised from ~ 37 to ~ 39). Finally, taking into account both the preparation and the analysis of the specimens, the amount of liquid and solid waste produced is sensibly reduced on the p 480 v2/6800 platform in comparison to the p 480 v1/4800 one (Table 5), as a result of important technological differences in all the steps. In particular, among others, while recapping by the p 480 v1 is done with new caps, the original caps are used by the p 480 v2, and while 11 tips/sample are used by the 4800, a unique tip/sample is used by the 6800.
The cobas HPV test detects 14 high-risk HPV types, is based on real-time PCR technology, has been clinically validated [10, 11] and provides partial genotyping for HPV16 and HPV18, while detecting the other 12 HPV types as a pool. More recently, it has been released for use on the newly launched cobas 6800 platform (already in use for other molecular assays). In this study we compared the analytical and clinical performance, as well as other system features, of the cobas 4800 and 6800 platforms for HPV testing in cervical cancer screening, according to the Meijer’s criteria . Compared to the cobas 4800 assay, the results by cobas 6800 showed an agreement > 98% for overall qualitative results between the two systems, a non-inferior clinical sensitivity and specificity, and excellent intra- and inter-laboratory reproducibility. These figures are in line with the results previously obtained in the studies conducted for the clinical validation of the cobas 4800 assay [10, 11]. The Ct values (that are inversely proportional to the viral load) obtained on the 6800 system have always been lower than those recorded by the 4800, most probably due to the different amount of target sequences analyzed by the two instruments (i.e., 25/150 μl by 4800 vs 25/50 μl by the 6800). Overall, the minor differences that emerged at the analytical level (i.e., Ct values) had no major impact on the clinical performance. Differences in semi-quantitative results in this setting are relevant only if they actually determine a change in the positive/negative result, i.e., if the value passes the threshold.
Comparison of HPV type distribution among the samples positive by both systems, and on 6800 reproducibility assays, showed a very high complete concordance for type(s) assignment (95.2% in 4800/6800 comparison, and 98.3% in both 6800 intra- and inter-reproducibility assays), in line with previous studies [10, 11, 13].
HPV16 (and to a lesser extent HPV18) has been shown to have a stronger association than other high-risk types for CIN2+ and invasive cancer ; as a consequence, partial genotyping for the management of HPV-positive women in the screening programs has recently been proposed as a triage test in the Australian  and Dutch  protocols, although its clinical value is still a matter of debate [17, 18]. In our study, HPV16 (either alone or mixed with other types) was detected in 19,4% (13/67) and in 26% (19/73) of the specimens by cobas 4800 and 6800, respectively, while complete type concordance was observed among the women with a diagnosis of CIN2+.
According to the European and national guidelines, HPV testing for cervical cancer screening must be centralized in large reference laboratories and performed by using clinically validated assays run on automated systems. As a consequence, the degree of automation for all the steps of the process (from sample preparation, starting from the original sample vial, to validation of the results) and the laboratory productivity are essential components to assure high-quality results and high throughput. In our study, we have shown that the workflow and several instrumentation features are sensibly improved on the new platform, as already shown also for other assays . Among others, it is a compact unit comprising the thermal cycler for the real-time PCR (which implies direct transfer of the reaction plate without operator interventions); reagents and consumables are provided in a cartridge format (that avoid loading/unloading operations at each plate run); it is provided by an on-board refrigerated storing compartment for reagents (allowing their subsequent use); it can accommodate a larger number of samples (up to 280); it has shorter turn-around time for assay results and hands-on time, and is user-friendly.
Our data on the HPV assay performed in a cervical screening context on the new cobas platform composed by the p 480 v2 plus the 6800 instruments, have shown high consistency of results with the cobas 4800 system in use and high intra- and inter-reproducibility, together with higher performance in workflow and laboratory productivity, thus demonstrating its suitability for use in large centralized laboratories included within population-based cervical cancer screening programs.
Atypical Squamous Cells of Undetermined Significance
Conformité Européenne/in vitro diagnostics
Cervical Intraepithelial Neoplasia
External quality control
International Agency for Research on Cancer
Internal quality control
Polymerase chain reaction
Bouvard V, Baan R, Straif K, El Ghissassi F, Benbrahim-Tallaa L, Guha N, et al. On behalf of the WHO IARC in Cancer monograph working group. A review of human carcinogens –part B: biological agents. Lancet Oncol. 2009;10:321–2.
Ronco G, Dillner J, Elfström M, Tunesi S, Snijders PJ, Arbyn M, et al. International HPV screening working group. Efficacy of HPV-based screening for preventing invasive cervical cancer: follow-up of European randomised controlled trials. Lancet. 2014;383:524–32.
Ronco G, Arbyn M, Meijer CJLM, Snijders PJF, Cuzick J. Screening for cervical cancer with primary testing for human papillomavirus. S1. In: Anttila A, Arbyn M, De Vuyst H, Dillner J, Dillner L, Franceschi S, Patnick J, Ronco G, Segnan N, Suonio E, Törnberg S, von Karsa L, editors. European guidelines for quality assurance in cervical cancer screening. Second edition, Supplements. Luxembourg: Office for Official Publications of the European Union; 2015. p. 1–68.
Ronco G, Biggeri A, Confortini M, Naldoni C, Segnan N, Sideri M, et al. Health Technology Assessment report: HPV DNA based primary screening for cervical cancer precursors. Epidemiol Prev. 2012;36(3–4 Suppl 1):1–72. Italian.
Meijer CJ, Berkhof J, Castle PE, Hesselink AT, Franco EL, Ronco G, et al. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. Int J Cancer. 2009;124:516–20.
Ratnam S, Jang D, Gilchrist J, Smieja M, Poirier A, Hatchette T, et al. Workflow and maintenance characteristics of five automated laboratory instruments for the diagnosis of sexually transmitted infections. J Clin Microbiol. 2014;52:2299–304.
Zorzi M, Del Mistro A, Farruggio A, de’ Bartolomeis L, Frayle Salamanca H, Baboci L, et al. Use of high-risk human papillomavirus DNA test as primary test in a cervical cancer screening programme: a population-based cohort study. BJOG. 2013;120:1260–7.
Del Mistro A, Frayle H, Ferro A, Callegaro S, Del Sole A, Stomeo A, et al. On behalf of the Veneto HPV-screening working group. Cervical cancer screening by high risk HPV testing in routine practice: results at one year recall of high risk HPV-positive and cytology-negative women. J Med Screen. 2014;21:30–7.
Zorzi M, Frayle H, Rizzi M, Fedato C, Rugge M, Penon MG, et al. A 3-year interval is too short for re-screening HPV negative women: a population-based cohort study. BJOG. 2017;124:1585–93.
Heideman DA, Hesselink AT, Berkhof J, van Kemenade F, Melchers WJ, Fransen DN, et al. Clinical validation of the cobas(R)4800 HPV test for cervical screening purposes. J Clin Microbiol. 2011;49:3983–5.
Lloveras B, Gomez S, Alameda F, Bellosillo B, Mojal S, Muset M, et al. HPV testing by cobas HPV test in a population from Catalonia. PLoS One. 2013;8:e58153.
Cobb B, Simon CO, Stramer SL, Body B, Mitchell PS, Reisch N, et al. The cobas 6800/8800 system: a new era of automation in molecular diagnostics. Expert Rev Molec Diagn. 2017;17:167–80.
Engesaeter B, van Diermen Hidle B, Hansen M, Moltu P, Staby KM, Borchgrevink-Persen S, et al. Quality assurance of human papillomavirus (HPV) testing in the implementation of HPV primary screening in Norway: an inter-laboratory reproducibility study. BMC Infect Dis. 2016;16:698.
Schiffman M, Glass AG, Wentzensen N, Rush BB, Castle PE, Scott DR, et al. A long-term prospective study of type-specific human papillomavirus infection and risk of cervical neoplasia among 20,000 women in the Portland Kaiser cohort study. Cancer Epidemiol Biomark Prev. 2011;20:1398–409.
Simms KT, Hall M, Smith MA, Lew JB, Hughes S, Yuill S, et al. Optimal management strategies for primary HPV testing for cervical screening: cost-effectiveness evaluation for the national cervical screening program in Australia. PLoS One. 2017;12(1):e0163509.
Luttmer R, De Strooper LMA, Steenbergen RDM, Berkhof J, Snijders PJF, Heideman DAM, Meijer CJLM. Management of high-risk HPV-positive women for detection of cervical (pre)cancer. Exp Rev Mol Diagn. 2016;16:961–74.
Rijkaart DC, Berkhof J, van Kemenade FJ, Coupe VM, Hesselink AT. Rozendaal, et al. evaluation of 14 triage strategies for HPV DNA-positive women in population-based cervical screening. Int J Cancer. 2012;130:602–10.
Wentzensen N, Arbyn M, Berkhof J, Bower M, Canfell K, Einstein M, et al. Eurogin 2016 roadmap: how HPV knowledge is changing screening practice. Int J Cancer. 2017;140:2192–200.
Vermehren J, Stelzl E, Maasoumy B, Michel-Treil V, Berkowski C, Marins EG, et al. Multicenter comparison study of both analytical and clinical performance across four Roche hepatitis C virus RNA assays utilizing different platforms. J Clin Microbiol. 2017;55:1131–9.
We thank the medical and technical teams at Roche Diagnostics in Monza, Italy, for their support. In particular, we thank M. Fumiani, M. Griffante, C. Calvi, P. Maceratesi for their help.
The cobas 6800 HPV assays performed in this study were funded entirely by Roche Diagnostics, Italy; no payments were made to any of the investigators. The funder had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Availability of data and materials
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The study has been approved by the IOV-IRCCS Ethical Committee (EC code 2017–07 plus EM 193/2017). The women who participate to the screening program give a written consent, and an additional specific consent for the study was deemed unnecessary, as all the samples were anonymized and the results of the study didn’t have any consequence for the women’s management.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.