Skip to main content
  • Research article
  • Open access
  • Published:

Cross design analysis of randomized and observational data – application to continuation rates for a contraceptive intra uterine device containing Levonorgestrel in adolescents and adults

Abstract

Background

To combine results from a randomized controlled study (RCT) and an observational study (OS) to evaluate discontinuation rate of a levonorgestrel-containing intrauterine contraceptive device (LNG IUD) in a real-life setting.

Methods

We included 253 parous and nulliparous women aged 21–40 years from our own phase II RCT. A total of 1607 women of all ages (including adolescents, < 20 years) were recruited from an OS. We applied the cross design synthesis (CDS) method recommended by the United States General Accounting Office. This method combines the different strengths of RCTs and OSs into one single estimate.

Results

Combined continuation rates for parous vs nulliparous women could be estimated more precisely as well as overall continuation rates after one (86.6%) and two years (78.5%), irrespective of age and parity.

Conclusion

Cross design synthesis allowed more precise estimation of continuation rates of an intrauterine device.

Peer Review reports

Background

For assessing the risk benefit profile of medical technologies, today, multiple data sources are leveraged, e.g., randomized clinical trials (RCT), observational studies (OS), health insurers’ claims data, etc. Data from these different sources are usually analyzed separately.

RCTs, mostly performed as phase II or phase III studies in the framework of clinical development programs, typically recruit a highly selected subset of the total patient population in order to achieve the highest possible internal validity. However, as a consequence, the external validity is low. Therefore, findings in patients purposely excluded from RCTs should complement the body of data. These patients are usually included in OSs, which are more representative of the daily routine clinical setting. Thus, OSs focus on external validity while their internal validity is compromised by confounding factors which can hardly be controlled.

There is uncertainty whether differing treatment effects observed in RCTs and OSs are due to different inclusion criteria or are caused by other hard to specify ‘real life’ factors. In the pursuit of raising internal and external validity of parameters like treatment efficacy, overall safety, treatment adherence, quality of life, etc. for informed healthcare decisions, combining data from RCTs and OSs is an avenue to go.

Research on different methods for merging data from RCTs and from OSs into one single analysis experienced increasing attention in the biomedical and biostatistical literature [1,2,3]. One of these methods is has been coined ‘cross design synthesis’ (CDS).

The aim of our study was to apply the method of CDS on investigating continuation rates of the levonorgestrel-containing intrauterine device (LNG IUD) (Mirena®), indicated for long-term reversible contraception. This LNG IUD provides higher contraceptive effectiveness than typical oral contraceptives as it does not rely on regular user adherence which is of special relevance for adolescents. As adherence and continuation are key for effectiveness [4] we investigated one and two year continuation rates of this LNG IUD by merging results from an RCT and an OS.

Methods

For our CDS we leveraged results from our own RCT and from an OS, published by Abraham et al. [5]. To our knowledge, these were the only large scale data sources available for our research question.

The RCT was a randomized clinical phase II study comparing 3 doses of the LNG IUD in a parallel open-label design. A total of 733 parous and nulliparous women, aged 21–40 years, seeking long term contraception, were included. Of those, 253 women took the LNG IUD analyzed in this study. Follow-up was for three years. The primary target parameter was contraceptive efficacy, measured by the Pearl Index [6]. The main results of the study were published by Gemzell-Danielsson et al. [7].

The observational data were taken from the Contraceptive CHOICE Project - a prospective cohort study that followed 9256 participants with telephone surveys at the Washington University School of Medicine in St. Louis. At enrollment women at risk of unintended pregnancy could start a new reversible contraceptive method after comprehensive counselling on all available reversible contraceptive options. Details of the Contraceptive CHOICE study have been published elsewhere [8]. We used the subset of continuation rates of the LNG IUD as data from the real-world setting for our CDS (Table 1). The individual patient data for the RCT were re-analyzed retrospectively to match the definitions used in the analysis of the OS as described in [5, 8].

Table 1 Percentage of women continuing levonorgestrel IUD at 1 and 2 years by age and parity in the OS (modified from Abraham et al. 2015, [5])

CDS was initially recommended by the United States General Accounting Office as a new strategy for medical effectiveness research [9, 10]. The aim is to combine the different strengths of RCTs and OSs into one single estimate by ‘extrapolating’ the RCT results to the initially excluded population by using OS data.

Simplifying the approach of Kaizar [3] we calculated the cross design estimator based on the results from the RCT dRCT and the results from the OS dOS stratified according to whether or not the women of the OS fulfilled the RCT’s major inclusion criterion, i.e. age of at least 20. The results for these strata were denominated as \( {d}_{OS_{in}} \) for the women of the OS fulfilling the RCT’s inclusion criterion and as \( {d}_{OS_{ex}} \) for the women of the OS not fulfilling the RCT’s inclusion criterion. The CDS estimator can be written as

$$ {d}_{CDS}={d}_{RCT}+\frac{n_{OS_{ex}}}{n_{OS}}\times \left({d}_{OS_{ex}}-{d}_{OS_{in}}\right) $$

where nOS stands for the number of women included in the OS and \( {n}_{OS_{ex}} \) for the number of women that were included in the OS and that did not fulfil the RCT’s inclusion criterion. The variance \( {S}_{CDS}^2 \) of the CDS estimator can be expressed as

$$ {S}_{CDS}^2=\frac{S_{RCT}^2}{n_{RCT}}+\frac{n_{OS_{ex}}^2}{n_{OS}^2}\times \left(\frac{S_{OS_{ex}}^2}{n_{OS_{ex}}}+\frac{S_{OS_{in}}^2}{n_{OS_{in}}}\right) $$

where \( {S}_s^2 \) and ns stand for the variance and sample size of the women belonging to the subscripted group s.

The CDS estimator is unbiased if treatment selection error for the patients in the OS, fulfilling the inclusion criteria of the RCT, and the patients, who do not, is constant [3]. The CDS continuation rates after one and two years of LNG IUD use were calculated for parous and nulliparous women as well as for the total population.

Results

Table 1 shows the results from the observational study on the LNG IUD published by Abraham et al. [5]. We selected continuation rates after 1 and 2 years grouped by age and parity as basis for our calculation.

As a first step we condensed the findings of the OS in two age groups: > 20 years (i.e. those and 20–25 and > 25 years) and < 20 years and added the respective results from the RCT (Table 2). The sample size of the OS was much higher (n = 1607) than for the RCT (n = 253).

Table 2 Pooled (age groups 20–25 and > 25) continuation rates (%) from the OS and RCT by age and parity

First, women > 20 years and parity: In the OS nulliparous women showed slightly higher continuation rates than parous women, especially visible after two years, 78% vs. 75%. In the RCT this trend was the other way round: parous women showed higher continuation rates than nulliparous women, i.e. 81% vs 70% after two years.

Comparing study types, parous women showed higher continuation rates in the RCT than in the OS, while it was the other way round for nulliparous women.

Secondly, women < 20 years and parity (only OS results available): Nulliparous women had lower continuation rates than parous women, e.g. 67% vs 73% after two years. The continuation rates of the younger women were consistently lower than for those 20 years and older.

Irrespective of parity the continuation rates for women > 20 years were lower in the OS than in the RCT, e.g., 76% vs 79% after 2 years.

In the OS the continuation rates for women < 20 years of age were lower than those for women > 20 years, 69% vs 76% after two years (Table 3).

Table 3 Pooled (age groups 20–25 and > 25) continuation rates (%) from OS and RCT by age

The cross design synthesis combines results of the two study types by parity but not by age, as women < 20 years were not included in the RCT.

The CDS continuation rates for parous women were 90 and 81% after one and two years, respectively. The corresponding figures for nulliparous women were lower, i.e., 73 and 68%. For all women, irrespective of parity, the CDS continuation rates were 87% after the first year and 79% after the second year (Table 4).

Table 4 Results cross design synthesis, continuation rates (%) by parity

Discussion

Merging of data from RCTs and OSs offers new possibilities to assess medical interventions in a broader sense. CDS combines results from studies with complementary designs in order to capture the designs’ strengths and minimize the studies’ weaknesses. We used this method to estimate continuation rates of an LNG IUS. To the best of our knowledge, this is the first study of this kind on any long-acting contraceptive.

We were able to enrich the RCT’s results for women > 20 years with data for women of all age groups, including adolescents (< 20 years), from an OS. Numerically the results did not change drastically but this was probably due to the low fraction of adolescents in the OS (9.7%). However, we obtained a more precise estimate of the continuation rates.

A limitation of our study is that only our RCT provided individual data sets while we could only use aggregated data from the OS. Therefore, only matching of the study populations for age was possible. Other comparisons were impossible as the study populations differed substantially, e.g. for ethnic origin: The RCT population was almost exclusively Caucasian whereas 48.1% of the OS population was Black [5]. Also for geographic location: The RCT was performed in northern and central Europe while the OS was performed in St. Louis, Missouri, USA.

Conclusion

Cross design synthesis allowed more precise estimation of continuation rates of an intrauterine device.

Abbreviations

CDS:

Cross design synthesis

LNG IUS:

Levonorgestrel-containing intrauterine contraceptive device

OS:

Observational study

RCT:

Randomized controlled study

References

  1. Verde PE, Ohmann C, Morbach S, Icks A. Bayesian evidence synthesis for exploring generalizability of treatment effects: a case study of combining randomized and non-randomized results in diabetes. Stat Med. 2016;35(10):1654–75.

    Article  Google Scholar 

  2. Verde PE, Ohmann C. Combining randomized and non-randomized evidence in clinical research: a review of methods and applications. Res Synth Methods. 2015;6(1):45–62.

    Article  Google Scholar 

  3. Kaizar EE. Estimating treatment effect via simple cross design synthesis. Stat Med. 2011;30(25):2986–3009.

    Article  Google Scholar 

  4. Potter LS. How effective are contraceptives? The determination and measurement of pregnancy rates. Obstet Gynecol. 1996;88(3 Suppl):13S–23S.

    Article  CAS  Google Scholar 

  5. Abraham M, Zhao Q, Peipert JF. Young age, Nulliparity, and continuation of long-acting reversible contraceptive methods. Obstet Gynecol. 2015;126(4):823–9.

    Article  Google Scholar 

  6. Gerlinger C, Endrikat J, van der Meulen EA, Dieben TO, Dusterberg B. Recommendation for confidence interval and sample size calculation for the pearl index. Eur J Contracept Reprod Health Care. 2003;8(2):87–92.

    Article  CAS  Google Scholar 

  7. Gemzell-Danielsson K, Schellschmidt I, Apter D. A randomized, phase II study describing the efficacy, bleeding profile, and safety of two low-dose levonorgestrel-releasing intrauterine contraceptive systems and Mirena. Fertil Steril. 2012;97(3):616–22 e611–613.

    Article  CAS  Google Scholar 

  8. Secura GM, Allsworth JE, Madden T, Mullersman JL, Peipert JF. The contraceptive CHOICE project: reducing barriers to long-acting reversible contraception. Am J Obstet Gynecol. 2010;203(2):115 e111–7.

    Article  Google Scholar 

  9. U.S. Government Accountability Office (GAO). Cross design synthesis: a new strategy for medical effectiveness research. 1992. https://www.gao.gov/products/PEMD-92-18.

  10. Cross design synthesis: a new strategy for studying medical outcomes? Lancet. 1992;340(8825):944–6. PMID: 1357352.

Download references

Acknowledgements

Not applicable

Funding

None.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated during the current study.

Author information

Authors and Affiliations

Authors

Contributions

TV, AF, and CG contributed to the study concept and design. RL, JE, and CG provided medical input to the study. TV and CG provided the statistical analysis. JE wrote the first draft manuscript. All authors contributed to the interpretation of the data, were involved in revising the manuscript for important intellectual content, and approved the final version of the manuscript.

Corresponding author

Correspondence to Christoph Gerlinger.

Ethics declarations

Ethics approval and consent to participate

This study reports on secondary analyses of published data. No new ethics approval was required.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vaitsiakhovich, T., Filonenko, A., Lynen, R. et al. Cross design analysis of randomized and observational data – application to continuation rates for a contraceptive intra uterine device containing Levonorgestrel in adolescents and adults. BMC Women's Health 18, 180 (2018). https://doi.org/10.1186/s12905-018-0674-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12905-018-0674-1

Keywords