Prediction of Treatment Option by Human Chorionic Gonadotropin (hCG) Levels in Ectopic Pregnancy using Machine Learning

Introduction

Ectopic pregnancy is an important cause of maternal morbidity and mortality; 1.3–2% of all pregnancies are diagnosed as ectopic pregnancies [1]. Ectopic pregnancy is diagnosed by serum human chorionic gonadotropin (hCG), urinary hCG, transvaginal ultra- sonography, computed tomography, vascular endothelial growth factor, disintegrin and metalloprotease-12 and hysterosalpingog- raphy [2]. In particular, the sensitivity and specificity of transvag- inal ultrasonography in diagnosis are currently 84.4% and 98.9%, respectively [3,4]. Changes in hCG levels, as well, are important indicators of ectopic pregnancy. Barash et al. reported that β-hCG values in approximately 99% of viable intrauterine pregnancies increase by about 50% in 48 hours [5]. The remaining 1% have a slower rate of increase or decrease, and these include miscarriages and nonviable intrauterine or ectopic pregnancies [5]. Moreover, several papers have suggested that ectopic pregnancy should be suspected when a patient in their first trimester has abdominal pain or bleeding [2,5].

Ectopic pregnancy treatments involve surgical intervention, medi- cal treatment using methotrexate (MTX), and expectant treatment [2]. There have been numerous studies on which of these is the best treatment. Recently, there have been reports of systems that use artificial intelligence to support diagnosis and treatment deci- sions, and one particular paper on ectopic pregnancy used machine learning. Alberto et al. developed a three-stage classifier to predict the treatment for ectopic pregnancy and tested this with four dif- ferent algorithms. The best of the four was the support vector ma- chine, with accuracy, sensitivity, and specificity of 96.1%, 96.0%, and 98.0%, respectively [6].

There are numerous machine learning methods, and the Support Vector Machine is one method for constructing a pattern discrim- inator for classifications [7]. Therefore, this machine is highly ac- curate at classifying the given information. However, one weak- ness of artificial intelligence is that it is a so-called “black box” regarding why the classifications were made [8].

On the other hand, random forests enable classification using a decision tree and allow us to know which input variables were in- volved in the classification. A random forest is a machine learning algorithm proposed by Leo Breiman in 2019 [9]. While decision trees may lead to overlearning, a random forest generates multiple, relatively simple decision trees by randomly selecting explanatory variables and then averaging the output of each to reduce variance and achieve predictions with a lower rate of error.

Until now, there has been no report of a support system for the selection of treatment for ectopic pregnancy using a random forest. The purpose of this study was to construct such a support system for the diagnosis and treatment selection of ectopic pregnancy us- ing a random forest and to confirm its accuracy. In addition, we investigated what explanatory variables were important in these decisions [10].

Methods

Study Design and Participants

Data were collected from all records of all cases managed for sus- pected ectopic pregnancy at Osaka Medical and Pharmaceutical University for December 2014-March 2021. This study was ap- proved by our institutional review board (No. 2830), and consent was not required due to its retrospective design. We identified the final treatment for all suspected ectopic pregnancies as either 1) laparoscopic surgery or 2) MTX or expectant treatment. Those who underwent laparoscopic surgery after MTX or observation were added to the group 1.

Variable Selection

Several indicators for the diagnosis and treatment decision for ec- topic pregnancy have been reported [1,11-13]. In this study, we considered all possible indicators and selected 17 variables asso- ciated with the prediction of treatment option in cases of ectopic pregnancy (Table 1). We assumed that abdominal pain and vaginal bleeding were the presenting symptoms on the first suspected day. We then determined the estimated pregnancy weeks from the last menstrual period or date of implantation. In addition, we named the HCG level on which we based our diagnosis as “Diagnosed HCG” and the days on which we measured it as “Diagnosed preg- nant days”. The HCG level on the day of the blood test one or two days prior to the diagnosis was designated as “-1 Diagnosed HCG” or “-2 Diagnosed HCG”, respectively, and the day of the blood test itself was designated as “-1 Diagnosed pregnant days” or “-2 Di- agnosed pregnant days”. If there was no blood test one or two days before the diagnosis, “0 as a meaningless value” was substituted. Substituting a meaningless value does not mean that the value 0 is a missing value, but rather that the variable is skipped and ana- lyzed by other variables.

                                                                                    Table 1: Database Variables

Random Forests

Random forest is a type of machine learning developed by Leo Breiman in 2019 [9]. Random forests are a combination of tree predictors in which each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest [14]. The candidates for the explanatory vari- ables in the database are X1, X2...X N, and the input values are entered as Input. The number of explanatory variables is then se- lected in random combinations and a decision tree such as Tree 1 is constructed. The same process is repeated to construct N decision trees (including Tree 2, Tree 3…, and Tree N). In each decision tree, the target objective variable is searched for and obtained as the result. The output values are then summed and averaged to obtain the final output value.

Model Verification

We created datasets from past records, and entered 17 explana- tory variables and two objective variables “required laparoscopic surgery ultimately or not” into a random forest, created a trained model with 80% of the data, and then checked our model’s accu- racy with 20% of the data. In addition, we compared the accuracy of these models with the prediction accuracy of two experienced obstetrician-gynecologists (OB/GYNs).

Individual names and IDs in this dataset were anonymized. They were not known to the AI model nor to the two OB/GYN physi- cians.Therefore, both the AI model and the OB/GYN physicians studied under equal and unbiased conditions.

Explanatory Variable Importance 

In a random forest, the importance of the explanatory variables can be expressed quantitatively using the remaining data that were not used for training [15]. The equation to derive the importance of the explanatory variables is shown below:

variable m and used it for decision tree i and MSE when we pre- dicted that it would not be correct.

�t: Number of trees

We have shown the explanatory variable importance to learn which variables were important for the decision in the model used in this study.

Performance Measures

To guarantee the validity of the results and to evaluate the classi- fication rule accuracy of the test datasets, we performed a fivefold cross-validation (CV), which is a technique used to assess how a classifier performs when classifying new instances of a task. Us- ing a conventional personal computer (Intel Core i9; 2.4 GHz; 32 GB RAM), we trained the dataset using random forests and an obtained fivefold CV for the training accuracy (TR ACC), test ac- curacy (TS ACC), and area under the receiving operating charac- teristic curve (AUC-ROC).

Statistical Analyses

To compare clinical parameters between those patients who under- went an operation, we used a t-test (P < 0.05 was considered signif- icant) for comparisons. Data were analyzed using SPSS software (Mac version 20.0 J; IBM, Chicago, IL, USA).

Results

In total, we identified 128 cases of suspected ectopic pregnancy at Osaka Medical and Pharmaceutical University in December 2014– March 2021. Among these, 52.3% (67) underwent laparoscopic surgery and 7.0% (9) had emergent laparoscopic surgery. MTX treatment and conservative treatment, including normal pregnan- cies or miscarriages, were 25.0% (32) and 25.0% (32), respective- ly. Three cases required MTX or conservative treatment followed by emergency laparoscopic surgery, and the machine learning sys- tem remembers the two processes.

Table 1 shows the explanatory variables. We extracted a total of 17 variables by referring to previous reports. For HCG levels, the third and second before diagnosis, the HCG level used for diag- nosis, and the estimated number of days of pregnancy were then extracted.

The mean age of all patients was 31.2 years, 11.7% (15) were un- dergoing fertility treatment, and 5.5% (7) were undergoing IVF. Among all patients, 55.4% (71) had vaginal bleeding and 54.6% (70) had abdominal pain. Regarding the ultrasound findings, 23.4% (30) had a fetal sac outside the uterus, 5.4% (7) had a fetal heartbeat, 53.1% (68) had an intra-abdominal mass, and 55.4% (71) had intra-abdominal bleeding.

The mean (range) estimated gestational age at the day of visit to the hospital was 48.3 (33–89) days. The mean (range) of HCG lev- els used for diagnosis was 5,424 (1.9-93,388) mIU/mL, the mean (range) of HCG measured one day before was 2,050 (0- 18,301) mIU/mL, and the mean (range) of HCG measured two days before was 1,608 (0-17,436) mIU/mL.

Table 2 shows the median and range of each variable for (1) lapa- roscopic surgery and (2) MTX or Expectant.

    Abbreviations: G, Gravida; AIH, Artificial insemination with husband’s semen; IVF, in vitro fertilization

                                              Table 2: Patient Variables in Every Treatment Group

Performance of the Random Forest

We achieved a TS ACC of 87.1% and an AUC-ROC of 0.93 for the dataset (Figure 1). The sensitivity and specificity were 90.4% and 83.3%, respectively (Table 3).

ROC: Receiver operating characteristic example

AUC: Area under the curve

                                                                     Figure 1: ROC and AUC for Treatment Decision-Making 

                                                        Table 3: Comparison of Random Forest Model and Obstetrician-Gynecologists

The same findings were judged by two qualified OB/GYNs and cross-checked with the actual results; their accuracies were 77.3% and 79.7%, their sensitivities were 82.1% and 82.1%, their spec- ificities were 72.1% and 77.0%, and their AUCs were 0.77 and 0.79, all respectively (Table 3). Therefore, this random forest model may be able to make better decisions than experienced OB/ GYNs.

Explanatory Variable Importance

Table 4 shows the explanatory variable importance as determined by the random forest model [16]. The most important explanatory variable was the HCG value prior to two of the diagnoses (-2 Di- agnosed HCG at 27.6%). The second most important factor was the number of days of pregnancy with HCG measured before two of the diagnoses (-2 Diagnosed pregnant days at 22.8%). The third most important factor was the HCG value used for the diagnosis (Diagnosed HCG).

                                                       Table 4: Explanatory Variable Importance

Decision Tree

Figure 2 shows a decision tree. Class 1 is the surgical group; Class 2 is the MTX or expectant group. This decision tree tells us the numerical value the model bases its diagnosis on when making a decision. For example, if the -2 Diagnosed HCG is below 38.75 mg/dl and the Diagnosed HCG is above 1854.5 mg/dl, the model will decide to need an operation. However, if the -2 DiagnosedHCG is less than 38.75 mg/dl and the Diagnosed HCG is less than1,854.5 mg/dl and less than 500.0 mg/dl, an operation is not re- quired. Moreover, if the date of presentation and the two previous diagnosed visits are the same and the diagnosed HCG is less than 3,396 mg/dl, it predicts that surgery is not needed if the presenta- tion date is 5.7-7.5 weeks of gestation

                                                                               Figure 2: Decision Tree   

Class 1 is the surgical group; Class 2 is the MTX or expectant group.

Unit: -2 Diagnosed HCG; mIU/L , -1 Diagnosed HCG ; mIU/L ,

Diagnosed HCG ; mIU/L, -2 Diagnosed pregnant days ; days, First visit day ; days This decision tree shows the basis on which the model made its decision by proceeding to the decision tree from the top.

Discussion

In this study, the appropriate treatment for a suspected ectopic pregnancy could be selected with a high accuracy of 87.1% (AUC 0.93) by inputting the 17 variables used in the usual diagnosis, in- cluding HCG levels. In the field of obstetrics and gynecology, ear- ly diagnosis is difficult due to the number of complex mechanisms, especially in cases of ectopic pregnancy [2,17]. This is because the initial symptoms of an ectopic pregnancy are similar to other complications, such as miscarriage and ovarian cyst torsion [18]. Currently, there are various technologies that can help detect ecto- pic pregnancies [19]. For example, there is ultrasonography and a number of blood tests that can measure multiple markers (e.g., be- ta-HCG and progesterone) [1,20,21]. Treatment needs to be started as soon as the diagnosis is confirmed in order to reduce the risk of rupture of the fallopian tubes or other organs. The three approach- es to the management of ectopic pregnancy are surgery (salpin- gostomy or salpingectomy), methotrexate treatment (MTX), or expectant management. Surgical treatment is the gold standard for patients with high HCG levels and intense symptoms of suspected ectopic pregnancy. There have been various reports on whether pa- tients should undergo expectant treatment or not. Rodrigues SP et al. showed that asymptomatic patients with β-HCG levels < 2000 IU/l could be treated with expectant management [9,22]. Joshua H et al. (2014) reviewed the management of ectopic pregnancy, and they indicated that β-hCG values in approximately 99% of viable intrauterine pregnancies increase by about 50% within 48 hours. The remaining 1% of patients have a slower rate of increase, and these patients may have pregnancies that are misdiagnosed as ei- ther nonviable or necessitating expectant management [5].

In addition, in recent years, information and communication tech- nologies (ICT) have been added to traditional diagnostic technol- ogies. Artificial intelligence (AI) is being applied in the clinical field and has been implemented in various other fields [23]. AI al- gorithms are computational models that attempt to solve problems that cannot be solved using statistical methods [24]. Therefore, the development of AI algorithms that can help predict or classify dis- eases from a knowledge base can be applied in various ways as clinical decision support systems (CDSS).

CDSSs based on computational techniques have been developed in the obstetrical field to predict pregnancy outcomes [25]. There have been various reports on the prediction of emergency cesarean section, some using random forests and some using CNNs. One problem with AI technologies is that they have been referred to as “black boxes,” or systems whose inputs are not visible to the user. However, in recent years, explainable AI (XAI) has been de- veloped [26]. De Ramon Fernandez et al. constructed a decision support system for ectopic pregnancy using three algorithms, in- cluding a support vector machine (SVM)6. They reported that the accuracy, sensitivity, and specificity in SVM were 96.1%, 96%, and 98%, respectively [6].

Although these algorithms are highly accurate, the problem is that they are black boxes that do not know what risk factors are involved in the final decision. To overcome these problems, we constructed a model to support diagnosis and treatment decisions for ectopic pregnancy using a random forest, which is an algorithm based on visible decision trees.

A random forest model is a type of machine learning that only analyzes input variables and then outputs appropriate decisions. The reason for the decision is a black box, but the basis for the decision is simply expressed as a decision tree in order to provide as much explanation as possible. In the random forest model, input values for the candidate explanatory variables in the database are X1, X2,…,Xn. The number of explanatory variables is chosen in random combinations, and a decision tree, such as Tree1, is con- structed. The same process is repeated to build N decision trees. In each decision tree, target variables are searched and obtained as results. Each decision tree is then constructed by the Classification and Regression Trees (CART) algorithm; however, the decision tree method has one drawback in that it can be prone to overlearn- ing [14]. Random forests have the advantage of being less prone to overtraining because the model is built by ensemble learning [27]. Moreover, in a random forest, explanatory variables are randomly selected to generate multiple relatively simple decision trees, and the output of each decision tree is eventually averaged in order to reduce variance and to make predictions with a lower error [28]. In addition, we used explanatory variable importance to determine which risk factors contribute to the decision support model we de- veloped. This method is not a decision tree analysis that elucidates the decision process, which has been used in the past, but an anal- ysis using machine learning. As well, it was written only to clarify the basis for the decision itself. The explanatory variable impor- tance is a quantitative expression of the importance of explanatory variables using the remaining data that were not used for training in the random forest [15]. The explanatory variable importance has been described in various scientific papers as an analysis of factors in diagnosis [29]. Furthermore, as a discussion of the weeks of gestation and the results of this study, Daniela Carusi et al. report that an ectopic pregnancy is possible if no fetal sac is seen in the uterus at 6 weeks of gestation, counting from the date of ovulation or implantation [30]. In this study, the -2 Diagnosed pregnant day attribute accounted for 22.8% of the explanatory variable impor- tance (total of 100%). The -1 Diagnosed pregnant days and Diag- nosed pregnant days attributes were the top explanatory variables at 6.1% and 5.6%, respectively.

On the other hand, abdominal bleeding, vaginal bleeding, and ab- dominal mass were the explanatory variables of the importance of less than 1%. These were variables that previous findings could have provided a basis for judgment, however, the results of this study indicated that they were less critical variables [31,32].

As a result of examining the explanatory variables in this mod- el, the most important one was found to be -2 Diagnosed HCG, followed by the -2 Diagnosed pregnant days, and then Diagnosed HCG. This indicates that not only are HCG level, symptoms, and ultrasound findings at the time of diagnosis important, but so is the HCG level and its trend before diagnosis.

There are limitations to this study, and generalizability is the main one. In order to maintain generalizability, this data was subjected to a fivefold cross-validation. As well, since the model was con- structed using data from a single facility and tested at that facility, further model construction needs to be considered in the future.

Conclusion

This model using a random forest was superior to the judgment of specialists in the field. Moreover, Moreover, this research is new not only in its use of artificial intelligence to predict treatment methods for ectopic pregnancies but also in the fact that it has pre- sented a numerical model involving multiple risks that, until now, have been judged empirically by humans. It is the first report of its kind to build a model that predicts the treatment of ectopic preg- nancy using a random forest. In the future, this study may help in building and elucidating a larger, more comprehensive prediction system for ectopic pregnancy.

Acknowledgement

We appreciate the contributions to this study made by Yoichi Ha- yashi at Meiji University, Department of Computer Science.

References

1. Farquhar, C. M. (2005). Ectopic pregnancy. Lancet, 366(9485),583-591.
2. Rana, P., Kazmi, I., Singh, R., Afzal, M., Al-Abbasi, F. A., Aseeri, A., ... & Anwar, F. (2013). Ectopic pregnancy: a re­view. Archives of gynecology and obstetrics, 288, 747-757.
3. Jurkovic, D., & Mavrelos, D. (2007). Catch me if you scan: ultrasound diagnosis of ectopic pregnancy. Ultrasound in ob­stetrics & gynecology, 30(1), 1-7.
4. Condous, G. (2006). Ectopic pregnancy: risk factors and diag­nosis. Australian family physician, 35(11).
5. Barash, J. H., Buchanan, E. M., & Hillson, C. (2014). Diag­nosis and management of ectopic pregnancy. American family physician, 90(1), 34-40.
6. Fernández, A. D. R., Fernández, D. R., & Sánchez, M. T. P. (2019). A decision support system for predicting the treatment of ectopic pregnancies. International journal of medical infor­matics, 129, 198-204.
7. Huang, S., Cai, N., Pacheco, P. P., Narrandes, S., Wang, Y., & Xu, W. (2018). Applications of support vector machine (SVM) learning in cancer genomics. Cancer genomics & pro-teomics, 15(1), 41-51.
8. Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019). Artificial intelligence, bias and clinical safety. BMJ Quality & Safety, 28(3), 231-237.
9. Breiman, L. (2001). Random forests. Machine learning, 45,5-32.
10. Efron, B., & Tibshirani, R. (1991). Statistical data analysis in the computer age. Science, 253(5018), 390-395.
11. Daponte, A. L. E. X. A. N. D. R. O. S., Pournaras, S., Zintzaras, E., Kallitsaris, A., Lialios, G., Maniatis, A. N., & Messinis, I.E. (2005). The value of a single combined measurement of VEGF, glycodelin, progesterone, PAPP-A, HPL and LIF for differentiating between ectopic and abnormal intrauterine pregnancy. Human reproduction, 20(11), 3163-3166.
12. Clayton, H. B., Schieve, L. A., Peterson, H. B., Jamieson, D. J., Reynolds, M. A., & Wright, V. C. (2006). Ectopic preg­nancy risk with assisted reproductive technology procedures. Obstetrics & Gynecology, 107(3), 595-604.
13. Murray, H., Baakdah, H., Bardell, T., & Tulandi, T. (2005). Diagnosis and treatment of ectopic pregnancy. Cmaj, 173(8), 905-912.
14. Rigatti, S. J. (2017). Random forest. Journal of Insurance Medicine, 47(1), 31-39.
15. Ellies-Oury, M. P., Chavent, M., Conanec, A., Bonnet, M., Picard, B., & Saracco, J. (2019). Statistical model choice in­cluding variable selection based on variable importance: A relevant way for biomarkers selection to predict meat tender­ness. Scientific Reports, 9(1), 10014.
16. Blanchet, L., Vitale, R., van Vorstenbosch, R., Stavropoulos, G., Pender, J., Jonkers, D., ... & Smolinska, A. (2020). Con­structing bi-plots for random forest: Tutorial. Analytica chim-ica acta, 1131, 146-155.
17. Thornton, J. (2020). Women are at serious risk of harm from late diagnosis of ectopic pregnancy. BMJ: British Medical Journal (Online), 368.
18. Breeze, C. (2016). Early pregnancy bleeding. Aust Fam Phy­sician, 45(5), 283-286.
19. Belics, Z., Gérecz, B., & Csákány, M. G. (2014). Early di­agnosis of ectopic pregnancy. Orvosi hetilap, 155(29), 1158-1166.
20. Scibetta, E. W., & Han, C. S. (2019). Ultrasound in early preg­nancy: viability, unknown locations, and ectopic pregnancies. Obstetrics and Gynecology Clinics, 46(4), 783-795.
21. Visconti, K., & Zite, N. (2012). hCG in ectopic pregnancy. Clinical obstetrics and gynecology, 55(2), 410-417.
22. Rodrigues, S. P., de Burlet, K. J., Hiemstra, E., Twijnstra, A. R., van Zwet, E. W., Trimbos-Kemper, T. C., & Jansen, F. W. (2012). Ectopic pregnancy: when is expectant management safe?. Gynecological Surgery, 9, 421-426.
23. Keskinbora, K. H. (2019). Medical ethics considerations on artificial intelligence. Journal of clinical neuroscience, 64, 277-282.
24. Diniz, P. H., Yin, Y., & Collins, S. (2020). Deep learning strat­egies for ultrasound in pregnancy. European Medical Journal. Reproductive health, 6(1), 73.
25. Lipschuetz, M., Guedalia, J., Rottenstreich, A., Persky, M. N., Cohen, S. M., Kabiri, D., ... & Sompolinsky, Y. (2020). Prediction of vaginal birth after cesarean deliveries using ma­chine learning. American journal of obstetrics and gynecolo­gy, 222(6), 613-e1.
26. Nagayasu, Y., Fujita, D., Ohmichi, M., & Hayashi, Y. (2022). Use of an artificial intelligenceâ?based rule extraction approach to predict an emergency cesarean section. International Jour­nal of Gynecology & Obstetrics, 157(3), 654-662.
27. Feng, Z., Mo, L., & Li, M. (2015, August). A Random For-est-based ensemble method for activity recognition. In 2015 37th Annual International Conference of the IEEE Engineer­ing in Medicine and Biology Society (EMBC) (pp. 5074-5077). IEEE.
28. Ishwaran, H., & Lu, M. (2019). Standard errors and confi­dence intervals for variable importance in random forest re­gression, classification, and survival. Statistics in medicine, 38(4), 558-582.
29. Murray, K., & Conner, M. M. (2009). Methods to quantify variable importance: implications for the analysis of noisyecological data. Ecology, 90(2), 348-355.
30. Carusi, D. (2019, March). Pregnancy of unknown location: Evaluation and management. In Seminars in Perinatology (Vol. 43, No. 2, pp. 95-100). WB Saunders.
31. Hendriks, E., Rosenberg, R., & Prine, L. (2020). Ectopic preg­nancy: diagnosis and management. American family physi­cian, 101(10), 599-606.
32. Biggs, W. S., & Marks, S. T. (2016). Diagnosis and manage­ment of adnexal masses. American family physician, 93(8), 676-681.