73 Predicting population-level vulnerability among pregnant women APPENDIX 1. Methodology Sensitivity analyses: XGBoost and Lasso regression XGBoost (extreme gradient boosting) is a machine learning technique that iteratively builds multiple shallow decision trees (1). Similar to RF, it is a flexible algorithm without assuming a functional form. Logistic regression, on the other hand, does assume a strong functional form, i.e. a linear relation between the independent variables and log odds. Logistic regression is a standard approach for binary classification with a long history in literature. The logistic regression analysis was conducted with lasso penalty to shrink coefficients towards zero such that less important variables are left out the model (2). Nested cross-validation The three techniques RF, Lasso and XGBoost each have their own set of hyperparameters that need to be chosen for the models. For RF, the default hyperparameter settings in the R-package ‘ranger’ (3) were used, as these default settings generally yield good performance. The parameter to choose for Lasso (R-package ‘glmnet’ (4)) was the lambda, which defines the penalty, and for XGboost (R-package ‘xgboost’ (5)) the number of trees and tree-depth. For Lasso and XGBoost we used cross-validation to choose the hyperparameters. In addition, as the models predict the probability of multidimensional vulnerability, we need to choose the threshold at which all predicted probabilities above that threshold are classified as multidimensional vulnerable ‘yes’ (and as ‘no’ below that threshold). To choose the hyperparameters and threshold probability, and finally to assess the performance of the models, we used nested-cross validation. Firstly, the dataset of 4172 women was split into six folds: 5 parts train-set, 1 part test-set (outer loop). Secondly, in the nesting step (inner loop), each train set from the outer loop was again split into five folds: 4 parts train-set, 1 part validation-set. During each split, we made sure that the percentage of multidimensional vulnerability was approximately equal in each part. Firstly, using the cross-validation of the inner-loop, we chose the hyperparameters: for Lasso, we chose the average lambda across the five validation folds and for XGBoost we selected the hyperparameters for which the average AUC over the five folds was highest. Secondly, using the defined hyperparameters, and the same inner loop, we selected the threshold probability that yielded the highest F1-value on the validation set (averaged over five validation folds). Thirdly, by utilizing both the selected hyperparameters and average optimal thresholds, we calculated the F1-value on the test set of the outer loop that has not been used in selecting hyperparameters and threshold. This three-step process was repeated for the 6 folds of the outer loop, yielding the F1-measure of the model averaged over the 6 test-sets. Using the average optimal thresholds (as well as the defined hyperparameters), we fitted the model one last time on all data. The final model can be utilized for predicting outcomes on new datasets. We used the final RF-model in our next analyses. 3
RkJQdWJsaXNoZXIy MTk4NDMw