In that most models adopted in geological and geotechnical engineering are abstractions of the real world, discrepancies between model predictions and field observations are generally a norm, rather than an exception (Ang and Tang 1984; Cheung and Tang 2005). With advances in mathematics and computational power, researchers could be afforded to derive complex models, which offer an opportunity to improve the fidelity in the model prediction; as such, the models in geological and geotechnical engineering are becoming more complex and sophisticated. On the other hand, simple and robust models are preferred by engineers in practice, as the complex and sophisticated models are more difficult to apply and the model predictions might be less robust. Thus, a dilemma exists between the choice of a complex model for fidelity and that of a simple model for usability and robustness. The issue of model fidelity and robustness is examined herein at a deep level. Generally, the residual (i.e., the discrepancy between model prediction and field observation) of a complex model is much lower compared to that of a simple model because of its higher fidelity. However, the parameters of the complex model are more difficult to characterize accurately and precisely, as a larger number of model parameters are involved in this complex model. This issue would be especially profound in the situation where only limited test data are available. Further, test errors (i.e., discrepancies between measurements and in situ performances) always exist. The limited availability of test data and the randomness of test errors can lead to difficulty in characterizing model parameters, and as such, the characterized model parameters are often uncertain. Thus, it is more appropriate to characterize the model parameters in a probabilistic manner, and the uncertainty in model parameters must be explicitly considered in the selection of solution models (Chiu et al. 2012; Zhang et al. 2014; Ching and Wang 2016). Note that at a given (or same) level of variation (or uncertainty) in the model parameters, the variability in the model prediction obtained from the complex model tends to be larger than that obtained from the simple model. For example, an overfitted model could be more sensitive to the measurement and modeling error (Yuen and Mu 2015). In other words, the complex model is generally less robust against (or more sensitive to) the uncertainty in the model parameters. Moreover, given the same amount of data, the characterized uncertainty in the model parameters of the complex model might be larger than that of the simple model, as the increase in the number of model parameters could lead to increased difficulty in the model parameters characterization. Thus, the predictive ability of the complex model is strongly degraded by the uncertainty in the model parameters.