The calibration of computational models of physical systems typically assumes that the computational model replicates the exact mechanism behind data generation. As a result, calibrated model parameters are often biased, leading to deficient predictive skills. This work will present a Bayesian inference framework for representing, quantifying, and propagating uncertainties due to model structural errors by embedding stochastic correction terms in the model. The embedded correction approach ensures physical constraints are satisfied, and renders calibrated model predictions meaningful and robust with respect to structural errors over multiple, even unobservable, quantities of interest. The physical inputs and correction parameters are simultaneously inferred via surrogate-enabled Markov chain Monte Carlo. With a polynomial chaos characterization of the correction term, the approach allows efficient decomposition of uncertainty that includes contributions from data noise, parameter posterior uncertainty, and model error. The developed structural error quantification workflow is implemented in UQ Toolkit (www.sandia.gov/uqtoolkit). We demonstrate the key strengths of this method on both synthetic examples and practical engineering applications.