The UQ Toolkit (UQTk, sandia.gov/uqtoolkit) is a collection of libraries, scripts and command-line tools for uncertainty quantification (UQ) in computational models. It offers a wide range of intrusive and non-intrusive methods for forward uncertainty propagation, as well as Bayesian methods for inverse UQ. In this talk, we will highlight the inverse modeling components of UQTk. In particular, the core Markov chain Monte Carlo capabilities, together with a higher-level model calibration library, will be detailed. The software enables Bayesian inference of computational model parameters, while allowing for flexible user-defined components such as likelihoods, priors and forward models. An important feature of the software is the capability to perform Bayesian inference with model structural error estimation. The core libraries are implemented in C++, and a Python interface is available for easy prototyping and incorporation in UQ workflows. We will demonstrate the embedded model error methodology, enhanced with surrogate modeling and uncertainty propagation with polynomial chaos, as well as its software implementation on a few DOE SciDAC relevant applications.