We propose a new method for cloth digitalization. Aiming for accurate cloth digitalization, we deviate from existing methods which learn from data captured under relatively casual settings. We propose to learn from data captured in strictly tested measuring protocols, and find plausible physical parameters of the cloths. However, such data is currently absent, so we first propose a new dataset with accurate cloth measurements. Further, the data size is considerably smaller than the ones used in current deep learning, due to the nature of the data capture process. To be able to learn from small data, we propose a new Bayesian differentiable cloth model to estimate and capture the complex material heterogeneity of real cloths. It can provide highly accurate digitalization from very limited data samples. Through exhaustive evaluations and comparisons, we show our method is accurate in cloth digitalization, efficient in learning from limited data samples, and general in capturing material variations.