A Stochastic Segmentation Model for Categorical and Continuous Features of various biological sequential

Abstract

Nowadays, Hidden Markov Model (HMM) has been widely used in analysis of various biological data for both smoothing and clustering. However, characterizing each hidden state by a single distribution, the classical HMM might have some limitations on the data whose hidden state is composed by a mixture of distributions (Heng Lian et al., 2006). To address this issue, we proposed a new stochastic segmentation model and an associated estimation procedure that has attractive analytical and computational properties. We combined the forward and backward filter together based on Bayes' theorem to calculate the posterior mean and variance. Besides, we developed an expectation-maximization (EM) algorithm to estimate the hyper-parameters. Furthermore, we utilized a bounded complexity mixture (BCMIX) approximation whose computational complexity is linear in sequence length. Another important feature of this segmentation model is that it yields explicit formulas for posterior means and probability of categorical states, which can be used to make inference on both categorical and continuous aspects of the data. Other quantities relating to the posterior distribution that are useful for making confidence assessments of any given segmentation can also be estimated by using our method. We perform intensive simulation studies (1) to compare the Bayes and BCMIX estimates (2) to evaluate the BCMIX estimates in terms of sum square error, Kullback-Leibler divergence and the identification ratio of true segments. We also applied our model on two biological data sets: (1) reduced representation bisulfite sequencing (RRBS) data (A.Molaro et al., 2011) (2) ENCODE Nimblegen tilled arrays (Sabo et al., 2006). Our model shows good performance on segmentation of these two sequential data. In RRBS data it can further help identify differential methylation region (DMR) while in microarray data it can discover the DNAsel Hypersensitive Sites (DHSs).