Kernel

$$\begin{eqnarray} K(x,y) &=& \phi(x)^T \cdot \phi(y)\\ \phi(x) &\in & \cal{F}\\ & & \phi(x): \mbox{feature vector} \\ & & \cal{F}: \mbox{feature space} \end{eqnarray}$$

Joint Kernel

$$\begin{eqnarray} K_J(x,y) &=& \phi(v_x,h_x)^T \cdot \phi(v_y,h_y)\\ & & x = (v_x, h_x), y = (v_y, h_y): \mbox{variables with visible/hidden parts}\\ & & v_x, v_y: \mbox{visible variables}\\ & & h_x, h_y: \mbox{hidden variables} \end{eqnarray}$$

Marginalized Kernel

$$\begin{eqnarray} K_M(x,y) &=& E_{h_x,h_y}\left[ K_J(x,y) \right]\\ &=& E_{h_x}\left[\phi(v_x,h_x)\right]^T \cdot E_{h_y}\left[\phi(v_y,h_y)\right]\\ &=& \phi_M(v_x)^T \cdot \phi_M(v_y)\\ & & \phi_M(v_x) = E_{h_x}\left[\phi(v_x,h_x)\right]:\mbox{Marginalized Feature} \end{eqnarray}$$

Marginalized Count Kernel on HMM
Marginalized Count Kernel on SCFG

参考文献
[1] Koji Tsuda, Taishin Kin, Kiyoshi Asai, “Marginalized kernels for biological sequences”,
Bioinformatics, Volume 18, Issue suppl_1, 1 July 2002, Pages S268–S275
https://academic.oup.com/bioinformatics/article/18/suppl_1/S268/232208
[2] Taishin Kin, Koji Tsuda, Kiyoshi Asai,”Marginalized kernels for RNA sequence data analysis”,