Kernel

\[
\begin{eqnarray}
K(x,y) &=& \phi(x)^T \cdot \phi(y)\\
\phi(x) &\in & \cal{F}\\
& & \phi(x): \mbox{feature vector} \\
& & \cal{F}: \mbox{feature space}
\end{eqnarray}
\]

Joint Kernel

\[
\begin{eqnarray}
K_J(x,y) &=& \phi(v_x,h_x)^T \cdot \phi(v_y,h_y)\\
& & x = (v_x, h_x), y = (v_y, h_y): \mbox{variables with visible/hidden parts}\\
& & v_x, v_y: \mbox{visible variables}\\
& & h_x, h_y: \mbox{hidden variables}
\end{eqnarray}
\]

Marginalized Kernel

\[
\begin{eqnarray}
K_M(x,y) &=& E_{h_x,h_y}\left[ K_J(x,y) \right]\\
&=& E_{h_x}\left[\phi(v_x,h_x)\right]^T \cdot E_{h_y}\left[\phi(v_y,h_y)\right]\\
&=& \phi_M(v_x)^T \cdot \phi_M(v_y)\\
& & \phi_M(v_x) = E_{h_x}\left[\phi(v_x,h_x)\right]:\mbox{Marginalized Feature}
\end{eqnarray}
\]

Marginalized Count Kernel on HMM
Marginalized Count Kernel on SCFG

参考文献
[1] Koji Tsuda, Taishin Kin, Kiyoshi Asai, “Marginalized kernels for biological sequences”,
Bioinformatics, Volume 18, Issue suppl_1, 1 July 2002, Pages S268–S275
https://academic.oup.com/bioinformatics/article/18/suppl_1/S268/232208
[2] Taishin Kin, Koji Tsuda, Kiyoshi Asai,”Marginalized kernels for RNA sequence data analysis”,