TR2020-166

Spatio- Temporal Graph Scattering Transform


    •  Chen, S., Li, M., Chen, X., Zhang, Y., Wang, Y., Tian, Q., "Spatio- Temporal Graph Scattering Transform", IEEE Transactions on Pattern Analysis and Machine Intelligence, December 2020.
      BibTeX TR2020-166 PDF
      • @article{Chen2020dec,
      • author = {Chen, Siheng and Li, Maosen and Chen, Xu and Zhang, Ya and Wang, Yanfeng and Tian, Qi},
      • title = {Spatio- Temporal Graph Scattering Transform},
      • journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
      • year = 2020,
      • month = dec,
      • url = {https://www.merl.com/publications/TR2020-166}
      • }
  • Research Areas:

    Computer Vision, Machine Learning

Although spatio-temporal graph neural networks have achieved great empirical success in handling multiple correlated time series, they may be impractical in some real-world scenarios due to a lack of sufficient high-quality training data. Furthermore, spatio-temporal graph neural networks lack theoretical interpretation. To address these issues, we put forth a novel mathematically designed framework to analyze spatio-temporal data. Our proposed spatio-temporal graph scattering transform (ST-GST) extends traditional scattering transforms to the spatiotemporal domain. It performs iterative applications of spatio-temporal graph wavelets and nonlinear activation functions, which can be viewed as a forward pass of spatio-temporal graph convolutional networks without training. Since all the filter coefficients in ST-GST are mathematically designed, it is promising for the real-world scenarios with limited training data, and also allows for a theoretical analysis, which shows that the proposed ST-GST is stable to small perturbations of input signals and structures. Finally, our experiments show that i) ST-GST outperforms spatio-temporal graph convolutional networks by an increase of 35% in accuracy for MSR Action3D dataset; ii) it is better and computationally more efficient to design the transform based on separable spatio-temporal graphs than the joint ones; and iii) the nonlinearity in ST-GST is critical to empirical performance.