Gaussian processes in combination with sequential Monte-Carlo methods have emerged as promising tools for offline nonlinear system identification. However, sometimes the dynamical system evolves in such a way that online learning is preferable. This paper addresses the online joint state estimation and learning problem for nonlinear dynamical systems. We leverage a recently developed reduced-rank formulation of Gaussian-process state-space models (GP-SSMs), and develop a recursive formulation for updating the sufficient statistics associated with the GP-SSM by exploiting marginalization and conjugate priors. The results indicate that our method efficiently learns the system jointly with estimating the state, and that the approach for certain scenarios gives similar performance as more computation-heavy offline approaches.