Mean Field Games (MFG) have emerged as a viable tool in the analysis of large-scale self-organizing networked systems. In particular, MFGs provide a game-theoretic optimal control interpretation of the emergent behavior of noncooperative agents. The purpose of this paper is to study MFG models in which individual agents obey multidimensional nonlinear Langevin dynamics, and analyze the closed-loop stability of fixed points of the corresponding coupled forward-backward PDE systems. In such MFG models, the detailed balance property of the reversible diffusions underlies the perturbation dynamics of the forward-backward system. We use our approach to analyze closed-loop stability of two specific models. Explicit control design constraints which guarantee stability are obtained for a population distribution model and a mean consensus model. We also show that static state feedback using the steady state controller can be employed to locally stabilize a MFG equilibrium.