Selecting system fonts is faster because your site request any fonts from external sources as compared to using non locally hosted google fonts where your site requests font files from fonts.googleapis.com.
But while system fonts are faster, the choices are very limited. But using google font shouldn’t slowdown the page speed that much. Especially if you optimize the site properly.
As Elvin mentioned, the fastest font would be system stack as it just uses the system font.
If you set the body font to system stack and leave everything else as inherit, that means they will all inherit from the body font which is system stack.