In my research installing Jetpack without disabling the modules *you don’t need* will indeed slow down your site. Yet, everything I’ve seen written recently seems to be based on the *old* jetpack (fair enough, it was pretty bad), yet now it’s better written, you can disable any and all modules (and there’s quite a few useful ones in one plugin) you don’t need, and doing so is pretty snappy (sure, if you’re number crunching other options might win, but it’s not like most infer).