Thanks for the suggestion!
There is a way to set sidebars, masonry etc.. for categories using filters, but it’s true there’s no Customizer settings for those yet.
The Blog Layout option in “Customize > Layout” includes categories.
The Masonry layout in the Blog add-on includes categories as well.
For specific widgets, a plugin like this is probably best: https://en-ca.wordpress.org/plugins/display-widgets/