Could you please give a bit more in-detail of the role of a theme in speed/seo vs the role of the builder? Do they both influence speed? or only one?

I know Divi is not the fastest (it's not that bad either), but if I use the builder, doesn't it still load all the same assets that cause the speed issues? (bigger style files, etc.)

Any other benefits to using your theme? + is it possible to go without any builder whatsoever ? For more minimal personal website for example (without knowing how to code php).