If we start selling in the EU, is it mandatory to sign up for VAT Services on Amazon or is this just an service offered by Amazon? Can we not register in other European countries externally and declare our EU sales outside of Amazon?
VAT service on Amazon is voluntary. You can register for VAT in all the relevant countries yourself, then do the returns yourself.
This thread might help: VAT Services on Amazon vs Manual VAT Registration
Also, the VAT calculation service where Amazon creates invoices on your behalf is also currently voluntary. With the EU VAT eCommerce package due for implementation in July 2021 I can see it either becoming mandatory or that you need to ensure all your products have the correct VAT product group.