You aren't alone. In the Macadmins Slack team there was some discussion and it turns out it seems to be failing for others too. Here's my error after downloading 18+ GB:
[2017-05-10 18:35:37.419] [Exception] System.AggregateException: One or more errors occurred. ---> System.IO.FileNotFoundException: /Users/cmcintosh/Library/Caches/XamarinInstaller/Universal/downloads/repository-10.xml does not exist
Unfortunately there's no Visual Studio channel so we're limited on forums to complain to unless you sign up for a Microsoft account and post comments on their instructions page: https://docs.microsoft.com/en-us/visualstudio/mac/create-offline-installer
