QoS and traffic shaping is the best way to manage this kind of situation.
Make sure that real-time (latency sensitive) traffic has priority, or even just do something as basic as throttling certain ports/devices on the network.
Just keep in mind that you've mentioned the download speeds, but upload is something to consider as well.
Worst case scenario, keep mobile data on your smartphone as a backup.
This is why I change the settings to run for longer. 60-90 seconds usually.
Also, you have a crap upload speed.