Gateway used for pool(s) or Acting as Effective Gatway or assigned to firewall(sic)
rule cannot be deleted
So I went to Firewall -> Manage Firewall and removed all the firewall rules relating to virtual hosts which mentioned that gateway. Still no dice. Then removed those virtual hosts too. Still no. Then removed the relevant host names from Firewall -> Host -> Manage. Still no. Then, during a support chat with Tech support, thought about other firewall rules, outgoing rules for eg. Lan to Wan and so on. Went through those, in a couple of them the old gateway was the Primary. Modified the rules suitably, then the gateway could be deleted.