2017.06.03 14:18:42 (870978045867425794) from Daniel J. Bernstein, replying to "James Pirruccello 🌁 (@jpirruccello)" (870964791904436225):
The DNS TTL was 10 minutes for a while before this. Surely the ahrefs misbehavior is another example of bad software-engineering incentives.
2017.06.03 12:36:40 (870952369017810945) from Daniel J. Bernstein:
We built two shiny new servers for web service. Changed DNS records. A day later http://ahrefs.com is still pestering the old servers.
2017.06.03 13:26:02 (870964791904436225) from "James Pirruccello 🌁 (@jpirruccello)":
1) You haven't excluded ahrefs with robots.txt yet? 2) How long was the DNS TTL set before the move? If not 5min, may be propagation delay.