Increase the TTL of your DNS records | The WeatherAPI Community Customer Community

Hello, I recently setup a PWS - it is a Ambient Weather WS-2902. I monitor the DNS traffic on my network, and noticed in the less then 24 hours DNS requests for rtupdate.wunderground.com has passed 60% of all DNS records on my network - so far at 18,936 hits.

Looking more into the issue, I noticed that rtupdate.wunderground.com has a DNS TTL of 60 seconds. I understand that part of the problem is likely Ambient Weather WS-2902 not doing its own DNS caching. However, with a 60 seconds TTL, it wouldn't matter much even if it were. 60 seconds is very short. Please increase your TTL to at least 300 seconds (5 minutes). Not only would this reduce the load on your own DNS servers, but it would also reduce the network load of people with a PWS by allowing ISP's or other DNS providers to maintain a cache of greater then 60 seconds. It would also reduce latency by allowing the DNS cache to be effective. Of course greater then 5 minutes would be even better, but I understand the desire to keep cache time low in case a DNS entry needs modification.

RackSpace has a very logical best practices for DNS TTL here: https://support.rackspace.com/how-to/about-ttl-best-practices/

They recommend using a TTL of 24 hours and then lowering it to 5 minutes the day before any infrastructure changes.

Google.com uses a TLL of 5 minutes. Facebook.com also has a TTL of 5 minutes. Microsoft.com uses a TTL of 1 hour - which I personally think is the best of both worlds between caching and the desire to quickly make changes. Interestingly enough, facebook's API endpoints (api.facebook.com and graph.facebook.com and graph.instagram.com) also use a TTL of 1 hour - which makes a lot of since. The primary domains will be used when a user types it in on a browser, but the API endpoints are almost constantly being hit in the background and so benefit the most when it comes to cached DNS records. I think this same logic would easily apply to rtupdate.wunderground.com - as you can tell its being hit very often.

Thank you for your time and consideration

Daniel

3 Posts
0 Reply Likes

Posted 1 year ago

1 year ago

Tim Roche, Official Rep

337 Posts
44 Reply Likes

HI Daniel, our ingestion is hosted on Amazon's AWS platform using their ELBs. Those load balancers may change IP addresses frequently so we keep the dns ttl low. It doesn't seem like it would help you much in your situation anyway. If you are seeing more than 1440 dns lookups per day (the number of 60 second blocks in a day) , your application is not obeying ttl at all

Comment

1 year ago

Daniel

3 Posts
0 Reply Likes

Hello Tim, thank you for your response.

Yes, I see that rtupdate.wunderground.com is a CNAME record for the ELB (prod-pws-ng-ingest-368531364.us-west-2.elb.amazonaws.com) and AWS says:

> Elastic Load Balancing uses a TTL setting on the DNS record of 60 seconds

However, when a CNAME record is returned in response to an A record query, the TTL of the original record is used. In this case it would be the TTL of rtupdate.wunderground.com. Since you are using DNS as load balancing, I definitely understand the desire to keep it low. But is 5 minutes really that long? Even if you have auto-scaling abilities, many times the warmup period of the services is as long as 5 minutes.

Of course you are also right that my Ambient Weather WS-2902 does not seem to be doing any type of DNS caching. I have a local DNS server that is able to do caching, but with a 60 second TTL, it doesn't help much - 300 seconds (5 minutes) would go a long way to help. I'll see how responsive Ambient Weather support is.

Comment

1 year ago

Tim Roche, Official Rep

337 Posts
44 Reply Likes

Appreciate the feedback. Thanks

Comment

1 year ago

Daniel

3 Posts
0 Reply Likes

I discussed this with Ambient Weather, and they will be creating a firmware update to do DNS caching

(Edited)

Comment