Higher than expected data request rate

  • 4
  • Question
  • Updated 7 years ago
I am requesting temperature data by API from wunderground at a rate of once per 5 minutes to populate my pvoutput, but my usage keeps showing >500 requests per day. How can I reduce the number of data requests? / why are there so many apparent requests? I would expect 24*12 ie 288 per day?

Thanks.
Photo of Steve

Steve

  • 4 Posts
  • 0 Reply Likes

Posted 7 years ago

  • 4
Photo of Brendan Hayes

Brendan Hayes, Official Rep

  • 962 Posts
  • 124 Reply Likes
We are still working on the problem where the max rate data gets carried into the next day. We know it happens right at midnight, if the API key is hit at the moment the day is turning from one into the next. We just need to nail down why its happening. We are still working on it, just trying to get the resources to get the fix out the door. If anyone still has the problem, submit a ticket to tech support with your api key and we can add more raindrops to your account to make up for the overage days.
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
I have had some email communication with William regarding my ticket, but have not heard anything since November 19th. My ticket ID is: VPW-969606.

I am seeing issues with traffic being reported higher than what is actually being requested throughout the day. My raindrops have been increasing (likely due to link click-throughs or something like that) and since the total requests are still low, I haven't had to use any. But I'm not sure what the report will show once I have more traffic.

If the multiple calls are a result of something I'm doing, I am happy to change my process to make it work. I have looked through my logs and the code several times though and can't see anything that would make these multiple calls.
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
New approach here. I just switched ISPs & now I have access to logs that I didn't before.

My Raw Access Log for today has 1519 entries in it.

I isolated 149 distinct IP addresses.

Of the 149 IP addresses, most originate in the US & China.
# US United States 59
# CN China 41
# DE Germany 6
# IE Ireland 6
# UA Ukraine 6
# CA Canada 5
# GB United Kingdom 4
# NL Netherlands 3
# VN Viet Nam 3
etc.

This is what Google Analytics says about today:
Ireland 5
New Zealand 3
United States 3
Germany 1
12 visitors total.

I'm assuming that this is part of my issue. There is clearly lots of activity that is not actual visitors.

I added a whole list of Chinese IP Addresses to my .htaccess to try to keep them out, but they are obviously one step ahead of me.

How do I keep this traffic out of my site so that it is not affecting my API call numbers?
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
Something to remember about the access log is that it lists requests for not only your page, but for all elements on the page as well (images, CSS, javascript, etc). So you'll want to look at the individual entries and see how many requests for your page (or web service / API) you are getting. I found that GA was 100% accurate on page counts with what I found in my logs.

BTW, I still haven't heard anything else back about my ticket. :(
Photo of Brendan Hayes

Brendan Hayes, Official Rep

  • 962 Posts
  • 124 Reply Likes
I think we have the bug fixed! The bug related to hits per minute max rate that was carrying the previous day's max rate into the next day. Please let me know if anyone see that problem continue. Thanks for the patience while we got through the problem, sorry it took so long.
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
Thanks for the reply DrinkingWeather.

So, if my logs say one thing & my GA says another, what does it mean??? Any ideas?
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
@redheadkelly:
I would have to see what your logs say to tell for sure. One thing to ignore in your logs are any requests for images or a css file since those are not going to be making a call to the api. I don't know how your using the API so I can't give you much more specific help unfortunately.

@WeatherUnderground
I still have the same invalid reporting happening. Can you confirm someone is still looking at my ticket?