Higher than expected data request rate

  • 4
  • Question
  • Updated 6 years ago
I am requesting temperature data by API from wunderground at a rate of once per 5 minutes to populate my pvoutput, but my usage keeps showing >500 requests per day. How can I reduce the number of data requests? / why are there so many apparent requests? I would expect 24*12 ie 288 per day?

Thanks.
Photo of Steve

Steve

  • 4 Posts
  • 0 Reply Likes

Posted 6 years ago

  • 4
Photo of Brendan Hayes

Brendan Hayes, Official Rep

  • 962 Posts
  • 123 Reply Likes
Please submit a Support ticket with your API key so we may investigate your individual case further.
Photo of Steve

Steve

  • 4 Posts
  • 0 Reply Likes
Seems to have resolved itself, am no longing burning ?evaporating raindrops!!
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
I'm having this same problem.

I thought maybe it was because I am still in development & refreshing my page a lot to see changes.

But I just got a notice that I am over my max limit for for today. I made a point to check my analytics all day as I was working on the site to make sure I wasn't doing to much and I looked fine.

And my Google Analytics says the only person that visited today was me, so I know it's not a traffic issue.

But my Wunderground analytics says that I went over on the 24th, which hasn't even started yet here in California. And I haven't even been sitting at the computer for the last hour.

I left a few windows open that call the API. Are they just refreshing really quickly or something?

I've used all my raindrops already. I'm afraid to work on it anymore. Ha.
Photo of Steve

Steve

  • 4 Posts
  • 0 Reply Likes
Ha - THOUGHT it had resolved itself, but problem still persists. I am not doing any development/testing just taking a single temperature reading every ?5 minutes to display on my pvoutput.

Don't know wahat is causing all these extra requests, though I still seem to able to get the data I need, even though all my raindrops are gone and I don't get any new ones (because of the >500 requests). ie haven't been blocked as yet. So I am letting it lie for now. - hey, it's a free service, don't want to moan too much ;) - can see that it might cramp YOUR work though. Don't know much about APIs, just that I needed one to fetch the data!!

Must check if I have multiple windows open - could be a starting point to investigate, as you mentioned, redheadkelly.

I guess wunderground is not California time-based; same situation with pvoutput, as it's based in Australian timezone for day start.

Steve.
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
I'm seeing the same issue with my stats. It shows me at 32 requests with a max of 5/minute so far for today and there is no way my app is getting hit that often yet. It used one of my raindrops yesterday, but I'm sure it will happen again today based on the incorrect reporting.

I have been scouring my access logs to see if there was any bot activity or anyone hitting the site repeatedly or scraping my results and there is no evidence of the activity.
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
Right? Weird.

I scoured my logs, too, but can't find anything.

I did open a support ticket as suggested. I don't really know if anyone did anything on it. Then I thought that the problem had been resolved when I turned off one of my plug ins.

Now it seems to be back, though. I see the same thing happened to Steve here. It went away & then it came back.

I sent another email to on my support ticket number, but I haven't heard back yet.

I'm so sad. I'm not really a programmer & it took me a long time to figure out how to use the API. I love the page I came up with & I hate that sometimes people are seeing a whole bunch of error messages there.

Although, it's not a lot of people. Google Analytics says I've had 100 unique visitors in the last month. Ha. ... which is exactly why this should not be an issue, though!

Suggestions on what to do next???
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
I haven't received any reply to my service ticket yet either. This definitely looks like an error with Weather Underground's reporting. I'm using the API and was planning on upgrading my account if bandwidth pics up, but since the numbers are not accurate, I don't have much confidence in the system to do so. I don't know what the issue is. So this is all speculation. I just want the service to work as described so I can focus on building out other features (including location weather caching to reduce my API calls even further).

I've had only 24 requests for a .php file that would possibly be making a call to the API and my stats show that I have 59 API calls so far. That's more than double. Hopefully, I get a response soon with some info on what it might be.
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
I did get a response when I first submitted my ticket, but it was just a promise to look into it. I never got any feedback on what might be causing it or what I might do about it.

Right now, I'm just using the free version for my personal website. But I'm considering using it on a new website. If the new site ever got to the point of making money, I would surely upgrade to a paid wunderground account.

You're right, though. Until I have more understanding of this problem, I am holding off on putting it on my new site.
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
Yeah. It's disappointing how little supported this API is. I'm seeing much larger than normal results for today as well. Still no word from WeatherUnderground...

If I find a better weather API out there with a similar pricing structure that is better supported or work as expected, I'll pass on the info.
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
Actually, for the price, I was totally happy with the wunderground API. Ha. But since we started this conversation yesterday, I have deactivated both the page & the widget on my site that use it. So, I guess I'm getting what I'm paying for. Ha.

Yeah... I would totally be interested if there is another one out there. I looked around a lot & haven't found anything else, but that doesn't mean it's not there. :)
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
Yeah. The pricing model is fine. I am just suspicious of the misreported usage. I would rather the issue be resolved and continue to use this API. I'll post whatever helpful info I can. I'm going to continue looking at my app to see if there is something that would possibly call the service multiple times. I've already gone over it several times before though, so I'm not sure what exactly I'll find.
Photo of Gary Stark

Gary Stark

  • 8 Posts
  • 0 Reply Likes
I'm seeing exactly the same problem. Massive apparent usage in terms of hits/minute - way up on what I normally see. But the total number of requests served fails to match the hits/minute.

For instance, at 00:23 hours, I'm seeing that we are already over our max rate (1000/minute) at 1236 - which is exactly the same rate as the prior day. Our normal rate is around 150/minute.

At the same time, the total number of calls per day was only 406.

How do you get to the situation whereby you can make 1236 calls in a minute, yet only have a grand total of 406 calls for the day?

Something is very wrong here.
Photo of visualweather

visualweather

  • 4 Posts
  • 0 Reply Likes
I have a similar issue.

Max data rate is around a third of daily usage. Which seems really strange.

Also, I submitted a ticket about increasing max minute rate but no response?

Can someone from Weather Underground please let me know who to talk to to get a rate larger than 10,000 calls per minute? And also whether we can buy extra raindrops.

And the person who is listed - he didn't reply to emails. So I'm at a loss of how to get the rate increased before our app starts running into problems.
Photo of sebasnn

sebasnn

  • 9 Posts
  • 0 Reply Likes
Hi guys, im having a similar problem.

Since we need to update the weather conditions every hour and its not in production yet, we've been using the API a few times everyday since last week (30 uses a day, max) and inside the Api Statistics section it seems that we've queried it lots of times, for instance, yesterday I did not turn on the app and still the Statistics shows 80 usages.

The good thing is that the conditions Series (inside the graphic) seems to be working fine (or at least it shows as many hits as we've done), so, are we not understanding something regarding the Forecast series? Or its not working properly?

We've the same situation in both graphics (conditions working as excepted and forecast way above the predicted usage, even while offline, as u can see in the jpeg)


Other than that, the API is awesome!

TY guys in advance!
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
I just recently (Yesterday) received an email to my ticket I submitted a month ago. There was no answer, just a message to submit a new ticket. Hopefully, this gets resolved. I'm only using the combined call, FWIW. I wrote a cacheing component to stop the duplicates from happening no matter what's causing them, but I still see a huge discrepancy in what's reported on WU vs. my apache access logs. I am checking right at the source to see if any call could be making it in.

I'll update this thread if there's any resolution to the issue.
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
Funny, I had completely disabled my wunderground page & widget when we were discussing it a month ago, but turned it on again yesterday to see if things have changed.

Clearly, DrinkingWeather, you're more qualified than I am to figure this out. I don't know how to do anything with my caching. So glad that you are on the job here.

I think I will open a new ticket, too. Maybe if they are flooded with tickets this week, they will see it as a priority.
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
Thanks for the confidence in my ability to figure it out. Hopefully, I won't disappoint. :) The more tickets they get about it, the more likely they are to look into it at least. I still haven't received any information back from them for my second request. Hopefully, it gets resolved before I'm able to really drive traffic to my site/app. I like the API and the pricing model. But this reporting has got to be fixed.
Photo of Brendan Hayes

Brendan Hayes, Official Rep

  • 962 Posts
  • 123 Reply Likes
I am so sorry, I had no idea this thread had gotten so long. I'm trying to get someone to look into it, and this thread certainly escalates the matter.
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
Oh thank you! Thank you! I would really love to turn my page back on!
Photo of Brendan Hayes

Brendan Hayes, Official Rep

  • 962 Posts
  • 123 Reply Likes
We believe we are on to something about how the max rate is reset at the end of the night. The daily stats look correct for the most part, but the max rate value might get stuck. We're logging requests over the weekend to see if we can pinpoint where it gets stuck. We'll continue working on the problem next week until it is cleared up. Thank you all for your patience.
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
There is a problem not only with the max rate, but also the daily count. I still have not received a reply to my support request. Once/if that happens, I am more than happy to provide details about any api requests that are actually coming from my site. My site gets very little traffic right now, so this small sample set should make it easier to match up requests with both systems.
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
This thread should probably be moved to "Problems" instead of "Questions". There are other threads there that are about the same thing. I'm still experiencing incorrect usage every day.

Can someone form wunderground provide me with details about all of my requests or contact me to help them troubleshoot the problem. I am more than happy to help fix this. I can provide apache logs, GA data, code samples, ... etc.

EDIT:
I just submitted a ticket: VPW-969606

I have not had any luck with submitting tickets in the past, but maybe this one will get better traction.
Photo of DrinkingWeather

DrinkingWeather

  • 16 Posts
  • 3 Reply Likes
Just an update, I've been receiving email communication from WU about this ticket. No resolution yet, but there is progress towards it. I'll be sure to post up the solution once it's known.
Photo of redheadkelly

redheadkelly

  • 19 Posts
  • 1 Reply Like
Just looking at my site again & wishing this was fixed. I have disabled everything wunderground just because it seems like this is a problem EVERY day. Hoping that the more I write here, the more attention this will get. Ha.
Photo of strydom

strydom

  • 8 Posts
  • 0 Reply Likes
I'm not sure if I'm having this problem or not -_- i have had 2 really high max(day) calls but i recon i could have clocked that up while i was trying to get it to look right on the page and everything that comes with it. I would say that wunderground should allow abnormal data requests for the first week or so, so that developers can get to grips with things. I have lost 2 raindrops already, clocked up 524 calls in one day and my site is privet! i don't know if this is correct but i have been refreshing my page a lot so maybe.
Anyway when i re-enable the weather settings on my site i will get back to you guys and let you know whats going on with my rates...
Photo of Brendan Hayes

Brendan Hayes, Official Rep

  • 962 Posts
  • 123 Reply Likes
Submit a ticket through our Tech Support with your api key and we can add more raindrops to help you through development.