API returning useless PWS StationID lists

  • 2
  • Problem
  • Updated 5 months ago
api.weather.com/v3/location/near?geocode=lat,lng&product=pws
When you call the location service in the API as above and ask for pws stations near a certain point it is supposed to return pws station names and IDs near that point.  It was working well enough early on but in the last few days the stations it returns are virtually all inactive.  They have NULL updateTimeUtc fields and return no data (204) when you try to get their conditions.

This is a widespread problem.  Even when I put my own location into the query I get back the "dead" stations in my area.  My own active station and others around me are not listed.  Even your website is suffering.  When viewing my own station on WU, nearby active stations are not showing up in the list of stations I can change to.

What happened?  It appears you have a major pws database issue affecting not only the API but your own website.  Is WU aware of this problem?  Is it being addressed?
Photo of Emery Wooten

Emery Wooten

  • 25 Posts
  • 14 Reply Likes

Posted 5 months ago

  • 2
Photo of elee

elee

  • 11 Posts
  • 1 Reply Like
I have the same problem.

This error starts about a week ago. At first I thought it was a problem in my code but @Emery indicates the problem very well.

I am located in Costa Rica, so it seems to be a general problem in the database.
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 656 Posts
  • 92 Reply Likes
Okay, so I am coming late to the party. 

I ran a test on this, and what I found was that for the 10 PWSs returned in the area I checked, only 1 was reported as updating, and as it happened, that one was also passing QC.  I don't have a reference for this (I haven't run this call before), but I am guessing from these notes/responses, that's not the sort of percentage that you were seeing before?

So first, solutions:  at the moment, if you need 10 good PWSs, you're going to need to iterate on either the upload time or the QC value.  I know this is far from ideal, but it's a solution.

I have already reported the problem.  It may be part of a wider locations issue that we have been experiencing (and very busy with) for the past week or two.  I don't know for certain yet.  As I know more, I will post more here. 

--Victoria
Photo of Emery Wooten

Emery Wooten

  • 25 Posts
  • 14 Reply Likes
Thanks for looking into this.  I just did a test and am now seeing active and updating stations appearing in lists like Chicago that were returning 10 dead ends before.  Hopefully the issue has been identified and things will be back to normal soon.

FWIW: Victoria's suggestion to check the last update time is a good one as it will prevent your script from making a fruitless call to the API.
Photo of elee

elee

  • 11 Posts
  • 1 Reply Like
HI,
After this note, today seemingly something must be happening, as some data bring at least 1 or 2 good PWS, and on the next call, bring back the 10 PWS without data.
I personally make the call every 30 minutes, integrating the coordinates of my location and it is extremely useful to determine the weather conditions, wherever I am, even if I am in constant movement.

My script what it does is make the call and turn it into a flat file in which it starts looking for which of the PWS IDs closest to me, has a value (1) and integrates it with another call that will download the conditions of the station closest to me.

This worked well, but now what it brings is 10 PWS with all in value (-1).


Hopefully we will have good news soon.
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 656 Posts
  • 92 Reply Likes
Elee, can you provide some locations that are returning no useful data?  We need to be able to look at this concretely.

Keep in mind the QC status can be -1 despite the station being active.  Not every station passes QC.

--Victoria
Photo of elee

elee

  • 11 Posts
  • 1 Reply Like

Hi, this can be one of the coordinates that can be an example. This is my location where my PWS (IALAJU1) is located, as you can see in the sample file below, not even my own station is listed.

9.95523218, -84.2303761

 

Below, an example of what the call returns:

{"location":{"stationName":["Alajuela","pws:ISJPASOD2","Alajuela","Cebadilla","Urbanizacion El Pino","Avenida Villanea","Targua 1530mts","San José","EscalanteWF","WF Escalante, CR"],"stationId":["IALAJUEL40","ISJPASOD2","IALAJUEL35","IALAJUEL27","IHEREDIA3","ISANJOS21","IALAJUEL28","ISANJOS22","ISJELEMP2","IGOICOEC5"],"qcStatus":[-1,-1,-1,-1,-1,-1,-1,-1,-1,-1],"updateTimeUtc":[null,null,null,null,null,null,null,null,null,null],"partnerId":[null,null,null,null,null,null,null,null,null,null],"latitude":[9.97848,9.92164,9.99366,9.95718,10.01144,9.92358,9.9016,9.9333,9.93377,9.93373],"longitude":[-84.27659,-84.17806,-84.31416,-84.33722,-84.12314,-84.10439,-84.1004,-84.0833,-84.06519,-84.06518],"distanceKm":[5.67,6.85,10.11,11.68,13.34,14.26,15.45,16.31,18.27,18.27],"distanceMi":[3.53,4.25,6.28,7.26,8.29,8.86,9.6,10.14,11.35,11.35]}}
Photo of Emery Wooten

Emery Wooten

  • 25 Posts
  • 14 Reply Likes
Here is something that is happening that might provide a clue for the developers.  If I use this call: api.weather.com/v3/location/point?postalKey=60601:US&language=en-US&format=json&apiKey=xxxxx  It returns the lat,lng for Chicago.  Then I use this call which incorporates the lat,lng from above: api.weather.com/v3/location/near?geocode=41.88,-87.63&product=pws&format=json&apiKey=xxxxx.  This yields 10 pws IDs which are all dead ends with NULL update times.

Now on the other hand if I use this call: api.weather.com/v3/location/near?postalKey=60601:US&product=pws&format=json&apiKey=xxxxx  which directly uses the postalKey instead of the lat,lng it will yield 3 of the 10 stations with active update times.

In other words using the postal code directly in the "near" call produces different results than using the lat,lng.  In my application I am using the "point" call to get the city name so with the lat,lng readily available I was using the second call to search for valid stations which should be a good way to do it.  However it produces bad results compared to the direct postalKey call.
------
My 2 cents worth.  The API should return no dead stations.  They should be filtered out upstream.  I do not see, but I may be missing something, any reason for returning dead stations.  They clutter the results and eat bandwidth.
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 656 Posts
  • 92 Reply Likes
I am pretty sure that the difference between using the lat/long and the zip code is the way locations are generated and how that is influenced by the problem we've been having with location services.  (See the article here:  https://feedback.weather.com/customer/en/portal/articles/2977006-pws-location-services-issues?b_id=17298)

I have not been included in any discussions about the non-functioning stations in the returns, but when I hear more I will let you know.

--Victoria