New API: Missing Current Condition Narrative and City of PWS

  • 3
  • Idea
  • Updated 3 months ago
Been working with the new api.  A couple of things clearly missing are as follows.  I

1.  Can not return the  Current Condition narrative of the PWS  such as 'sunny' or 'cloudy'.  One can use the Forecast call to get a forecast condition.  In my case using the latlong coordinates of a PWS.  However forecast 'narrative' could very much be different than the current condition surrounding a PWS

2. Can not return the City of the PWS.  

Both of these seem fundamental to me and hopefully, they can be added in the future
Photo of VG

VG

  • 1 Post
  • 0 Reply Likes

Posted 3 months ago

  • 3
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 607 Posts
  • 84 Reply Likes
I will pass this along.

Victoria Gardner
victoria.gardner@ibm.com
Photo of John KUBIATOWICZ

John KUBIATOWICZ

  • 9 Posts
  • 1 Reply Like
It sounds like you are suggesting that we somehow duplicate the functionality of The Weather Company -- which has access to lots of weather data that we do not have.  I only have one weather station. My neighbor has one.  Our neighborhood and city have many.  So -- someone with the right access and models could get a good sense of current conditions in a region (like say a complete zip code or city).

Forgive me if this sounds snarky, but the point of having the API to do that for us is that we don't have to write our own weather modeling to glean current conditions over a small region from the data of multiple PWSs.  Also, given the limits on queries, it would seem that querying all the local PWSs to get a good sense of the current conditions might run afoul of limits. 
Photo of John KUBIATOWICZ

John KUBIATOWICZ

  • 9 Posts
  • 1 Reply Like
p.s. In California, here, we are the pinnacle of micro-climates.  I see less rain than my neighbor.  I am interested in the status of my city (or zipcode) rather than just my weatherstation.  Herein lies the challenge... 

Just out of curiousity, is it not possible to run an "extraordinary" short-term forecast that produces the current conditions?  What happens to your models when the expiration time approaches?  (or perhaps it doesn't work that way -- I'm not anything close to a weather expert).
Photo of John KUBIATOWICZ

John KUBIATOWICZ

  • 9 Posts
  • 1 Reply Like
p.p.s Aren't there lots of people who have weatherstations that don't produce rainfall totals?  Doesn't that mean that they no longer get information on current conditions about whether or not it is raining in their region or not?
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 607 Posts
  • 84 Reply Likes
Well, and as I said, I personally would probably just use the history or the forecast and call it a day.  I still have not understood the reason for needing that narrative version for a home user.  It seems to me that you're pretty close -- that everyone who wants it wants it to duplicate the forecast.

Recent history calls will come close to duplicating current conditions.  

It's unlikely that the PWS API is going to provide all the data that you would need to make your own version of a Wunderground page.  It's to provide data that people need for their own personal use.  

--Victoria
Photo of John KUBIATOWICZ

John KUBIATOWICZ

  • 9 Posts
  • 1 Reply Like
Ah.  Perhaps I haven't understood the "recent history" portion of the API.  I will look at it.

I can't really speak for everyone here, but I think that most of them just want something that says what the current conditions are in a queried region (like the one they requested the 5-day forecast on).  For many of them, this is almost as interesting as the forecast.  It says "what's going on in my town right now?"  It is the cool thing that kids (and adults alike) can see from a query (look -- I'm able to find out that there are clouds right now with this rasberry-PI!).  As you have seen from the blog traffic, the Wunderground served a lot of communities other than just the single home user -- schools, small-time home automation providers, hobbyiests.  I think that people are looking for the equivalent of the current conditions fields of the original WU API.  Something that doesn't require a lot of post-processing to turn into a graphic. 

It is also something that seems like it ought to be really easy to provide if you are already doing the (much more complex) provisions of 5-day forecast. 

My $0.02.  Perhaps
Photo of Joel

Joel

  • 6 Posts
  • 3 Reply Likes
Agreed. I just started to update my personal scripts and I ran into this same issue.

VG, it looks like you can use the location search utility API to look up your PWS ID and return a place id. Then you can use that place id when getting the forecast, which hopefully would be more accurate to your PWS. No idea if this new API (or the old one really) updated the actual observed weather conditions to be that precise :-)
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 566 Posts
  • 81 Reply Likes
Yes, Joel.  Forecasts are pretty much never as precise as a single PWS.  Thank you!

--Victoria
Photo of Claude Felizardo

Claude Felizardo

  • 27 Posts
  • 6 Reply Likes
How often do we need to look up the "place id"?  Is this "place id" going to be the same for a given PWS or could it change every once in a while?  Daily or weekly or hourly?

My concern is having to make several API calls that will eat into our minute or daily limits. 

If it does change periodically can we look it up at startup of our program, use a cached value until we get some kind of error indicating it is no longer valid then do a new lookup only as needed?

Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 566 Posts
  • 81 Reply Likes
I hear you about the calls, Claude!

If you're calling your own PWS, I guess I would be inclined to use the PWS ID where possible, because you will have to know if that changes.  (Because you would have to be the one changing it.)  Similarly if you're calling someone else's PWS, the ID should fail if it's unknown, but with something like a lat/long, it may do the best it can and give you something you don't want.  

Generally a PWS isn't going to move except under unusual circumstances.  But they do come and go.  

If you have a particular place you like, you could set up something like this.  Call the PWS by ID (if you can), and then if that fails, do the "near" call and see if you can find something similar close by.  I would only do the searching/revalidation "on error".  So if everything is fine, go ahead, else, revalidate/refind.  

Does that make sense?

--Victoria
victoria.gardner@ibm.com
Photo of Claude Felizardo

Claude Felizardo

  • 27 Posts
  • 6 Reply Likes
yes.  makes sense.  I would only be interested in my own PWS, again to make sure it is uploading correctly. 

If I get a chance to fix a package that could be used by others then I would keep do the steps you outlined above. 
Photo of grahamck

grahamck

  • 5 Posts
  • 0 Reply Likes
If I look at the WU page for my PWS I can see a "current conditions" attribute and this is the only thing that I am missing from the previous API that I really would like to have. I can get it from openweathermap and inject it into my home automation system, but I sure would love to be able to get it from WU. Am i missing this someplace in the new API?  

In case you are curious of the possible use cases here is one of many - my home automation system will turn off image based motion detection on my UV cameras when it is snowing at night. Previously I would get the condition "Snowing" from the WU API - not perfect, but pretty darned good.....

Otherwise it appears that I can get everything I need from the new API. Forecast precipitation being the biggie with irrigation season coming up.......

Thanks.

(Edited)
Photo of grahamck

grahamck

  • 5 Posts
  • 0 Reply Likes
That's too bad and will make me move to another service to get this data. Not sure what my motivation is to provide data to WU these days.
Photo of John KUBIATOWICZ

John KUBIATOWICZ

  • 9 Posts
  • 1 Reply Like
Victoria, there seems like a lot of loss for folks with the new free API.  Isn't the point to keep providers of the data happy?  Why the major shift?  The whole point of WU was crowd-sourcing: Add your bit of data and get interesting aggregated information like forecasts and current conditions. 

I looked at the "historical" API as you mentioned, and it doesn't see to provide anything like current conditions (in the form of the icon/text indicating what is going on).  It is just a historical list of data from a weather station.  Is there some business model behind not providing limited aggregation to users? 
Photo of grahamck

grahamck

  • 5 Posts
  • 0 Reply Likes
^^^ exactly ^^^
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 587 Posts
  • 83 Reply Likes
Keep in mind you're talking about WU as it was before.  And then it was sold, and that company was sold, and now it's in a whole different context.

I am not privy to conversations about business models.  However, I do get a clear sense that the aggregated data is considered highly valuable.  And if users want it, they need to pay for it.

"Add your bit of data and get interesting aggregated information like forecasts and current conditions."  But this is exactly what we're doing!  

I've said this elsewhere, but switching APIs is like learning a new language.  Things you were used to saying one way before cannot be adequately translated.  As far as I can tell, there is an obsession here with this faux-current conditions narrative from before that flies in the face of reality.  It was never "current" at one spot.  For a while the iPhone app had a slider so you could send in what was reality at your location (whether or not you had a PWS).  It was almost never perfect where I was.  

If you want to know what the current conditions are, go to WU, on a browser or in an app.  I don't understand why a very small subset of users are so insistent that we provide narrative "conditions" through the API for free, when we're providing it on the branded products that everyone already has access to.  I have offered any number of suggestions for working around this in general.  The data to create your own version of Weather Underground using our own data just isn't going to happen anymore.  Maybe that's why the prior incarnations of this business were not successful.  I don't know.  But it sure does seem, from my perspective, that they opened the business model up for serious abuse, and IBM seems to want to learn from that lesson.

I'm sorry you're unhappy about this, I truly am.  But this is the reality that we're in.  

--Victoria
Photo of grahamck

grahamck

  • 5 Posts
  • 0 Reply Likes
In my case you totally and completely misunderstand your user base. A computer application cannot "go to an app on it's iphone and look up the conditions". If you consider "current conditions" as the data that I am sending you, then rest assured that sending you that data is highly optional for me. There are plenty of ways for me to understand the data I am sending you, and a myriad of places that I can download weather forecasts for use by algorithms that serve a lot of functions. When it comes to "is it raining, or is it snowing", these are never absolutely perfect, of course not. But they are close enough for the work that I use them for. I am currently getting that data through OpenWeatherMap since you no longer provide it. No longer providing that key piece of information means that I will no longer be using WU, which means that i have very little reason to send you my data anymore. I'm not sure if the new corporate model really cares if I send you data that you intend to aggregate with other data and attempt to sell back to me? But that will be the eventual outcome.  Hopefully that is clear, not partly cloudy, pun intended. ;-)

Really the only thing left for me to do is tell my weather station to stop sending you data. <shrug> TBH - it's annoying, but really not that big of a deal to have to change how I collect data to a method that doesn't include WU.
(Edited)
Photo of Bees

Bees

  • 4 Posts
  • 1 Reply Like
It's sad that we can't even get the current condition and corresponding icon. Simple to allow us. Not asking too much. I use it for my personal weather wallpaper and that section is missing (as shown). C'mon WU, bend a little for us!

Photo of John Kubiatowicz

John Kubiatowicz

  • 1 Post
  • 3 Reply Likes
Victoria, IBM seems to have a fundamental misunderstanding of their data providers.  Monetizing a crowd-sourced platform is tricky.  People have many reasons for choosing to contribute their data.  Perhaps IBM should do a bit more to understand their data sources?

People you are hearing from are paying the weather company with data to get a service.  They purchased weather stations, wrote custom upload code in some cases, and are paying for internet service, power, and maintenance costs to maintain a feed of data to IBM.  They are paying with the currency of data.  What do they want for this payment?  That is the question that IBM has not really understood, in my opinion. Instead, you are responding in a condescending tone and suggesting that all these people are unable to deal with the new language of the API and were all previously abusing the platform.  You are confusing syntax (language) with semantics (processed results).

Why do people contribute data?  Well, for the aggregated results of thousands of data providers (semantic results in the form of processed data), not for the unaggreggated data from their own weather stations. Any programmer worth their salt can incorporate a database into their weather feed.  Thus, the historical queries are not really all that amazing, nor is the current status of their data feeds.

You have presented a fundamental paradox, which reveals your actual position.  Either the Current Conditions Icon is worth very little (so much that clearly people can reproduce it by downloading their own data) or it is worth so much that IBM is unwilling to give it out to peons ("you are not going to be able to duplicate the WU home page").  Which is it?  Despite what you may or may not personally believe, the reason many people go to the trouble of uploading their data (and the cost!), is to get the aggregated current conditions.  Others want a forecast.  Both of these things were part of the value proposition that caused people to chose to upload their data feeds. 

I fully believe that the query traffic overloaded the servers that IBM chose to devote to WU.  I fully believe that the free API was used by lots of people that were not paying at all (not even in the form of data).  However, treating your data-paying customers like they are trying to cheat the system and get something for free or something that they shouldn't reasonably expect without paying IBM a bunch of money is both insulting and reflective of not understanding what motivated people in the first place. People may just take their data elsewhere.

Now you (mostly) seem like a reasonable person, so I have to assume that you have neither the power nor inclination to try to investigate what your data-paying customers want out of the crowd-sourced system.  Such is the state of those hired to deliver an open-loop message without any chance to incorporate feedback.

Personally, this seems very sad.  I always liked to hold up the Weather Underground as one of my favorite examples of a crowd-sourced platform.  I like to talk about it to our industrial partners -- even foreign ones.  This will no longer be the case.

As a closing thought, I suggest that IBM consider what the cost might be (to them) of providing the current conditions ICON/text as part of the "free" (to data-paying customers)  5-day historical query and decide whether the data feed might be sufficient to pay for a current condition icon. And, I would ask that you treat those on this forum as PAYING customers -- people paying with data.

That might have been a whole dollar worth of commentary, to which I'm sure you don't care.

So long and thanks for all the fish.
(Edited)
Photo of Carson Booth

Carson Booth

  • 1 Post
  • 0 Reply Likes
This is the most intelligent description and response of the situation I have seen to date. Thank you and as a data payer for years (IBerlin1622), I have spent the last 2 weeks porting over to DarKSky for this exact reason described above.
Thank you and sad to exit WU in the coming weeks.
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 607 Posts
  • 84 Reply Likes
I have put the request for this change in the queue to be analyzed and decided upon (by people other than me) in due course.  
Photo of elee

elee

  • 11 Posts
  • 1 Reply Like
Victoria, please help us to obtain this simple information. Maybe you think it was something vague, but in my case it was the most accurate data of any other weather system on the market. Please, we only need that information to be able to continue with WU. It is a request that abounds throughout this forum. Do not make us go to other services. Let us all make WU even better. Please.
Photo of Victoria Gardner

Victoria Gardner, Official Rep

  • 607 Posts
  • 84 Reply Likes
I will pass the request on to the people who make such decisions.

--Victoria