Unable to access https API

Developer's Corner Scene Scripting
#1

I have used Lua scripting and the socket.http module to access current weather conditions from the NOAA website. Last month, NOAA shutdown the legacy website and has launched a new API that requires https as well as a user-agent header. Unfortunately socket.http does not support https (just coerces url to http). I have come across the following examples of ways to make https requests from Lua but have not been successful for the reasons indicated for each. To simplify the testing, I am just testing them here against a popular test page.

This one works, but I cannot add a custom user-agent header using this format.

local curl, response, http = luup.inet.request( “https://httpbin.org/get”, nil, nil, nil,nil )
luup.log(“status=”… http)
luup.log("response= "… response)

This format allows a custom user-agent header, but does not work.

local curl, response, http = luup.inet.request{
url = “https://httpbin.org/get”
}
luup.log(“status=”… http)
luup.log("response= "… response)

Same for this one.
local http = require(“ssl.https”)
local res, code, headers, status = http.request{
url=“https://httpbin.org/get”
}
luup.log(“status=”… status)
luup.log("response= "… res)

Any idea what I might be doing wrong such that these approaches aren’t working? Any other approaches that I should be using instead?

#2

Without knowing what you are running this on, it is difficult to know. I suspect that you are running a very old vera and firmware which does not support the SSL version required for that website. I have had similar problems initially on openLuup and had to upgrade the lua socket.

#3

I’m running this on a Vera Edge with firmware version 1.7.4.

#4

You need to use the LuaSec library, which should already be installed.

It has the same request API as the http library, but uses https.

local https = require "ssl.https"

local status = https.request (...)

Even using an HTTPS url, the normal library does not use SSL.

#5

Thanks for confirming I’m heading the right direction. I believe that library you are recommending is the same library as the third option I tried above, which does not work (“failed to test code” error in Vera luup test). I believe I need to use the general form, rather than the simple form you show in your suggestion, for the NOAA site so that I can force a user-agent header. Am I messing up the syntax somehow?

#6

The reason your code isn’t working is that the return values can be [tt]nil[/tt], and in your log calls you are trying to concatenate them as strings. You can’t concatenate [tt]nil[/tt], and you need to guard against it. Make sure you are looking in the Luup log file. This kind of error will be exposed there.

That said, there’s something odd going on with that site that is making https return those nil values. If I change the URL to my own web site, it works fine. Here’s your code, modified. Works on my sandbox Vera3:

local http = require("ssl.https") local res, code, headers, status = http.request{ url="https://www.toggledbits.com/reactor" } luup.log("status=".. tostring(status)) -- use tostring() to guard nil luup.log("response= ".. ( res or "nothing")) -- use logical operator to guard nil

This also requires a bit more work to get the actual response…

local http = require("ssl.https") local ltn12 = require "ltn12" r = {} -- init empty table local res, code, headers, status = http.request{ url="https://www.toggledbits.com/reactor", sink = ltn12.sink.table( r ) } luup.log("status=".. tostring(status)) luup.log("response= ".. table.concat( r, "" ) )

#7

Fantastic. It works and I don’t even need to provide a special user-agent header. Thanks for the help all and especially rigpapa.

In case others are interested, I have posted the basic request code here that returns a JSON object containing all of the current weather conditions. This may be a good replacement for the the WU plugin now that WU is discontinuing their free API. In my case, this comes from MSP airport. You can look up your nearest weather station using a different endpoint in the same API. https://forecast-v3.weather.gov/documentation

local http = require("ssl.https") local ltn12 = require "ltn12" r = {} -- init empty table local res, code, headers, status = http.request{ url="https://api.weather.gov/stations/KMSP/observations/current", sink = ltn12.sink.table( r ) } luup.log("status=".. tostring(status)) luup.log("response= ".. table.concat( r, "" ) )

#8

For those of you that don’t want to do Lua, the SiteSensor plugin will make requests from an API that returns JSON, and you can pick and store elements of the response in state variables on the device.