Using Grafana with DataYours

I’ve installed DataYours on my VeraLite (running the last firmware 1.7.1040) a few days ago, and I would like to use Grafana to access the Whisper database located on my Vera.
As I read in another thread, I copied, from DataYours Github repository, the file graphite_cgi.lua into Vera’s directory /etc/cmh-ludl.
But when I create a Graphite datasource in Grafana, pointing to URL http://192.168.1.34:3480, with access type “Server”, no authentication and Graphite version 0.9, and “Save and Test”, I get an error message “HTTP Error Bad Gateway”.
Any idea of what is missing or wrong ?
Thanks in advance,
Eric

The file you used is a WSAPI cgi file, and this is not supported by Vera. Was there some documentation somewhere which suggested that it should work?

Yes, in your reply #21, dated October 17, 2016 in thread http://forum.micasaverde.com/index.php/topic,39612.15.html.
But I probably misunderstood the context.
So, this file graphite_cgi.lua does not solve my problem, as a conclusion.
What do I need to do ?
Best regards,

[quote=“ericferon, post:3, topic:200643”]Yes, in your reply #21, dated October 17, 2016 in thread http://forum.micasaverde.com/index.php/topic,39612.15.html.
But I probably misunderstood the context.[/quote]

Yes, afraid so. That thread, and indeed this one, is in the context of openLuup, which was designed to overcome many of the shortcomings of Vera.

So, this file graphite_cgi.lua does not solve my problem, as a conclusion. What do I need to do ?

No, it doesn’t. The problem is that to write an HTTP handler within Vera’s Luup, the request has to be of the form:

http://VeraIP/port_3480/data_request?id=...

But you can’t configure Grafana to generate such a request, because it wants to send things like:

http://VeraIP/metrics/find?query=*

The best that I could manage just now was to get it to generate:

http://VeraIP/port_3480/data_request/metrics/find?query=*

But that’s not good enough, because it has no id=… parameter.

What you’d need to do is:

[ol][li]write a standalone server which services another port and leverages the CGI file you downloaded, or,[/li]
[li]configure Vera’s lighttpd server to use WSAPI (see WSAPI) which is possible, I believe.[/li][/ol]

(2) above would allow the use of that CGI file directly. FWIW, I’ve asked whether the WSAPI CGI gateway can be included in a future Vera release, but who knows whether/when that would happen.

So (1) is what needs to be done. I never got around to it because openLuup was my route around the problem. Of course, that’s a solution for you too if you have a spare machine (or spare capacity) to run it.

OK, but there is a graphite plugin in Grafana (Graphite data source | Grafana documentation), that I used to create the link to DataYours.
I guess (and I hope !) that this plugin “transforms” the usual Grafana request to the one expected by any Graphite server, like DataGraph.
It expects, as “Url” parameter, the HTTP protocol, IP, and port of the graphite-web or graphite-api install.
In DataYours, isn’t it “http://VeraIP:3480” ?
Best regards,
Eric

Yes, that’s what I was using too.

I guess (and I hope !) that this plugin "transforms" the usual Grafana request to the one expected by any Graphite server, like DataGraph. It expects, as "Url" parameter, the HTTP protocol, IP, and port of the graphite-web or graphite-api install.

Yes, and that request is what is handled by the [tt]graphite_cgi.lua[/tt] file. But in order to run that CGI you need to go through an HTTP server which supports it, and, out of the box, Vera doesn’t.

In DataYours, isn't it "http://VeraIP:3480" ?

No, that’s deprecated in the latest firmware releases. In fact, under some security settings, that port is actually closed. You have virtual access to that through Vera’s port 80 using the [tt]/port_3480[/tt] syntax.

Using http://veraIP/port_3480 instead of http://veraIP:3480 already works better, as the datasource is recognized in Grafana.
But indeed, no metrics can be imported.
I guess that it could be possible to develop a grafana datasource plugin, that implements the query method needed by Datayours (cfr Legacy data source plugins | Grafana documentation).
Anyway, thank you for your help.

Yes, any JavaScript savvy person should be able to make a datasource plugin, but that’s outside my skillset, I’m afraid.

I have some free time in the coming days.
So, I would like to examine how to adapt a Grafana datasource for DataYours.
If I want to retrieve the data for a device (f.i, 044), a serviceId (f.i TemperatureSensor1) and a variable (f.i CurrentTemperature), which id and query parameters should I send to DataYours ?
Thank you in advance,

It’s broadly as described in the documentation under the DataGraph section (page 22.)

http://<yourVeraIP>/port_3480/data_request?id=lr_render&target=Vera-12345678.321.urn:upnp-  org:serviceId:TemperatureSensor1.CurrentTemperature

For example, on my Vera Edge, the request:

http://172.16.42.14/port_3480/data_request?id=lr_render&format=json&target=Vera-45101161.010.urn:system:serviceId:ProcMeminfo.MemAvail

returns

[{
  "target": "Vera-45101161.010.urn:system:serviceId:ProcMeminfo.MemAvail",
  "datapoints": [
  [92000, 1580119200],
  [91980, 1580122800],
  [91988, 1580126400],
  [92048, 1580130000],
  [92048, 1580133600],
  [92044, 1580137200],
  [91956, 1580140800],
  [92108, 1580144400],
  [92116, 1580148000],
  [92092, 1580151600],
  [91996, 1580155200],
  [91928, 1580158800],
  [91976, 1580162400],
  [91968, 1580166000],
  [91952, 1580169600],
  [91956, 1580173200],
  [91960, 1580176800],
  [91620, 1580180400],
  [91936, 1580184000],
  [92008, 1580187600],
  [92036, 1580191200],
  [92012, 1580194800],
  [92104, 1580198400],
  [92088, 1580202000],
  [91940, 1580205600]
  ]
}]

You can add &from and &until parameters to specify the date/time intervals you require.

HTH

Thank you.
when I send the request
http://192.168.1.6/port_3480/data_request?id=lr_render&target=Vera-45115871.080.urn:upnp- org:serviceId:TemperatureSensor1.CurrentTemperature&format=json,
I get the message :
[string “module (“L_DataWhisper”, package.seeall)…”]:622: unable to open Whisper file ‘/mnt/usb/dataMine/whisper/Vera-45115871.080.urn^upnp-org^serviceId^TemperatureSensor1.CurrentTemperature.wsp’
Although, the file exists :
root@MiOS_45114227:/mnt/usb/dataMine/whisper# ls -l *080*
-rw-r–r-- 1 root root 454512 Jan 31 14:34 Vera-45115871.080.urn^upnp-org^serviceId^TemperatureSensor1.CurrentTemperature.wsp
-rw-r–r-- 1 root root 454512 Jan 31 13:52 Vera-45115871.080.urn^upnp-org^serviceId^TemperatureSetpoint1.CurrentSetpoint.wsp
Any idea of the root cause of this message ?
Thanks in advance,

Odd. I’ve triple checked the spelling, as, no doubt, have you.

It looks like a permissions problem, although the request should only need read access, which it seems to have.

You might try opening up the file permission to all, and see if that changes anything.

No, it does not change anything : same error message.
Is there any authentication needed ?
K.r,

Well, it may be a directory permission somewhere along the tree. However, from your directory listing, it seems as though the file is updating ok?

What’s the permissions on the folder?

Can your cd into that folder and
touch test
?
C

OK, I found.
There was a hidden character in the urn string.
Thank you for your help,

2 Likes

Phew! Well done.