I do and have done - then i can see the device when i click on whisper tab. then i hover over it and select “day” and this brings up a graph below - this graph never populates?
Another thought - do i need to have an actual “Whisper” Directory of any old directory where the files will end up? e.g does datayours2 require a specific path? I currently use /Veralogs.
All seems fine other than it just wont watch a device even though it adds it as if it is watched?
I’m still puzzled by this. So much is working. I have yet to be totally convinced that the directory path is correct. One thing to try is to directly command the DataWatcher daemon to start recording a variable (or a whole class of variables with a wildcard *)
The user guide has a section which documents this component in detail. Can you give that a try?
I’m away from my development system just now, so don’t have the details to hand, but it is the [tt]watch[/tt] parameter to the URL /id=lr_DataWatcher2.
You should get an acknowledgement of success and the data should start populating. The configuration should show it is being watched, and stored.
could you confirm two things for me:
- what the path should be exactly - or what you have working via Cifs
- what confirmation I receive after excepting the large watch this button - is it a pop up or another page or what that states i have successfully added the watched parameter.
Thank you again - we will get this working 
p.s wont be able to try this until tonight just FYI.
It doesn’t matter where it is. I have had Whisper databases installed on Vera itself, USB, and CIFS mounted drives. Just two things to make sure:
[ol][li]The directory is actually accessible (ie. successfully mounted)[/li]
[li]The Whisper path in the [tt]carbon.conf[/tt] file points to the same place (and starts and ends with “/”)[/li][/ol]
As an example, one of my Whisper databases is on a USB mounted as an external drive on an Apple TimeCapsule
[ul][li]The path to it on the USB is: [tt]/DATAMINE/whisper/[/tt][/li]
[li]The relevant mount parameters in my Startup Lua are: [list]
[li]local device = “//172.16.42.100/DATAMINE”[/li]
[li]local directory = “/dataMine”[/li]
[/list][/li]
[li]The [tt]carbon.conf[/tt] parameter is [tt]LOCAL_DATA_DIR = “/dataMine/whisper/”[/tt][/li][/ul]
2) what confirmation I receive after excepting the large watch this button - is it a pop up or another page or what that states i have successfully added the watched parameter.It's a one-liner of the form:
[tt]watch=110.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperature[/tt]
which appears where the button was, in the tan-coloured lower half of the page.
Since there appear to be some browser problems with formatting (although, for me, it works the same way in Safari on an iPad and Firefox on an iMac) you might try it with Firefox.
Hi akbooer,
I had a play around again last night and identified the following:
I can connect to my cifs share on my windows 7 machine with out issue. I set up 3x different folders and tried mounting to each and writing to each and every time vera mounted fine and datayours created a file in the respective folder that represented a node being watched. This I believe proves that the folder locations and mounts etc are working fine. please tell me if you think otherwise.
I have noticed that if I don not have “dataDash2.conf and Datawatcher2.conf” in the /www directory I get the error where by the home page for data your wont load. if i put these files back and blank them out they become populated again and I can then see the datayours2 home page again. should these two file not be created automatically if they do not exist? could this point to a possible problem?
I have also used firefox for all tests and have added and monitored many different nodes in the hope i may start to get results. sadly no joy yet. I have noticed text in the bottom half however similar to that you stated i should see when i should receive confirmation that a niode is being watched. to test that i added “” around the config file parameters so things error’ed out and as such I was able to get the below:
[string “module (“L_DataDaemon2”, package.seeall)…”]:80: bad argument #1 to ‘lines’ ("/nas/"storage-schemas.conf: No such file or directory)
so as such i know i can see the text you mention i should be able to. I then obviously removed the “” from the config so things worked again.
I'm still puzzled by this. So much is working. I have yet to be totally convinced that the directory path is correct. One thing to try is to directly command the DataWatcher daemon to start recording a variable (or a whole class of variables with a wildcard *)The user guide has a section which documents this component in detail. Can you give that a try?
I sis have a look at this but could not find the details to get me started - i think its just because i a new to datayours and not sure relay what i was looking for - if you could give me a nudge in the right direction i shall do my best to get these results for you.
Only other thought is do I have a miss match of luup files? I have installed all luup files attached in the first post of this threat only and then the two other file you listed (datadeamon2.lua and datadash2.lua) as updates that I assume have over written the ones on vera already.
Lastly in my start up lua i have 5 eliments that are stated:
require “L_DataWatcher2”
require “L_DataCache2”
require “L_DataGraph2”
require “L_DataDaemon2”
require "L_DataDash2
But the pdf manual only mentions the requirement for 4. Datadeamon is not listed as required in the manual?
could this all possibly related to the fact i have not upgraded from the initial release and as such am missing something?
all food for thought and sorry to be a pest - this is an odd one.
@LightsOn, this is a brilliant piece of diagnostic work - I applaud your efforts (which, hopefully, will bear fruit.)
I’m convinced.
I have noticed that if I don not have "dataDash2.conf and Datawatcher2.conf" in the /www directory I get the error where by the home page for data your wont load. if i put these files back and blank them out they become populated again and I can then see the datayours2 home page again. should these two file not be created automatically if they do not exist? could this point to a possible problem?[s]Definitely a 'possible problem'! They should not be required, won't be generated automatically, and certainly shouldn't generate the error that you report. If they're absent, then every Whisper file created will have just one archive - one hour sampling for a week - so not very useful, and I've never tried without.[/s] [--Edit: this is wrong, I was thinking you were talking about storage-schemas.conf and storage-aggregation.conf in the Whisper directory--] These files ARE generated automatically, and it shouldn't be a problem if they are not there. So once AGAIN, my unit testing is seen to be inadequate.
I have also used firefox for all tests and have added and monitored many different nodes in the hope i may start to get results. sadly no joy yet. I have noticed text in the bottom half however similar to that you stated i should see when i should receive confirmation that a niode is being watched. to test that i added "" around the config file parameters so things error'ed out and as such I was able to get the below:[string “module (“L_DataDaemon2”, package.seeall)…”]:80: bad argument #1 to ‘lines’ ("/nas/"storage-schemas.conf: No such file or directory)
so as such i know i can see the text you mention i should be able to. I then obviously removed the “” from the config so things worked again.
Good thinking! So we can scratch out that possibility.
... could not find the details to get me started - i think its just because i a new to datayours and not sure relay what i was looking for - if you could give me a nudge in the right direction i shall do my best to get these results for you.I will do, but I'm hoping that it won't be necessary.
Only other thought is do I have a miss match of luup files? I have installed all luup files attached in the first post of this threat only and then the two other file you listed (datadeamon2.lua and datadash2.lua) as updates that I assume have over written the ones on vera already.Possible. I think it's time I consolidated all the changes and reposted the whole lot.
Lastly in my start up lua i have 5 eliments that are stated:require “L_DataWatcher2”
require “L_DataCache2”
require “L_DataGraph2”
require “L_DataDaemon2”
require "L_DataDash2But the pdf manual only mentions the requirement for 4. Datadeamon is not listed as required in the manual?
could this all possibly related to the fact i have not upgraded from the initial release and as such am missing something?
BINGO! (perhaps.) You absolutely, should NOT have a require statement for L_DataDaemon2 in your Startup Lua. It will load the module and then prevent the subsequent modules (which explicitly include their own) from creating their own contexts. So this will create a problem for DataDash, in this case.
I think this may be it. Remove it and see what we get. If not, then you’ve given us lots to try further.
all food for thought and sorry to be a pest - this is an odd one.
NO, I’m sorry to be such a poor developer and diagnostician.
Hi,
so have Removed the extra start up lua entry as discussed.
i also removed the two files:#
These files ARE generated automatically, and it shouldn't be a problem if they are not there.
and now i get “No handler”
I think it may make good sense to get a check on all luup files to ensure no old files from the previous version have crossed over into ones listed in this post? could you post a nre zip package perhaps?
Here’s a complete set (minus the Whisper2 library, which hasn’t changed).
The Configuration page is augmented with three buttons (image attached) which access the storage rules files (schemas and aggregation) and generate a directory listing of the Whisper database, with key parameters.
This image also shows that my system has 3 Veras feeding into one of the databases which has logged 128,000 entries over the last two weeks that it has been continuously running (ie. Luup hasn’t restarted for that time)… just to show that it can and does work well, once it works!
PROGRESS!!! ;D
What ever is in those files has immediately worked. I have the message stating a variable is being watched and did not received the initial error “No handler”
I am just no waiting for population of data to take place so i can confirm all is working. if so i can only assume the file.zip on the leading post in this thread has a file that is old or corrupted in some way?
Will keep you posted.
24 hours or so in and this is just fantastic. Remote viewing of graphs works great. Logging of all data works great. really pleased with this and cifs to a w7 machine working like clock work - cracking
I still need to get my head round the Scemas when there is not a pre-set in place - need to do some reading here as still don’t quite understand this fully. to be fair there are pre-sets for most so i don’t need to worry a great deal but would be good to fully understand it.
Once question i could not find an answer to. Can I have 2 sets of data on one graph? such as a motion sensor status and light status?
Glad it’s working for you at last! Also, especially nice to see remote access to graphs being of some use. The whole of DataYours works (ie. All the daemons) by sending HTTP requests to each other - this is how multiple machine variable watching and multiple databases work. So all the functionality is available externally. It’s true, though, that only a little of it is documented. But the rendering interface corresponds closely to the Graphite Carbon one so that is well documented, although only partially implemented.
One of the things that is implemented is multiple graphs, with the constraint that the data needs to come from matching archives (ie. Sample rate and retention have to match for the time interval plotted.) the syntax for multiple plots is defined in the original Carbon docs referenced several times in the DataYours user guide. You can also plot saved graphs over any timescale.
The DataYours ‘trick’ to multiple plots is to use the ‘select’ option from the Whisper tree map for all the metrics you want to plot on one graph and then save them with a name. That graph name then appears on the Graphs tree map and, selecting a timescale, you can plot your graph.
For understanding more about whisper archives, reread the relevant bit of the user guide and the reference Graohite docs. There are also other tutorials out there on Whisper which may help.
I’ll post the syntax for multiple plots separately, but you can always see with your browser by hovering over any link (for example, to plot a metric) the URL that is going to be invoked (OK, some are quite long.)
Multiple plots:
DataGraph can be invoked directly to graph any variable(s) over any timescale, see the Graphite documentation at [url=http://graphite.readthedocs.org/en/latest/render_api.html]http://graphite.readthedocs.org/en/latest/render_api.html[/url]. from/until times can be in relative (d/w/m/y) or absolute format (although I have implemented ISO 8601 standard format rather than the proprietary Graphite one - there are commonalities.) Thus to plot a single (temperature) variable for the last day, you would use:
/data_request?id=lr_render2&from=-d&target=Vera-35104005.004.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperatureValue lists “{…}” enable plotting of multiple variables:
/data_request?id=lr_render2&from=-d&target=Vera-35104005.{004,308}.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperaturewilcards “*”, and character lists “[…]” and ranges are not supported.
Titles and legends can be added:
/data_request?id=lr_render2&from=-d&height=360&aliases={a,b}&title=Temperatures&target=Vera-35104005.{004,308}.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperatureThere’s no reason to stick to the same variable type, here’s a plot of temperature and humidity (on the same axes) over the last week:
/data_request?id=lr_render2&from=-w&target=Vera-35104005.{004.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperature,007.urn:micasaverde-com:serviceId:HumiditySensor1.CurrentLevel}…the only limitation being that the archive retentions and sampling match over the plotted timescale.
As mentioned elsewhere, these plots are available through the MiOS servers (if you are on UI5) with the addition of the appropriate server and authorization parameters. so it would be easy to set up an externally hosted webpage with references to current plots drawn from your Whisper database.
Hi Akbooer,
I installed your app following the instructions of your guide. I can access the dashboard, display all the devices and variables. I’ve activate the archive of a variable but I can’t see the plot of values.
I can see the real time value of each variable when I mouse over but I continue to see the “watch” option not the “plot”.
Any suggestion ?
Tnks Donato
The plot option only appears on the Whisper tree map elements, not the Device one.
Are the files being updated? What does the configuration page DataCache display tell you about stored metrics?
[quote=“akbooer, post:75, topic:181793”]The plot option only appears on the Whisper tree map elements, not the Device one.
Are the files being updated? What does the configuration page DataCache display tell you about stored metrics?[/quote]
Hi akbooer,
I don’t see any metrics stored in the dashboard. Following is the configuratin page of DataCache :
DataCache2 CONFIGURATION at Sat Jul 19 10:35:54 2014
DAEMON = {
VERSION = “2014.07.15 DataDaemon2 @akbooer”,
client = “DataCache2”,
destinations = {},
errors = {count = 0},
ip = “192.168.1.64”,
start_time = “Sat Jul 19 06:45:59 2014”,
udp = {
last_received = {[“127.0.0.1:2003”] = “Vera-35106052.035.urn:micasaverde-com:serviceId:EnergyMetering1.Watts 735 1405758952”},
last_sent = {},
received = 54,
sent = 0}}
DATACACHE = {
VERSION = “2014.07.10 DataCache2 @akbooer”,
stats = {
cpu = 0.009,
updates = 54},
tally = {Vera-35106052.035.urn:micasaverde-com:serviceId:EnergyMetering1.Watts = 54},
whisper = “/dataMine/Whisper/\013”}
[aggregator] = {
DESTINATIONS = “127.0.0.1:2004\013”,
LINE_RECEIVER_INTERFACE = “0.0.0.0\013”,
LINE_RECEIVER_PORT = 2023,
MAX_AGGREGATION_INTERVALS = 5,
MAX_DATAPOINTS_PER_MESSAGE = 500,
MAX_QUEUE_SIZE = 10000,
REPLICATION_FACTOR = 1,
USE_FLOW_CONTROL = “True\013”}
[cache] = {
CACHE_QUERY_INTERFACE = “0.0.0.0\013”,
CACHE_QUERY_PORT = 7002,
ENABLE_UDP_LISTENER = “True\013”,
LINE_RECEIVER_INTERFACE = “0.0.0.0\013”,
LINE_RECEIVER_PORT = 2003,
LOCAL_DATA_DIR = “/dataMine/Whisper/\013”,
LOG_UPDATES = “False\013”,
MAX_CACHE_SIZE = “inf\013”,
MAX_CREATES_PER_MINUTE = 50,
MAX_UPDATES_PER_SECOND = 500,
UDP_RECEIVER_INTERFACE = “0.0.0.0\013”,
UDP_RECEIVER_PORT = 2003,
USER = “”,
USE_FLOW_CONTROL = “True\013”,
WHISPER_AUTOFLUSH = “False\013”}
[relay] = {
DESTINATIONS = “127.0.0.1:2003\013”,
LINE_RECEIVER_INTERFACE = “0.0.0.0\013”,
LINE_RECEIVER_PORT = 2013,
MAX_DATAPOINTS_PER_MESSAGE = 500,
MAX_QUEUE_SIZE = 10000,
RELAY_METHOD = “rules\013”,
REPLICATION_FACTOR = 1,
USE_FLOW_CONTROL = “True\013”}[/size]
My Whisper database is on a USB stick .
Tnks
So the metrics show that you have one channel configured (energy meter) that has updated 54 times. It should be storing the data somewhere and that file should show as one big box on the whisper display which has a popup menu with plot options.
However, you whisper directory configuration parameter has an illegal control character at the end, probably from the editor you used to create it. Remove that, and all should be well.
[quote=“akbooer, post:77, topic:181793”]So the metrics show that you have one channel configured (energy meter) that has updated 54 times. It should be storing the data somewhere and that file should show as one big box on the whisper display which has a popup menu with plot options.
However, you whisper directory configuration parameter has an illegal control character at the end, probably from the editor you used to create it. Remove that, and all should be well.[/quote]
ok I try … tnks for your fast reply…
Actually, I suspect that this may be a common problem, and I note that it is not just that parameter with the problem, but all ones that you have edited.
Here’s an updated Lua file which strips out control characters from parameter values, so should fix things.
[quote=“akbooer, post:79, topic:181793”]Actually, I suspect that this may be a common problem, and I note that it is not just that parameter with the problem, but all ones that you have edited.
Here’s an updated Lua file which strips out control characters from parameter values, so should fix things.[/quote]
Hi akbooer,
To activate the change is sufficient to reload Lua files ? or I have to replace some other files ?
tnks
Donato