dataMine 2 documentation

Hi,

It is fairly simple. Log-on via the ALTUI portal [url=https://vera-ui.strongcubedfitness.com/Veralogin.php]https://vera-ui.strongcubedfitness.com/Veralogin.php[/url], select your Vera and wait for the page to load.

Then in the URL look for the ?id=lr_ALTUI… and remove everything after the =lr_ (ALTUI…) and add dmPage. Hit enter.

Too funny, I am just trying the Open button you see on the DM2.0 plugin on ALTUI and that launched DataMine just fine. I now I played with this in the past so I cannot recall it is just my version of if it was build in by amg0. Anyhow, if that doesn’t work, the above will.

Cheers Rene

Hi RWoerz,

thanks for the in-depth explanation

however, how does one extract the data file from /dataMine directory?

is it viewable under windows?

a question, how does one copy

[quote=“RWoerz, post:16, topic:190542”]dataMine 2 and Linux for dummies like me.

Not being a Linux expert I’ve struggled for a while trying to get dataMine 2 working on my Very Edge UI7. Starting but not limited to the fact that when you plug the first USB drive into a Vera no matter what size the drive is it creates one 500MB partition which isn’t easy to undo.

I’ve even spent time with Vera support asking if the 2nd USB drive had mounted and according to them everything was working as expected, whatever that means. I’m sure had I known Linux this would have been easy but there is so much info out there it’s hard to know what’s current and what’s not.

One big issue for me is the way many people who understand Linux post things. One post showed the output of what was said to be a “file” command. The output was interesting but they didn’t show the command itself so it was of little help.

If the folks at Vera are reading this it would be very helpful to have a tab that gives some basic systems info. Like what drives are mounted, how full are they, IP address, when a sensor was last seen in a human readable form not Epoch time. Epoch time, that’s Linux speak for the number of seconds since midnight Jan 1st 1970, I think that’s right.

Some posts say you can have Vera logging turned on some say no. Anyway I’m still not sure exactly what was needed but I’ll give you what I did and hopefully someone can tell us what wasn’t.

One note to start, I’ll show all Linux and Vera commands in quotes i.e. “lsusb” so you don’t type in the quotes just what’s between them. Also if you don’t know this Linux is case sensitive so be aware type everything in as shown.

The whole DataMine thing may just work for you through the Vera interface but that wasn’t what happened with me.

First thing I would plug in an old 1GB USB drive then turn logging on in the Vera “Settings, Logs” tab. Vera will grab hold of it and hopefully not think the next drive you plug in belongs to it.

Second you’ll need to download a program called PuTTY www.putty.org/, it allows you to “terminal” into the Vera box and run Linux commands in a DOS like window. Unix/Linux was around long before DOS and is a bit of a steep learning curve but to get what you need for dataMine you’ll only need a few commands.
Be forewarned you can KILL your system with the wrong command here so be very careful what you type in.

When you run PuTTY you’ll type in the IP address of you Vera box, you’ll find the IP at the top of your browser page when you’re logged into your Vera. Next make sure the port number is 22 and SSH is selected, then press the Open button.

If all is working a DOS like window will open and after several seconds a username prompt will appear. Type in “root” and press return. Next a password prompt will appear, my password was printed on a label on the bottom of my Vera, yours may vary. Type in your password and hit return. If all is well you’ve just logged into Linux on your Vera machine. I don’t think it drops you into the root directory so to make sure your are type in “cd /”, (cd space forward slash), that should put you at the root directory of your machine.

One problem I found with the version Linux installed on the Vera is that it doesn’t support many of the traditional tools supported on many Linux system. So if you Google for a command to show the UUID of a USB drive most of the commands you’ll find don’t work here. All of the commands I show here worked on my Vera.

First you’ll need to get the UUID of your USB drive. Before you plug in your new dataMine logging USB drive type in the following command “fdisk -l” (lower case L). At the end of the output you’ll see something like this which is my 2GB Vera logging drive with its 500MB partition.

Disk /dev/sdb: 2003 MB, 2003828736 bytes
62 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 3844 * 512 = 1968128 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 273 524705+ 83 Linux
/dev/sdb2 274 1018 1431890 83 Linux
root@MiOS_45011987:~#

Now plug in your dataMine drive and run the same command “fdisk-l”. You should see the newly mounted drive. Mine is a 16GB USB drive and it looks like this.

Disk /dev/sda: 16.0 GB, 16008609792 bytes
255 heads, 63 sectors/track, 1946 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 1 1947 15632384 c Win95 FAT32 (LBA)

The important thing to note here is the /dev/sda1 which you’ll type into Vera’s dataMine SetMountPoint. Yours may be different so write it down!

Next type in “lsusb” this will give you your drives UUID and should look something like this. Mine is the Cruzer and its UUID is 0781-5530, write yours down!

root@MiOS_45011987:~# lsusb
Bus 001 Device 002: ID 050d:0237 Belkin Components F5U237 USB 2.0 7-Port Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 003: ID 0403:6001 Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC
Bus 001 Device 004: ID 0781:5530 SanDisk Corp. Cruzer
Bus 001 Device 005: ID 8644:800a
root@MiOS_45011987:~#

Next install dataMine 2 on your Vera but make sure to leave the PuTTY window open, go to Vera’s “Apps, Install apps” tab then search for dataMine. There will be two pick dataMine 2. After the install is complete go to Vera’s Devices tab and find the dataMine 2 graphing device and open it. Then open the Advanced tab. Near the top of the page you’ll find 4 tabs, Parms, Variables, New service and Commands. Select the Variables tab. Under the Variables tab add the following information. In SetDataDirectory type in “/dataMine/”, after you type that in you have to right click in any of the white space on the page to update the information, strange but it’s the way you have to do it. In SetMountUUID you’ll have to type in your USB drives UUID. Then last but not least in SetMountPoint type in the mount point for your system each time right clicking in the white area in the web page. It will flash on the screen that the data has been updated.

The SetDataDirectory crates a dataMine folder in the root directory. The SetMountUUID and SetMountPoint attaches your USB drive to the dataMine folder. Now go back to the PuTTY window and type in “df -h”. It should show your drive attached to /dataMine directory and the used and available space.

root@MiOS_45011987:/dataMine# df -h
Filesystem Size Used Available Use% Mounted on
rootfs 9.4M 5.3M 4.1M 56% /
/dev/root 9.5M 9.5M 0 100% /rom
tmpfs 61.9M 3.4M 58.5M 5% /tmp
/dev/mtdblock6 9.4M 5.3M 4.1M 56% /overlay
overlayfs:/overlay 9.4M 5.3M 4.1M 56% /
tmpfs 512.0K 0 512.0K 0% /dev
/dev/mtdblock10 64.8M 9.9M 54.8M 15% /storage
/dev/mtdblock10 64.8M 9.9M 54.8M 15% /etc/cmh-firmware
/dev/mtdblock10 64.8M 9.9M 54.8M 15% /etc/cmh-backup
/dev/mtdblock9 7.8M 7.8M 0 100% /mios
/dev/sda1 14.9G 184.0K 14.9G 0% /dataMine

At this point you should be able in Vera to go to the Devices, dataMine 2 graphing, dataMine tab and select the “dataMine graphing page (local access)” and bring up a new tab in your browser called dataMine - graphing. If it asked select the USB Drive with the correct UUID. Under the Configuration tab add the sensors you want to graph. If you want to see where the data files for the sensors get built type in the PuTTY window “cd /dataMine” then “du -h”. That’s all I got hope it helped. Please if I messed something up PLEASE let me know.

root@MiOS_45011987:~# cd /dataMine
root@MiOS_45011987:/dataMine# du -h
16.0K ./database/1/raw
32.0K ./database/1
16.0K ./database/2/raw
32.0K ./database/2
8.0K ./database/3/raw
24.0K ./database/3
96.0K ./database
184.0K .[/quote]

New question. OK call me blind but in the DataMine2’s PDF on page 16 it says: “If update is specified, the graph will automatically be updated periodically. The update time is in minutes.”

I don’t seem to be able to find this variable in Vera or the WEB page. The “Display real-time graph” is grayed out".

Did the 7.0.19 firmware brake the cifs on the VeraPlus or is it just me?

50 12/15/16 22:01:03.706 luup_log:205: dataMine: Error reading tmpfile during mountcheck <0x77101520>
06 12/15/16 22:01:03.706 Device_Variable::m_szValue_set device: 205 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mmountLocatione[0m was: now: #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x77101520>
06 12/15/16 22:01:03.707 Device_Variable::m_szValue_set device: 205 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mmountTypee[0m was: now: #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x77101520>
50 12/15/16 22:01:03.728 luup_log:205: dataMine: Mounting dataMine storage (//192.168.1.2/home/datamine) to (/dataMine/) <0x77101520>
50 12/15/16 22:01:03.770 luup_log:205: dataMine: mount command output:'mount: mounting //192.168.1.2/home/datamine on /dataMine/ failed: No such device
50 12/15/16 22:01:03.795 luup_log:205: dataMine: Error reading tmpfile during mountcheck <0x77101520>
06 12/15/16 22:01:03.796 Device_Variable::m_szValue_set device: 205 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mmountLocatione[0m was: now: #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x77101520>
06 12/15/16 22:01:03.796 Device_Variable::m_szValue_set device: 205 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mmountTypee[0m was: now: #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x77101520>
50 12/15/16 22:01:03.817 luup_log:205: dataMine: Mount point error: :://192.168.1.2/home/datamine <0x77101520>

I tried reverting to the previous VeraPlus firmware 7.0.17 (1.7.2138) but I stil cannot mount cifs.
Just checked the files on my NAS and the last time the raw data file was updated was around the time I upgraded my VeraPlus to 7.0.19 (1.7.2414).
Not sure how to debug the situation. Anyone available to assist?

managed to get it back up and running by issuing the command presented by @mfp in this post → http://forum.micasaverde.com/index.php/topic,38219.0.html

Had the same after updating, they were missing. Probably the kernel modules from firmware 7.0.19 have been updated, then a reinstall is needed and normal.

How can you change the Graph theme under graph options?

This is not a drop down. is there any documentation over configurable options?

@Peren - you can type the name of the theme as free text in the Graph Options. Available options are on page 17 of the documentation available from http://code.mios.com/trac/mios_datamine_2/raw-attachment/wiki/WikiStart/DataMine2%20Overview%20v1.914.pdf

for convenience it says…

Graph Theme ? sets the theme of the graphing area, controlling colours, grid lines, borders, spacing and the like. There are now 4 new themes: classic (default), gray, grid-light and sandsignika. Further themes can be downloaded from https://github.com/highcharts/highcharts/tree/master/js/themes and saved in the /www/dm/themes directory. You will need to reload the GUI (usually press F5) after changing the value to see the new theme.

@RWoerz

the manual is referring to the page http://vera-ip/dm/graph.html?ref=&period=&update= where the data can be updated in real time and not the main dataMine gui. The auto update feature on the main dataMine gui is currently broken and as you note the button is greyed out and disabled.

Hi,

Just my 2c

I found i had to select the tick box under the 3 variables that we put the UUID data into to get it to work… But beside that, thanks heaps @RWoerz

Cheers,
Josh

Hello -

Hoping someone can give me a help on a new DM install on a VeraSecure.

I keep getting the “select USB” window during initialization.

lsusb Output

root@MiOS_55001685:/# lsusb Bus 001 Device 011: ID 05dc:a817 Lexar Media, Inc. Bus 001 Device 003: ID 05e3:0608 Genesys Logic, Inc. USB-2.0 4-Port HUB Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 004: ID 0e8d:7662 MediaTek Inc. Bus 001 Device 005: ID 04e2:1420 Exar Corp. Bus 001 Device 006: ID 04e2:1420 Exar Corp. Bus 001 Device 010: ID 1bc7:0026 Telit

FDISK Output

[code]Disk /dev/sda: 15 GiB, 16039018496 bytes, 31326208 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xba0d28b4

Device Boot Start End Blocks Id System
/dev/sda1 2048 31324159 15661056 c W95 FAT32 (LBA)
[/code]

I don’t see anything mounted:

root@MiOS_55001685:/# mount rootfs on / type rootfs (rw) /dev/root on /rom type squashfs (ro,relatime) proc on /proc type proc (rw,noatime) sysfs on /sys type sysfs (rw,noatime) tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime) /dev/mtdblock6 on /overlay type jffs2 (rw,noatime) overlayfs:/overlay on / type overlayfs (rw,noatime,lowerdir=/,upperdir=/overlay) tmpfs on /dev type tmpfs (rw,relatime,size=512k,mode=755) devpts on /dev/pts type devpts (rw,relatime,mode=600) debugfs on /sys/kernel/debug type debugfs (rw,noatime) /dev/mtdblock10 on /storage type jffs2 (rw,relatime) /dev/mtdblock10 on /etc/cmh-firmware type jffs2 (rw,relatime) /dev/mtdblock10 on /etc/cmh-backup type jffs2 (rw,relatime) /dev/mtdblock9 on /mios type squashfs (ro,relatime) root@MiOS_55001685:/#

But even if I mount manually to /datamine/ (directory manually created by me/root), it still doesn’t seem to help.

Pointers?


EDIT:
I saw I had a capital M in /datamine/. I made that fix but still not mounting:

50 07/14/17 18:47:09.735 luup_log:337: dataMine: Initialising dataMine System (2.000) <0x76a31520> 06 07/14/17 18:47:09.736 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1merrorStatuse[0m was: true now: false #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:0 <0x76a31520> 06 07/14/17 18:47:09.736 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mChannelCnte[0m was: 0 now: 0 #hooks: 0 upnp: 0 skip: 0 v:0x1504b40/NONE duplicate:1 <0x76a31520> 06 07/14/17 18:47:09.737 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mChannelRece[0m was: 0 now: 0 #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x76a31520> 50 07/14/17 18:47:09.766 luup_log:337: dataMine: Error reading tmpfile during mountcheck <0x76a31520> 06 07/14/17 18:47:09.767 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mmountLocatione[0m was: now: #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x76a31520> 06 07/14/17 18:47:09.767 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mmountTypee[0m was: now: #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x76a31520> 50 07/14/17 18:47:09.794 luup_log:337: dataMine: Mounting dataMine storage (<span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 12.48px; background-color: rgb(240, 244, 247);">/dev/sda1</span>) to (/datamine/) <0x76a31520> 50 07/14/17 18:47:09.841 luup_log:337: dataMine: mount command output:'' <0x76a31520> 50 07/14/17 18:47:09.869 luup_log:337: dataMine: Error reading tmpfile during mountcheck <0x76a31520> 06 07/14/17 18:47:09.870 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mmountLocatione[0m was: now: #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x76a31520> 06 07/14/17 18:47:09.870 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mmountTypee[0m was: now: #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x76a31520> 50 07/14/17 18:47:09.897 luup_log:337: dataMine: Mount point error: ::<span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 12.48px; background-color: rgb(240, 244, 247);">/dev/sda1</span> <0x76a31520> 06 07/14/17 18:47:09.898 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1merrorStatuse[0m was: false now: true #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:0 <0x76a31520> 50 07/14/17 18:47:09.904 luup_log:337: dataMine: ERROR: Startup state is 'uninitialised'! <0x76a31520> 06 07/14/17 18:47:09.905 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mChannelCnte[0m was: 0 now: 0 #hooks: 0 upnp: 0 skip: 0 v:0x1504b40/NONE duplicate:1 <0x76a31520> 06 07/14/17 18:47:09.905 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mChannelRece[0m was: 0 now: 0 #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x76a31520> 50 07/14/17 18:47:09.905 luup_log:337: dataMine: Reinitialising configuration structure <0x76a31520> 50 07/14/17 18:47:09.906 luup_log:337: dataMine: Startup complete <0x76a31520> 50 07/14/17 18:47:24.213 luup_log:337: dataMine: Delete backups complete - No files found <0x6f631520> 06 07/14/17 18:47:24.242 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mdiskTotale[0m was: 8192 now: 8192 #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x6f631520> 06 07/14/17 18:47:24.243 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mdiskUsede[0m was: 7284 now: 7288 #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:0 <0x6f631520> 06 07/14/17 18:47:24.243 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mdiskFreee[0m was: 908 now: 904 #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:0 <0x6f631520> 06 07/14/17 18:47:24.244 Device_Variable::m_szValue_set device: 337 service: urn:cd-jackson-com:serviceId:DataMine1 variable: e[35;1mdiskUsedPcnte[0m was: 89 now: 89 #hooks: 0 upnp: 0 skip: 0 v:(nil)/NONE duplicate:1 <0x6f631520> -2-=========================================================================== -3-=========================================================================== rootfs on / type rootfs (rw) /dev/root on /rom type squashfs (ro,relatime) proc on /proc type proc (rw,noatime) sysfs on /sys type sysfs (rw,noatime) tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime) /dev/mtdblock6 on /overlay type jffs2 (rw,noatime) overlayfs:/overlay on / type overlayfs (rw,noatime,lowerdir=/,upperdir=/overlay) tmpfs on /dev type tmpfs (rw,relatime,size=512k,mode=755) devpts on /dev/pts type devpts (rw,relatime,mode=600) debugfs on /sys/kernel/debug type debugfs (rw,noatime) /dev/mtdblock10 on /storage type jffs2 (rw,relatime) /dev/mtdblock10 on /etc/cmh-firmware type jffs2 (rw,relatime) /dev/mtdblock10 on /etc/cmh-backup type jffs2 (rw,relatime) /dev/mtdblock9 on /mios type squashfs (ro,relatime) -4-=========================================================================== -5-=========================================================================== SetDataDirectory-/datamine/ SetMountUUID -05dc-a817 SetMountPoint -<span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 12.48px; background-color: rgb(240, 244, 247);">/dev/sda1</span> SetManualMount -0 -6-=========================================================================== {"dbVersion":2,"Events":{"count":0,"last":0},"Version":"2.000","Variables":[],"Graphs":[],"guiConfig":[],"nextId":1} -7-===========================================================================


FINAL EDIT. Delete this noise if you want! (sorry)
I reread the message above my post and also checked the three check boxes. All is well now.

Thanks!

Hi

How can the scale range on the Dashboard dial be changed. I need to increase it quite a bit for my application?

Thanks, Thomasx

Can anyone expand on the details around Online Status on page 5 of the documentation? I have the grey radio button.

?Online Status? indicator ? this is green if the web interface is communicating with Vera, and grey if
Vera is not communicating with the browser. It displays red if communication is OK, however there
are errors within dataMine (eg. Initialisation error, or error writing data to the USB).

Does it mean there is a problem or values might not be updating when grey? I can sit here and watch it but like most people we probably always want to see green. :slight_smile:

[quote=“thomasx, post:33, topic:190542”]Hi

How can the scale range on the Dashboard dial be changed. I need to increase it quite a bit for my application?

Thanks, Thomasx[/quote]

Check page 8 of the documentation, that might help you out.

Hello,
I am rather a new user of Vera.
I would like to log the data coming from Fibaro wall plugs (that work through Z-wave) connected to my Vera plus controller.
I got a USB successfully mounted on the USB drive and made it to log the data. Attached is a screenshot:

Now, while trying to install the DataMine plugin for Vera, the MiOS Marketplace “doesn’t find any unit to connect with.” Am I missing any steps?
My computer and the controller are on the same network and I can access the controller via putty and can also see the web interface of the controller.
Please suggest how to install the plugin and proceed with the data handling?