Tools: openHAB Item file generator

Plugins & Plugin Development openHAB MiOS Binding
#21

[quote=“gduprey, post:19, topic:185623”]My thinking would be to not “penalize” the non-duplicate, non-troublesome item names. Depending on how someone has maintained their Vera setup, all their names may come out fine, where as having all the device IDs means that someone will very likely want to edit every single item they keep.

I have about 320 out of 1300 item definitions with dups, but most are things I don’t think I care about nor will likely leave an item defined for. Most of the things I actually care about are fine.[/quote]

The latest version will append the various (scoped) "id"s only when a duplicate is detected.

I’m hoping it will fix all of the cases occur in practice but there are some off-beat cases where it won’t so I’ve left in the duplicate-detect-and-print stuff.

I had to install xsltproc on my stripped down ubuntu box, and for the benefit, doing either is a pretty minor additional step.

That’s odd. My default Ubuntu/Linarto on the Odroid C1 had it pre-installed. There must be install variations on Ubuntu 14.04 LTS.

I would expect gawk (being pretty new and updated regularly) is pretty i18l compliant and I'm not doing anything in there that would typically break strings with i18l chars in them.

Well, with the version I have now, it’s all contained in the XSLT. Only duplicate-reporting is done outside, so it could potentially break with non Latin-1 assumptions. To be fair, I’m not even sure that Item names can be anything other than ASCII-7, so it could all be moot anyhow.

As for integrating it all into xslt, that would be great! I am not super up on xslt (I've written a few minor transformations and can follow what you did mostly) and wouldn't quite know how to do duplicate checking and resolution. But I'm all for a one-stop shop style experience and minimal dependencies/files.

Well, there’s a version now. Hopefully it’s better than the last one (at generating unique-ish Item names)… time will tell.

My (so far, one) duplicate name within the same device is my power meter. For reasons that make no sense, the device seem to be publishing a LightSensor (it has no light sensor, so this is bogus) and a generic sensor level (which doesn't seem to match anything reported in the more power-specific items). The items are:

Number DUP_30_Panel1MasterCurrentLevel "Panel 1 Master Current Level [%d]" (GDevices,GRoom1) {mios="unit:nightmare,device:30/service/urn:micasaverde-com:serviceId:GenericSensor1/CurrentLevel"} /** SAMPLE=[48] **/ String DUP_30_1_Panel1MasterCurrentLevel "Panel 1 Master Current Level [%s]" (GDevices,GRoom1) {mios="unit:nightmare,device:30/service/urn:micasaverde-com:serviceId:LightSensor1/CurrentLevel"} /** SAMPLE=[0] **/

I have a few of those myself, where Vera has “added” Thermostat StateVariables to Lights (etc). Fun. Lovely little corruptions like that used to occur frequently in UI4, due to threading/concurrency problems that [ultimately] caused LuaState corruptions.

Thanks for all the help in testing this, and in pushing me to do better!

#22

Is there a solution for non-linux or mac users?

I have 500 devices and looking for a simpleway to Jumpstart OpenHab conversion.

Thanks!

#23

Cygwin maybe? The script is fairly simple and there is definitely a Cygwin version of the core xsltproc routine.

Alternatively I can give you instructions so you can send me a copy of your user_data (privately) and I’ll run the process over it. It has sensitive data in it, so this isn’t the safest model, but I’ll treat it with care.

#24

[quote=“AgileHumor, post:22, topic:185623”]Is there a solution for non-linux or mac users?

I have 500 devices and looking for a simpleway to Jumpstart OpenHab conversion.

Thanks![/quote]

Perhaps use one of the Linux live file system CD’s? Boot from it into a live linux environment and run it from there?

http://www.linux.com/learn/tutorials/377416:the-five-best-linux-live-cds

#25

Great Idea!

I enabled the Windows 8.1 Feature “Hyper-V” and was able to boot from this ISO without needing to install anything.
http://www.sysresccd.org/Download

Thanks!

[quote=“BOFH, post:24, topic:185623”][quote=“AgileHumor, post:22, topic:185623”]Is there a solution for non-linux or mac users?

I have 500 devices and looking for a simpleway to Jumpstart OpenHab conversion.

Thanks![/quote]

Perhaps use one of the Linux live file system CD’s? Boot from it into a live linux environment and run it from there?

http://www.linux.com/learn/tutorials/377416:the-five-best-linux-live-cds[/quote]

#26

If you can find some sort of XSLT Processor for Windows (eg SAXON), then the steps are really just:

a) In a browser, run http://MIOS_IP:49451/data_request?id=user_data&output_format=xml
Where MIOS_IP is the 192.168.x.x type of IP address of Vera. Save the results to a file (my script calls this [tt]user_data.xml[/tt])

b) Run this XML through an XSLT Processor, using the supplied Transform file ([tt]miosTransform.xslt[/tt]).
If it’s done without parameters, it’ll assume your config is called “house” (this is the default inside the top of the .xslt file)

The output of this, if the XSLT Processor is working correctly, will be the Text file itself. I looked for a “safe” online site to do it, but it was never clear how they treated (and how long they kept) any files/content uploaded.

#27

@AgileHumor was nice enough to PM me his details, and I converted both XML’s into their counterpart openHAB Item files.

Since he runs 2x Vera’s, it highlighted a problem with the uniqueness-detection I was using - so I’ll have to give some thought to how to address multi-Vera, multi Item file, situations a little better.

On a plus note, he uses a lot of Plugins that I’ve not been exposed to yet, so I should be able to glean some better defaults, and ServiceId aliases, for the 1.7 version.

I think the conversion tools are sufficient for check-in at this point, so I’ll go ahead and make a PR to get “this version” into 1.7

#28

New version uploaded, this one:

[ul][li]Omits several internal ServiceId/StateVariable combinations (usernames, passwords, Plugin config, Plugin internals/implementation data)[/li]
[li]Makes Camera1/Timeout a Number Item[/li]
[li]Makes the “System” attributes (at the end of the file) a commented-out block per this thread.[/li][/ul]

#29

These tools have been submitted to openHAB for formal inclusion under PR #2080.

I’ll continue to use Box.net for the “latest” versions, with periodic check-pointing back to openHAB’s GitHub account as needed. Once approved, and merged into openHAB, others will have a more standard mechanism to add extra rules to the conversion/generation system.

#30

@guessed, finally got a moment to implement OH and the item file generator. Even though it will be a few days before I will get back to it…

Whoever may decide to venture…

How simple has @guessed made this process? Extremely!

Thanks mate and I look forward to getting looking at this instead to work stuff.

#31

[quote=“Brientim, post:30, topic:185623”]@guessed, finally got a moment to implement OH and the item file generator. Even though it will be a few days before I will get back to it…

Whoever may decide to venture…

How simple has @guessed made this process? Extremely!

Thanks mate and I look forward to getting looking at this instead to work stuff.[/quote]

Hazah!!

#32

[quote=“Brientim, post:30, topic:185623”]@guessed, finally got a moment to implement OH and the item file generator. Even though it will be a few days before I will get back to it…

Whoever may decide to venture…

How simple has @guessed made this process? Extremely!

Thanks mate and I look forward to getting looking at this instead to work stuff.[/quote]

Cool, great to hear! Looking forward to seeing your try out the Binding also.

The Generator is getting stronger as more people use it and give feedback, so the more the merrier! There are some very exotic devices/plugins out there :wink:

Looks like @watou found an issue in a part of what’s generated so I’ll try to sort that out today.

#33

New version uploaded, this one:

[ul][li]Fix generated Item formatting for Number items, add special-cases for Watt-values
[list]
[li]Number Items use [%.4f] as format[/li]
[li]Integer items separated out to use [%d] as format[/li]
[li]Watt values use [%.1f W] as format[/li]
[li]kWh values use [%.1f kWh] as format[/li]
[/list][/li][/ul]

This should address the problems that @watou saw in the GitHub review feedback on the PR.

#34

This has been approved, and merged into the openHAB 1.7 (current) codeline. I’ve updated the links in the first post to indicate how to get the nightly version (from openHAB GitHub) and the untested version (Box.net)

#35

@guessed, this is maybe more of a question than suggestion.

/* Device - Stairs - Motion */
Number StairsMotionId “ID [%d]” (GDevices) {mios=“unit:main,device:344/id”}
String StairsMotionDeviceStatus “Stairs - Motion Device Status [%s]” (GDevices) {mios=“unit:main,device:344/status”}
String StairsMotionConfigured “Stairs - Motion Configured [%s]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/HaDevice1/Configured”}
DateTime StairsMotionLastUpdate “Stairs - Motion Last Update [%1$ta, %1$tm/%1$te %1$tR]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/HaDevice1/LastUpdate”}
DateTime StairsMotionFirstConfigured “Stairs - Motion First Configured [%1$ta, %1$tm/%1$te %1$tR]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/HaDevice1/FirstConfigured”}
Contact StairsMotionTripped “Stairs - Motion Tripped [%s]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/SecuritySensor1/Tripped”}
DateTime StairsMotionLastTrip “Stairs - Motion Last Trip [%1$ta, %1$tm/%1$te %1$tR]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/SecuritySensor1/LastTrip”}
DateTime StairsMotionBatteryDate “Stairs - Motion Battery Date [%1$ta, %1$tm/%1$te %1$tR]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/HaDevice1/BatteryDate”}
Number StairsMotionBatteryLevel “Stairs - Motion Battery Level [%d]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/HaDevice1/BatteryLevel”}
Number StairsMotionCurrentTemperature “Stairs - Motion Current Temperature [%.1f F]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/TemperatureSensor1/CurrentTemperature”}
Switch StairsMotionArmed “Stairs - Motion Armed” (GDevices,GRoom17) {mios=“unit:main,device:344/service/SecuritySensor1/Armed”}
Number StairsMotionCommFailure “Stairs - Motion Comms Failure [%d]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/HaDevice1/CommFailure”}
Switch StairsMotionStatus “Stairs - Motion Status” (GDevices,GRoom17) {mios=“unit:main,device:344/service/SwitchPower1/Status”}
String StairsMotionCurrentLevel “Stairs - Motion Current Level [%s]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/urn:micasaverde-com:serviceId:LightSensor1/CurrentLevel”}
String StairsMotionArmedTripped “Stairs - Motion FIXME ArmedTripped [%s]” (GDevices,GRoom17) {mios=“unit:main,device:344/service/SecuritySensor1/ArmedTripped”}

/* Device - Stairs - Temperature Sensor */
Number StairsTemperatureSensorId “ID [%d]” (GDevices) {mios=“unit:main,device:345/id”}
String StairsTemperatureSensorDeviceStatus “Stairs - Temperature Sensor Device Status [%s]” (GDevices) {mios=“unit:main,device:345/status”}
Contact StairsTemperatureSensorTripped “Stairs - Temperature Sensor Tripped [%s]” (GDevices,GRoom17) {mios=“unit:main,device:345/service/SecuritySensor1/Tripped”}
String StairsTemperatureSensorArmedTripped “Stairs - Temperature Sensor FIXME ArmedTripped [%s]” (GDevices,GRoom17) {mios=“unit:main,device:345/service/SecuritySensor1/ArmedTripped”}
DateTime StairsTemperatureSensorLastTrip “Stairs - Temperature Sensor Last Trip [%1$ta, %1$tm/%1$te %1$tR]” (GDevices,GRoom17) {mios=“unit:main,device:345/service/SecuritySensor1/LastTrip”}
Number StairsTemperatureSensorCurrentTemperature “Stairs - Temperature Sensor Current Temperature [%.1f F]” (GDevices,GRoom17) {mios=“unit:main,device:345/service/TemperatureSensor1/CurrentTemperature”}

/* Device - Stairs - Light Sensor */
Number StairsLightSensorId “ID [%d]” (GDevices) {mios=“unit:main,device:346/id”}
String StairsLightSensorDeviceStatus “Stairs - Light Sensor Device Status [%s]” (GDevices) {mios=“unit:main,device:346/status”}
String StairsLightSensorCurrentLevel “Stairs - Light Sensor Current Level [%s]” (GDevices,GRoom17) {mios=“unit:main,device:346/service/urn:micasaverde-com:serviceId:LightSensor1/CurrentLevel”}
Contact StairsLightSensorTripped “Stairs - Light Sensor Tripped [%s]” (GDevices,GRoom17) {mios=“unit:main,device:346/service/SecuritySensor1/Tripped”}
String StairsLightSensorArmedTripped “Stairs - Light Sensor FIXME ArmedTripped [%s]” (GDevices,GRoom17) {mios=“unit:main,device:346/service/SecuritySensor1/ArmedTripped”}
DateTime StairsLightSensorLastTrip “Stairs - Light Sensor Last Trip [%1$ta, %1$tm/%1$te %1$tR]” (GDevices,GRoom17) {mios=“unit:main,device:346/service/SecuritySensor1/LastTrip”}

I mean I can do this manually no issues, but there doesn’t seem to be a point to have the separate device that Vera needed if the params are in the root object.

Again, could be an easy hand edit.

#36

I’m not sure of the comment or the question, so can you restate what you’re looking for?

I can see that this “LightSensor” is part of some sort of 3-in-1 (Light, Temp, Motion) and it seems to be repeated (different Device Id’s). During transformation/conversion I don’t look at the Device to see if there are “other” things that might also represent it (other devices) as it’s fairly common for parent-Devices to have/expose similar services as the child devices (sometimes with the same purpose, and sometimes with a different purpose)

Other than that, the LightSensor will be generated differently in the newer XSLT files. It’s now a Number instead of String.

#37

Thanks, just checking. I removed the additional Vera created devices as it was dupe data.

#38

I got openHAB setup on Rasberry Pi with Rasbian 1.6.2. I turn on the exec bit on these file. I am doing everything thru putty.

When run sudo bash miosLoad.sh 192.168.0.5
I got
Loading MiOS Unit Metadata from 192.168.0.5…
Failed to load, Check IP Address supplied.

When run http://192.168.0.5:49451/data_request?id=user_data&output_format=xml using browser, I got the xml result.

#39

Play around a little more
I am running a vera3 and vera 2 in bridge mode both UI5.
When run bash miosLoad on the vera 3 i got the error failed to load, Check IP Address supplied. Browser got xml data
When run on the vera 2 i have no error and got the user_data.xml file

#40

[quote=“bigmonkey70, post:39, topic:185623”]Play around a little more
I am running a vera3 and vera 2 in bridge mode both UI5.
When run bash miosLoad on the vera 3 i got the error failed to load, Check IP Address supplied. Browser got xml data
When run on the vera 2 i have no error and got the user_data.xml file[/quote]

The script shouldn’t need to be sudo’d or bash’d, it internally has [tt]#!/bin/bash[/tt] script declaration, so it should just need the execution-bit to be set on it.

The miosLoad.sh script is very simple, so I’m surprised it’s failing. It would be worthwhile checking that the RPi itself can see the Vera3 box (directly) using “ping” and/or by running it’s internal commands directly (it’s just a wrapper call to curl to fetch the XML)

I’ve seen cases in past environments where some hosts are visible, and others not, depending upon how the Network was setup (in one case, due to an incorrect NetMask)

When run bash miosLoad on the vera 3...

I take this to mean you’re running the script ON the RaspberryPi AGAINST the Vera 3. The script itself is not designed to be run ON the Vera 3.