Well, tomorrow after gardening I’ll have time to implement your virtual http sensors for sure
Very poor indeed!
Both units are going to @LibraSun!! He will make much better use of them than I will have opportunity to.
I am on day 5 and while @akbooer is still working on refactoring the bridge, and I on my side have been adding a few bindings here and there (i.e reporting dead nodes, doorlock last trip, HEM report time, etc), I can see indeed that everything is fixed. The only problem I had from a sensor not unripping… I discovered was due to me having rebooted the z-way server while the sensor sent its untrip frame.
I am still having mental twitches and spasms from my traumatic experience with Vera to want to control things I probably don’t need to… frantically looking at logs, expecting things to have a delay or to not work a certain % of the time… but nothing.
I have been thinking of making my “untrip all sensors” scene work as z-way will not allow their virtual sensor devices to untrip through a command. But… Now I think it is likely completely unnecessary. I wrote a lot of code to workaround Vera’s failures which I can now remove.
As I am looking through logs, I am also learning new things… Not a trace of any Got CANs and… some old devices which I didn’t think had instant status now report them correctly. The got CAN is caused by the vera receiving a frame type it is not expecting and therefore deciding to shutdown its zwave API port of 0.1s. The problem is for the vera is that
- This is completely misdiagnosed. The switch is sending a data frame reporting its status while the vera is waiting for an ack frame in response to its command
- The vera responds in the worse possible way, by shutting down communication which is a strange habit the vera has to always take the only worse course of action than doing nothing: shutting down or reboot.
I am seeing a lot of my security class sensors updating their states and needing to exchange NONCE frames with the controller. This is where the NONCE_get flood from the vera comes from and again, the vera takes the only worse course of action than doing nothing: It blocks its command queue from sending any more commands, I am guessing in an attempt to quiet the network and get the NONCE report frame it was expecting. It does so for 20s or 2s and I haven’t figured out why and when it decides which. This is an attempt to prevent a race condition in the network. The problem is that it is never getting it because the secure class device is a battery powered sensor. It already sent it NONCE report frame and is asleep. That NONCE report frame was likely unexpected by the vera either because it shut down its communication due to a Got CAN or the vera was so slow to react that it was not yet expecting the NONCE frame and threw a got CAN.
Basic fundamental design principles vera is violating:
Never shutdown your coms with the serial API.
Never block the command queue while waiting for a frame. You can pace the commands out to say 1 command every 0.1 - 0.5s but never blocking it altogether.
When you have these, it will force you to deal with the got CAN appropriately and process these frames and figure out a better way to prevent race conditions.
Day 6 of this now blog like thread… And I took the liberty to change its title to make it more descriptive of what it is about.
All is well on z-way and the bridge for zwave. The only one thing which could potentially improve the bridge would be the implementation of async polling.
I was planning on keeping the zigbee network on the vera plus and see how well it fares while having the option of moving to home assistant. As if the vera was begging me to take it down it gave me another exit code 245 luup reload last night, the second one is about 1 week time. In itself, it isn’t a big problem anymore since it is only a zigbee controller connected through verabridge, hosting no plugin or scenes so it could be tolerated. But… why bother to maintain an additional piece of software in the system which I could move to a better stack and library which is already up and running (ZHA component of home assistant?)
The biggest challenge is that there is no bridge to home assistant equivalent to the vera bridge and the zway bridge. Home Assistant is also much more complex with many other device types rendering such a bridge practically impossible so… Given the fact that I have only a dozen of devices I decided to manually bind openLuup devices to their Home Assistant’s corresponding entities. This requires:
- Creating devices on openLuup and setting them up (as switches, dimmers, sensors etc)
- Adopting all the devices previously on vera to Home assistant and setting them up on Home assistant
- Creating scenes triggered by openLuup device actions to send a command to Home Assistant
- Creating automation on Home Assistant to trigger commands to openLuup to update their corresponding device status.
It’s not migration per say since zigbee networks cannot technically be migrated from one controller to another, it is re-creating the network and binding it to openluup manually. The good thing is that it is a lot of copy paste of short yaml and luup code but it is tedious and doable. I can provide instructions for people who would want to do this.
The gain is that I finally am rid of the vera software altogether and have a much better zigbee support (I can now add Tradfi or Hue and any other ZLL protocol and many other devices) with a more reliable platform… To be clear, I am still using the vera plus hardware, which is placed in a central location in the house, and sending both its zigbee port over IP to home assistant and the uzb dongle plugged into its usb port also over IP to z-way and the whole thing is controlled by openLuup.
I would not have done it this way for zwave because of the number of devices. For zigbee though I have 11 nodes and 35 resulting virtual devices. It’s already pretty tedious.
I agree it is not pain free. Any migration will involve some level of pain. The four questions in my mind to sway the decision are:
- How long are we waiting to wait? There are readily more stable platforms today.
- How much confidence do we have on the new platform? This is rather subjective but given the track record and the way this is coming. Mine is pretty low. I see no sign that the fundamental low level problems of the old have all been addressed and neither do I feel that they were even understood and most of the dev team is the same. Neither am I given the ability to test it and as I discovered, the zwave stack is not even finished yet so I have no idea what we are testing?
- How much pain am I willing to go through? As it stands, from all the documentations and directions I am seeing, it will be astronimically more painful to migrate to ezlo’s platform than to the openluup-zway combination.
- What benefits will I get from the new platform? For all the “love it” comments and new stuff being added, we, in the community have been doing them better more reliably and more computationally efficiently over the years. Yes they took more work and there is benefit from making them more accessible and user friendly but they won’t be new, exciting or more reliable. Overall, for my personal use, reducing the potential benefit to 0 or rather negative given the risks from immaturity above.
For me it was a no brainer…
To each his own - my glass is half full
Seriously, you have made immeasurable contributions to Vera and the community at large and I personally respect your decision to pursue alternatives. Your transition further reinforces my belief is that this community, at this very moment, is critical to the success or failure of the platform and as long as I am participating, I will do whatever I can to help ezlo succeed, otherwise I am wasting my time and theirs. In that spirit, if the day comes where I see it as you do, I will transition without looking back and in a manner that ensures that I don’t do anything that could chip away at ezlo’s future success.
I completely agree with you. This community is absolutely critical to the success of eZLO and we have to applaud and encourage the openness of the staff here. I too hope for their success. They just ran out of time for my specific use case. I don’t, by any stretch of the imagination, believe that the task is easy. I am not shutting the door to ever starting a new setup with this upcoming platform. For my current one however, I can’t afford the instability and the time spent on it anymore.
We are all very blessed to have contributors and testers like you!
Full migration is complete and more. I added the icon for the barometer device type which was a zigbee endpoint vera did not support but ALTUI did. It was just missing a good icon. It was quite a bit of work given the amount of devices I have but I have now full access to all the sensors of my zigbee devices. The vera would only see one of the two smoke/CO sensors and as I said was missing pressure measurements.
I am absolutely thrilled by this solution. The system (openLuup/ALTUI/Zway/Home Assistant) all running in one machine has reached rock solid stability, full feature/device support and no latency running over a dozen vera plugins along with pretty much anything you can imagine for integration.
This goes to emphasize how powerful this community (and the home assistant one) are. To @blacey’s point. None of this would be possible without he contributions of @akbooer, @amg0, @rigpapa, @reneboer who wrote all the plugins I couldn’t write myself.
I found very ironical that I started looking to stabilize the setup by suspecting the hardware and ran the whole vera firmware in a much more powerful machine, updating packages OS and kernel to finally end up with using the vera plus hardware with all of the mios software disabled replaced with nearly all community developed software. It’s been a long journey…
It is not to forget that vera started the whole thing and had a lot of brilliant ideas and designs. It just took the wrong turn at some point.
So are you running all this on a Vera Plus as hardware? too funny
Yes, I hadn’t appreciated that either. I thought you ditched all your Veras?
I gave away my 2 test units to fellow member @LibraSun . I kept my oldest vera plus which has quite a bit of bad blocks on its storage and turned it into a remote radio source at the central location of the house. It sends the two zwave and zigbee radio serial ports over IP to my NAS which contains all the software. (I had to change the zwave radio and plugged in a uzb dongle in it to get the zway license).
The flash is a bit too fragile for me to give it away and risk screwing somebody’s system, but using it that way makes it write nothing on its flash drive. I also extrooted it. It is a much better (re)use and gave the hardware much more value than with the current mios firmwares.
I even wrote a script to modify the vera plus to use it in this fashion.
It literally never reboots and is a workhorse when not running anything mios on it.
Optionally you could even make it a wifi AP as well but I opted not to do that…
Thank you for the detailed write-up @rafale77!
So it seems to me in order to use Z-Way you need z-way supported hardware (either UZB sticks or the RazBerry) , it does not sound there is any re-use of the radio on the VeraPlus possible with the Z-Way plugin, even with a license key?
I like to try the RazBerry board, just because it seems to me it has bigger antennas than the UZB (and no license required), do you think it’s possible to export the VeraPlus z-wave network config and import it into the RazBerry (allowing for a seamless transition without changing device ids and device names), or would it require starting from scratch basically?
I did it!
But to do that, you need the UZB! The migration path will be vera -> UZB -> RAZ
The licence key for z-way is stored in the hardware so no it isn’t possible to use the vera’s radio. My script is more useful for people who would want to use home assistant (both zwave-zigbee) or openhab, homeseer, domoticz… for zwave.
Please read my readme draft for which I posted a link a few posts above which explains this. You actually have a couple options I described there and I highly recommended @DesT to do it through the UZB even if he wanted to have the RAZ as the final host to keep the controller id at #1.
Yes thank you very much @rafale77 and @DesT.
I read the full document and also the links to using ser2net. Since my plan so far was to use a Raspberry and RazBerry add on board I Wanted to see if I can skip the UZB purchase. But I can set up the network again, I have about 20 devices or so.
I will NOT recommend for sure, to use zway directly on the raspberry using a SD card. The SD card will die very fast 'cause of the IO from zway.
You can boot from an USB drive instead OR as I did, use a VM for zway and the RaZberry just for the ser2net
Ah, that is good info, thank you! I had not counted on the IO being a problem.
Have anyone success running it off a USB drive on the PI4,
I have installed buster and used the “usb-boot” tool (https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=196778&hilit=usb+boot.zip). But after installing z-way I can download apps in Z-way or upgrade using the management tools.
2GB sd card, and a 128gb USB SSD. openLuup works perfectly in this setup
Do you mean running zway off of a usb flash drive?
On a Pi, booting from a USB drive is indeed the preferred option, very similar to the vera extroot idea, the sd card or on board flash are too limiting and unreliable.
What error are you getting? What management tool?