@Rigpapa: do you have plans to adapt your excellent Reactor plugin for the new Ezlo firmware ? If yes, any idea about time schedule ?
It’s been a couple of weeks since I did any work on the new firmware. When I last left it, it was not close to being what is necessary to create Reactor or any of my other plugins. There has been, I believe, just one release since I paused, and that is advertised to address one of my important needs, which was actually a generic need likely applicable to many plugins, not just mine. They are still filling holes in the API. It’s great to get past this one hurdle, but there are many others on the critical path to making a supportable user-facing product. The current process is “find a fire, put it out”, so I suspect it’s just going to be fits and starts until some day there’s enough coverage that it hangs together. Frankly, it’s frustrating and exhausting, and that’s why I tabled it for a couple of weeks.
Timing-wise, no, there will not be Reactor when eZLO releases the new firmware, based on what I believe was the most-recently stated timing–August/September. I don’t see how that would be possible.
So from a developers point of view how does the new Ezlo firmware APIs compare to the old Vera firmware APIs? In layman terms.
Are Ezlo going in the right direction with their API?
As a user I want to see the functionality we have today with Vera, where we have a HTTP API we can send one line http commands to the Ezlo hub to run scenes, control devices, set variables etc from other devices on the LAN.
I am surprised we haven’t seen any web GUI yet aka UI8. And they are saying they want to release the products in August / September. Seems a bit optimistic to me.
Here is the list of your requests from us:
- Http client - done and already available in previous build
- Mechanisms for using code from one plugin in another - will be available in this week release
- Possibility to install lua libs - will be available in this week release
- Devie settings API - preparing API description and will make it available on Wednesday, implementation is done and will be available in next release.
Well, it depends. If they will still release code every two weeks, missing things could land very quickly, even if missing from day 0. I’ve starting to take a look to port a couple of my plugins, as @rigpapa, but it was too early a couple of weeks ago and now, as @reneboer suggested, it’s still missing a real guidance on how to customize UI.
Thanks, Oleh. This is all good progress, and I’ll be happy to play with these things and give you feedback. I do wish it had not taken two months to get these issues addressed, though. You guys may be "sprint"ing on your side, but for me/us it’s anything but.
Any news on the following from my notes?
- No persistent runtime environment across script invocations. Any state that the plugin needs to preserve across scripts or script invokations currently has to be pushed through the storage API, which is tedious and will be a source of bugs. I had suggested making the environment that the
startupscript executes in the saved and assigned environment for all subsequent script invocations. That is, if I create a global in the
startupscript, that global should be visible in all of my event handlers, etc.
- No unified logging module with severity and possibly facility filtering (this is a common feature of modern firmware systems). Ideally this would include logging via syslog/UDP.
- Inconsistency in behavior of add_device(): throws hard error if missing device_type_id or gateway_id in object passed, but logs error and returns nil if category is not provided. This inconsistency is repeated across the API in various calls and is, frankly, maddening. Please adopt a single, unified strategy for error handling. Given that
pcall()/xpcall()makes code messy and more difficult to trace and debug, my preference is for consistent use of return values. Specifically, I have a strong preference for the
value,errortwo-parameter return status style, where
nilwhen an error occurs and
erroris a table, not a string, containing a documented, stable
codeand a text error message. The
codebeing stable and documented is vital to proper error handling.
- Inconsistency: in
add_device(), the device type ID key is named “device_type_id”, which is consistent with other fields, but in the structure returned by core.get_devices() it comes back in camel-case as
deviceTypeId. Same for
batteryPowered. This is maddening and a guaranteed source of a lot of casual errors in future, unnecessarily. Strongly recommend that go one way or the other with your naming convention. Do it now, before there are a hundred plugins that you’ll break by changing it.
- I recommended that you establish, now and firmly, a convention for the IDs of plugins (and their directory names). For example, I would expect to name my plugins with my namespace, as in “rigpapa-reactor” or “toggledbits-reactor” (for directory name and plugin ID in config.json), so avoid collisions with other plugins. Eventually, somebody is going to use the same name for two very different things. If you include a namespace component to the required names/identifiers, you solve a lot of future problems up front. I think the same convention should be applied to device type IDs and any category/subcategory created by a plugin.
- Also a consistency issue, there seems to be some variation in the structure of the table/data received by event handlers. The documentation describes what appears to be a “standard” structure, yet
timerevents have a very different structure, for example. This should also be fixed immediately and the pain of breaking taken entirely in pre-release.
- I’d like to see notice (to the plugin) of the system shutting down (completely or in preparation for a restart).
- Millisecond (or better?) time clock access, since we don’t have access to LuaSocket.
- We need system and runtime environment information, including location of controller (lat/lon), current timezone and offset, localization parameters (time and date formats, etc.), current house mode, firmware version, hardware platform, etc.
- The documentation for
gateway_idas deprecated yet the code enforces it as mandatory. [Andrew said it’s a bug]. Fixed yet?
- What the difference (in add_device()) between “type” and “device_type_id”? What are the constraints of these fields? I was told the type field was going to go away; it also seems to be required and so stands in the way.
- Documentation does not adequately document relationships between objects or data, in particular category, subcategory, device types, and items. We need a comprehensive list of what items are available for what devices. The docs last seen just have separate lists of the various values, but do not establish any relationship between them.
- Your network module needs a WebSockets implementation. It would be nice if all protocols in the network module were loadable, rather than the module becoming a large monolithic carrier of all protocols. The “script” model of the firmware seems ill-suited to building protocols for use with the network module (or any other).
- The structure of scene data is… wow… does it really need to be that complex?
- Every firmware update wipes out all plugins and configuration.
- At last look and discussion, there was no infrastructure in the firmware for it to create UIs for its configuration settings and other plugin-specific, non-device presentations (i.e. there’s no way for me to implement the condition, expression, or activity editors for Reactor; no way for DelayLight to be told what devices to supervise for triggering or control when triggered). This was apparently still in the design and consideration phase just a month ago. Can you update progress? Is it going to work for mobile and web remote and local?
- I haven’t studied the subsystem in detail, but the cross-platform product apparently being used to develop the apps for iOS and Android doesn’t have a web interface beyond its development/test environment, which would require a web user to install additional software (companion apps in browser, etc), and would then present a mobile interface in the web viewport. This is a great concern for many reasons.
- Skype chat is a horrible way to capture and track bugs/requests. We need a bug/issue tracking system. If the length of this list is a surprise to you, this is exactly why.
Most of the foregoing was discussed before 4/28.
Of course, my overriding feedback/criticism is that many of these things, including the four things that just took two months to get, were foreseeable by studying the capabilities of the current Vera firmware and at least considering them as core requirements to cover in the new firmware. This clearly has not happened, and it’s a great disappointment. If the strategy was to use me and the rest of the plugin developers as a filter to tell you what parts of the old system need to have analogues in the new, the time for that was about 18 months ago. Now everybody… you, me, reneboer… we’re all just firefighting. And trying to have these discussions on a Skype chat with people in at least three timezones and everyone else trying to get their fair share of time as well… it’s exhausting, and in large part why I needed to take a break.
For the record, I do not blame you for any of my grievances here, Oleh, and I know you have a lot of demands placed on you. Thanks for everything you are doing to support us.
It’s missing a lot of core functionality that they could easily have foreseen themselves, or gotten explicitly from us 18 months ago. It’s fixable, but it didn’t need to be this way, and it places everyone under stress to have to handle important functional requirements this late in the cycle.
They’ll get there. Honestly, it’s hard for me to say they are going in the right direction with their API, because I don’t see that they have one. It appears outwardly like they sprinted to code rather than understanding the full breadth of functional requirements. I think they may even have thought at one point that they were going to do everything themselves, so they didn’t feel like they needed outside involvement, they could just address their own needs internally and quickly. Now that we’re approaching the two year mark since the acquisition, I’m guessing they see they bit off quite a bit more than they could chew, and replacing a system that, while buggy and hard to work on, carried a decade of maturity in engineer and customer experience, is not easy to replace without some serious navel-gazing beforehand. But it was clear from the first look, their entire perspective on building plugins for the system seemed to be from the thinking that building their own ZWave module as a plugin would tell them what they needed to know. That certainly gave them some information, but far from a comprehensive view, and the system’s capabilities then (and still now) reflect that. For example, the ZWave module owns all of its devices, and doesn’t need to affect any device outside of its control. Reactor, on the other hand, owns almost no devices, but both reads and manipulates the devices of other subsystems. There was originally no way for Reactor to do any of that (I’ve been told that’s been addressed, but I haven’t verified it). Both sides have tunnel vision; it’s why we should have been talking together about this 18 months ago, not now.
Well, I think you have the latter (HTTP API), but definitely not the former. Also, security has, I think, upped the ante on using the HTTP API, so I’m not sure it’s going to be something you do as casually as it has been on Vera in the past.
Thanks for your insight and efforts. Sounds rather disappointing however. Where are the original developers that worked for Vera Control LTD before the take over? Surely they knew what types of requirements 3rd party plugins would require and they knew the old architecture and how it worked. @melih Why weren’t those guys heading the new development teams?
If they have increased the security on the user HTTP API and it now needs sockets and all that I won’t be happy. Vera is simple, we can send simple one line HTTP commands to Vera “Luup Requests” if I can’t do the same on Ezlo firmware its going to break lots of my integrations I have done with other LAN devices. Its not like we haven’t told them already we still wanted this functionality.
Here is just one good example. Blue Iris camera software, I have some virtual motion sensor devices in Vera, when a camera in Blue Iris detects motion it sends a one line HTTP command to set the tipped variable to 1 for that virtual motion sensor device in Vera.
And after so many seconds have passed Blue Iris sends a second HTTP command to set the variable back to 0.
If you can no longer do this on the Ezlo hub then that integration is just knackered.
We are thinking on it now, will back to you with the answer asap.
Planning to have unified logging module in the next build, after this week release - July 16th.
Unified strategy for error handling is very important for us and update is in progress now. Preparing the documentation and will send it to you about how it will work in the next build, after this week release - July 16th.
Its aready done for Zwave API.
Planning to have it in the next build, after this week release - July 16th
It is in our roadmap and will be available in next releases. Approx. end of July.
Latest release already has time clock access.
Implemented and will be available in this release - July 6th.
Will send you latest revision of the documentation with fix - July 6th.
Will send you latest revision of the documentation with updates of this information - July 6th.
You mean WS server ?
Http client is a separate module and now we are implementing http server.
In current implementation with the websocket API you can execute any lua script.
Its fixed and will be available in this release - July 6th.
Thanks for all your feedbacks.
Not a surprise, we have a special place in confluence with the list of raised questions.
No, I mean a WS client. I need to be able to connect to, for example, an AWS lambda that’s a ws endpoint. Available? I haven’t noticed, so it may be, I just haven’t seen it.
OK, but please do all the breaking things for everything before your stated August/September release. You have an opportunity to perfect right now that you will never again see.
I can’t say I’m happy to hear this, given that it was one of the first points I brought up back in April. I know there is some resistance to this because of the first generation Atoms and the memory constraints, at least, that’s where it seemed to be coming from at the time, and I was told then that it would be considered. So this response tells me that it hasn’t been, and as I said then, it’s an important architecture issue that I feel strongly is going to have a very, very high impact on system performance in the long run. It’s more than just an inconvenience. It speaks to the repeated processor time of having to completely restore or rebuild data structures on every single event notification of every kind. Anything you do once can happen quickly enough to make it seem it’s not an issue. When you’re doing it several hundred or thousand times a minute, the dragon will rear its head.
@rigpapa, thanks for your answer.
It is in our roadmap and will be available in next releases - end of July.
It will be available in the next release - July 16th
The key point for us is performance and now it works much faster because lua script is not taking additional resources from CPU or RAM if we don’t need it.
During realisation of saving state the FW will save all conditions and states declared in this environment. If a developer needs just some single state from the script - he cannot control it and it will take much more time to save and restore this state.
Now a developer can make a choice of what exactly he needs without other unnecessary states and data. This way of handling states and data already working in our big plugins for Zwave and Zigbee and they are working extremely fast.
One of the easy ways to work with it – just create additional script to save and restore data and execute them in the beginning and at the end of each script.