openLuup: Suggestions

Plugins & Plugin Development openLuup
#41

Would it be possible for the extension plugin to schedule backups of the directory structures to defined/default landing zones (locally, remotely etc.) at specific intervals ? I know this could be achieved at a low level but I’m thinking of the novice who’s not familiar with Linux. So giving others the ability within a plugin to perform this operation may bring comfort to others to know that they can fall back to a known working backup that’s being managed on their behalf.

[ol][li]Control over interval (daily, weekly, monthly) and time the backup occurs.[/li]
[li]Define landing zones or default to a backup directory locally.[/li]
[li]Variables (within the plugin) that indicate when the last backup occurred whereby this could be checked within the plugin.[/li]
[li]Restore from a backup (if possible) and restart[/li][/ol]

#42

All very possible, of course. But what are you backing up and what failure are you trying to minimise? It sounds like more than just the user_data.json file (which currently gets checkpointed every 6 minutes, but could easily be versioned, like the log file.)

#43

openLuup is as solid as a rock - this we know and love. I’m trying to avoid having a hardware failure. These systems are my first SSD’s and I have no idea how long they’ll hold out being in a fan-less PC. So at best, I’d might have a backup from months ago. It’s just one of those things I thought about today… You know, like remembering to back things up…

#44

Oh, so you’re talking a full OS backup utility within openLuup? Sounds like a new plugin to me, but do-able. The problem is always the user interface. Generally, more options == harder to use. Appropriate media would have to be online.

#45

Merely a suggestion, I haven’t seen it mentioned and it may be one of those things where no one really needs it. Then again, we often don’t need something until that something nips us…

#46

I’m proposing to make the openLuup HTTP server (on port 3480) serve all the icon files. AFAIK this means that a port 80 server won’t be required at all and the icons should work if remotely logged in. You wouldn’t need the [tt]/www/skins/default/…[/tt] directory tree and the icons would all be found in the [tt]/etc/cmh-ludl/icons/[/tt] directory, where the [tt]get_files[/tt] utility puts them.

Additionally, and in line with the above, I may also consider the use of the [tt]/etc/cmh-ludl/files/[/tt] directory as an alternate place to look for .xml, .json, .lua files. This is different from Vera behaviour, but I’m not sure whether there is a downside. The upside being that this is also where the [tt]get_files[/tt] utility place those.

Thoughts / complaints?

#47

It would allow to separate internal things of the plugin files.

But as it changes the way of storing files, perhaps openLuup should expose a special param in the namespace luup, to let the plugins to change their behaviour if they directly use these files.

#48

No, I’m dead against this… openLuup should behave like a real Vera from a programming POV. I was trying to think about how to make file management a bit easier. We do already have a problem with plugins manipulating files through trying to uncompress them, etc., and truly the best way to address that would be to write a dummy [tt]pluto-lzo[/tt] which does nothing but add a file extension. If you’ve got a plugin that looks for its files in [tt]/etc/cmh-ludl/[/tt] then that’s where they should be. For the vast majority, however, they don’t know or care about those files.

So probably a bad suggestion of mine.

#49

Hi akbooer,

For many platforms you can make a logical link between folders so if you need them in /etc/chm-ludl/icons you can make a link to the /www/skins/default/icons so both would have identical content.

I have plenty of other uses for a port 80 http server on my PI so I do not see a down side there.

Using the /files folder may have implications for plugins that need to rewrite some files (like mine) as they will do that in the /etc/chm-ludl/ folder. The files folder is what you pull from (multiple) Veras and may not be the identical files in all cases, for me they are not. It does save copying some files manually at setup and new versions, but I do not mind too much. Having said that the most perfect would be a UI function to pull all the latest files for one specific plugin from one of your Veras to the openLuup unit. I can see if I can spend some time on that the coming weeks.

Cheers Rene

#50

Yes, that’s actually the mechanism I suggest in the User Guide for linking [tt]/www/skins/default/icons[/tt] and [tt]/www/skins/default/img/devices/device_states[/tt]. However, some servers don’t allow you to link outside of the [tt]/www/[/tt] tree to avoid security issues, so a link to[tt] /etc/cmh-ludl/icons/[/tt] may not work.

I have plenty of other uses for a port 80 http server on my PI so I do not see a down side there.
Ah, that's not the point. Of course a port 80 server is generally an integral part of a system. The issue is that if you make a tunnel to port 3480 somehow, for remote access, then you do not have access to port 80. So arranging that AltUI does not use port 80 at all is definitely a plus.
Using the /files folder may have implications for plugins that need to rewrite some files (like mine) as they will do that in the /etc/chm-ludl/ folder.
Yes indeed. That was the point I was trying to make with [i]"If you've got a plugin that looks for its files in /etc/cmh-ludl/ then that's where they should be."[/i]
The files folder is what you pull from (multiple) Veras and may not be the identical files in all cases, for me they are not.
Yes, this is an excellent point - I have openLuup linked simultaneously to UI7 and UI5 systems and it's a bit of a mix and match to decide which .json files and icons to use.
Having said that the most perfect would be a UI function to pull all the latest files for one specific plugin from one of your Veras to the openLuup unit. I can see if I can spend some time on that the coming weeks.
That would be great. The getfiles utility is very basic - I just put it together to get started and haven't touched it since (too much else to do!)
#51

Just one further thought on the [tt]files/[/tt] directory…

…this is entirely analogous to the situation between [tt]/etc/cmh-ludl/[/tt] and [tt]/etc/cmh-lu[/tt] … if the file is not found in the first location, then the second one is searched. So I’d now propose to emulate this behaviour (and, possibly, rename the [tt]files/[/tt] directory to [tt]cmh-lu[/tt] ?)

#52

Perhaps, you should use /etc/cmh-lu as a “read-only” folder and put all openLuup stuff, and use current /etc/cmh-ludl for extra files for plugins (dl for download I guess)

#53

On the current development version, you use a package “pretty” in “wsapi.lua” (surely from GitHub - lunarmodules/Penlight: A set of pure Lua libraries focusing on input data handling (such as reading configuration files), functional programming (such as map, reduce, placeholder expressions,etc), and OS path management. Much of the functionality is inspired by the Python standard libraries.).
Is it for debug purpose ? LuaDist (on Windows) lacks this package, and I have to comment this.

#54

You were not the first to find it! It’s fixed already. Yes, part of the testing process. Shame on me (again.)

#55

Is it possible to have a backup of “user_data.json” ?

When doing the backup will then have to be specified, but I have had several times a crash of this file (no more content) when I stop brutally openLuup (kill the process)

#56

Well, I would say that you only have yourself to blame! Why would you kill the process? Just use the [tt]&id=exit[/tt] request.

It is a fairly large file, and I imagine that if you actually hit the few tens of milliseconds that it takes to write (which would be unlikely, since it only happens every six minutes) then it might well be empty.

I could version it, as I do for the log file and as Vera does, but it wouldn’t be compressed. However, I don’t see disk space as a problem these days since we are not actually on a Vera!

#57

I use for the moment openLuup on a windows machine, and sometimes I stop the .bat that has launched openLuup.

When the problem occurs, openLuup is in a loop of reloading, and in this case if I kill the process, I lost the file.
Sometimes, it’s my fault (the loop), and one time it was after (or during) an upgrade of ALTUI.

#58

Yes, interrupting an AltUI upgrade is not a good plan. Understand your reasons for wanting this. Would versioning help? Although I still can’t see what’s wrong with a separate, manual, backup, or, indeed, starting again from a startup.lua file?

#59

Apologies if that topic has already been covered (but my searches led me to be believe it hasn’t)… Would it be possible to maintain a list of plugins that have been tested to work with openLuup or conversely proven not to work ?

I am in the process of migrating all (possible) plugins and scenes from my Vera Lite to a Raspberry Pi running openLuup and have been stumbling on 3 plugins that I could not yet migrate…

  1. Smart Virtual Thermostat: runs but cannot read sensors or actuate heater switches… I suspect this is due to the numbering offset of devices in openLuup that do not get parsed properly in the Lua implementation file but did not yet have time to probe/debug this… Will report my findings later.

  2. Weather Underground: the D_ xml file is encrypted so does not run within openLuup… I am writing a small plugin based on the pusblished WU API to at least capture current temperature and humidity (that I need for some other devices and scenes)… Will post the files when it reaches beta stage if this is of interest to anyone.

  3. System Monitor: rather a “nice to have” than a “must have” but by design it cannot read from the remote Vera. If I find the time (and get approval from the original plugin developer to reuse much of his code), I am thinking of splitting the plugin into a) a lua module with a timer (not a device as my goal is to free memory) on the vera that serves a json file with the relevant data and b) a plugin on the openLuup machine that reads that json by an http GET and displays it exactly like the original plugin… Any comment on that approach welcome…

Thanks.

#60

Well, it has been asked before, but not adequately answered! It’s possible to do, but high maintenance, and the forum is littered with attempts to do similar for other systems (eg. UI5/6/7 and now the other radios on the Plus) all of which fall quite rapidly into a state of disrepair. I’m afraid I simply can’t be an expert on everybody else’s plugin!

You would be more than welcome to start a thread and maintain it. It would mean that both you and I could edit it, but nobody else. This is a limitation with this forum software.

There are some general rules: nothing encoded or licensed will work; ones that expect and manipulate compressed files; anything that reads or writes raw ZWave data to device #1; … There are threads here on generic classes of plugin, eg. AV and alarms.

I am in the process of migrating all (possible) plugins and scenes from my Vera Lite to a Raspberry Pi running openLuup and have been stumbling on 3 plugins that I could not yet migrate...

Happy to help. Always pleased to hear that another plugin IS compatible.

1) Smart Virtual Thermostat: runs but cannot read sensors or actuate heater switches... I suspect this is due to the numbering offset of devices in openLuup that do not get parsed properly in the Lua implementation file but did not yet have time to probe/debug this... Will report my findings later.

Maybe numbering, but more likely the fact that the parent/child relationship is not maintained by bridged devices… everything is a child of the bridge. This is something I must change, and don’t see a fundamental barrier in doing so.

2) Weather Underground: the D_ xml file is encrypted so does not run within openLuup... I am writing a small plugin based on the pusblished WU API to at least capture current temperature and humidity (that I need for some other devices and scenes)... Will post the files when it reaches beta stage if this is of interest to anyone.

Yes, that would be useful. Early versions of this plugin were not encrypted, but later ones are. Actually, I had been thinking of adding WU to the Netatmo plugin that I wrote. It would be very straight-forward to do, since all the infrastructure for child devices, etc, is in place.

3) System Monitor: rather a "nice to have" than a "must have" but by design it cannot read from the remote Vera. If I find the time (and get approval from the original plugin developer to reuse much of his code), I am thinking of splitting the plugin into a) a lua module with a timer (not a device as my goal is to free memory) on the vera that serves a json file with the relevant data and b) a plugin on the openLuup machine that reads that json by an http GET and displays it exactly like the original plugin... Any comment on that approach welcome...

It rather depends what parameters you need. CPU usage and memory are key and trivial to obtain. Other things, like logs, are definitely harder. It also depends on how successful you are in migrating the rest of your logic and plugins to openLuup. The idea, of course, is to relieve Vera of so much of the work that it just runs and runs without any attention. Certainly, most of mine, now, are just used for ZWave device access and often run continuously for two or three weeks without spontaneously reloading.

Does any of this help?