openLuup: Version Log

akbooer · August 22, 2018, 11:34am

Master Branch: 2018 Release 8.10

I’ve just merged the lastest development branch with the master to roll-up all the various changes and bug fixes which have accumulated.

This is not quite the master release I wanted to make, but it does fix a couple of significant problems, and, of course, include the latest things like Data Historian. However, it doesn’t include digest authorization in luup.inet.wget() or a couple of other things I wanted to add. It seemed necessary, though, with recent interest in openLuup, and in particular fixed a long-standing install error.

I’ve been a bit short of quality development time recently.

Thanks to all who have helped to make improvements over the last few months (and before!)

akbooer · November 15, 2018, 11:10am

Development Branch: 2018 Release 11.15

Fixes synchronisation issue between system Mode attribute and openLuup HouseMode variable if luup.attr_set() was used to change mode. Thanks to @DesT for pointing this one out.

Other minor code improvements.

akbooer · November 27, 2018, 3:54pm

Development Branch: 2018 Release 11.26

enables historian mirroring to external Graphite and InfluxDB databases via UDP
implements missing “actions” request

In addition to the internal Graphite database used by the Data Historian, all archived variables can be mirrored to an external Graphite and/or InfluxDB database. In order to make this efficient in terms of CPU and network resources, the transaction-free UDP protocol is used, avoiding any possibility of system deadlock.

InfluxDB instructions for configuring a UDP port are here:

[UDP protocol support in InfluxDB | InfluxDB OSS 1.7 Documentation](InfluxDB supported protocols)

DataYours may also be used as a destination for the Graphite historian mirror.

To enable sending to Graphite or InfluxDB, the following openLuup system attributes (for example) with IP and port number should be set in the Lua Startup:

luup.attr_set ("openLuup.Historian.Graphite_UDP", "127.0.0.1:2003")
luup.attr_set ("openLuup.Historian.InfluxDB_UDP", "172.16.42.129:8089")

Either of those target databases may then be queried for archived variable data. Grafana queries work well, with auto-population of variable name menus.

As ever, please alert me of any problems.

akbooer · January 17, 2019, 3:35pm

Master Branch: 2019 Release 01.17

New baseline release for 2019. Rolls up previous 5 months of development branch changes.

akbooer · January 22, 2019, 8:07pm

Development Branch: 2019 Release 01.22

Schedule VeraBridge remote actions using <job> rather than <run> invocation.

This addresses a technical issue with remote Veras not being able to keep up with openLuup systems running on faster processors. It mitigates the lack of a properly managed command request queue on Vera and apparently decreases the chance of a Vera restart.

Most users should notice little difference, but, as ever, if you find a problem, don’t hesitate to raise an issue.

akbooer · January 29, 2019, 10:28pm

Development Branch: 2019 Release 1.29

console uses HTTP tables
console CSS in virtual file system
VeraBridge can link to openLuup systems

The look and feel of the console pages can now be changed through modifications of the openLuup_console.css file in the virtual file system file.

akbooer · February 12, 2019, 8:32pm

Master Branch: 2019 Release 2.12

improved Console HTML pages
VeraBridge links to openLuup as well a Vera systems
fix Graphite API CGI Unix epoch time handling (used by Grafana)
fix SSL tslv1.2 error in openLuup_install.lua script

akbooer · March 25, 2019, 9:47am

akbooer · April 17, 2019, 5:07pm

akbooer · May 2, 2019, 3:58pm

Development Branch: 2019 Release 5.2

This release incorporates a significant addition for plugin developers: CPU time logging.

A number of features and tools support this:

plugin devices now have a new attribute cpu(s) which is constantly updated, with microsecond reolution, and available at a code level using luup.attr_get (“cpu(s)”, devNo)
Console > Scheduler > Jobs page now includes a table column for CPU times, and the table is sortable by clicking on the column headers
Console > Scheduler > Startup Jobs page has now been renamed to Plugins and has an additional table showing the total system CPU load in % (since last Luup restart) and the cumulative CPU time per plugin device

Note that plugin startup jobs generally run for a short time and terminate (although they may schedule later processing using luup.call_delay() or suchlike.) The Startup Jobs table is, therefore, essentially static, and records the exit state and any failure message.

The Plugin device CPU usage table, on the other hand, updates to reflect the same per-device time as the new device attribute cpu(s).

Most jobs are run on behalf of the system (typically HTTP server requests and scene timers) but the job tag of a plugin action runs as part of that device. The Jobs page shows current and recently completed jobs (which remain in the list for three minutes after termination.)

Other internal updates are also included in this release.

akbooer · June 2, 2019, 5:40pm

Development Branch: 2019 Release 6.2

This release incorporates a couple of changes discussed on this thread:

Includes:

missing serviceId in device_create() or append() sets attributes, not variable
device category_num set by device type, if otherwise missing

Note that, after further testing, I’ve not made a change to luup.variable_set(), because Vera doesn’t allow a missing serviceId there.

Haven’t yet done anything with subcategories.

Also, the console menu structure has changed a bit, with tabs to connect groups of related pages . It’s also totally configurable by a JSON file, so you can have any set of menus you like. You can revert to the old style if necessary. The home page gives an index allowing quick access to any menu page.

akbooer · June 7, 2019, 5:46pm

Development Branch: 2019 Release 6.7

I have finally bitten the bullet and switched to a CSS Framework for the console pages.

I’m using W3.CSS which is very lightweight, fits right in with the openLuup ethos, and claims to be…

smaller and faster than similar CSS frameworks.

When run for the first time it downloads a local copy of the CSS file to the www/ folder in the openLuup tree. It’s only 23 KB. This allows seamless operation in the case of operating offline.

The look & feel has inevitably changed, hopefully for the better, and it makes maintenance and updates much easier. The influence of AltUI is probably plain to see. The menu structure remains fully configurable with an external JSON file, should you really want to do that. There are three pre-defined sets to choose from.

Not everything is quite fully implemented, but there is more functionality on the way.

Happy to receive feedback on this change, as ever.

AK

akbooer · July 25, 2019, 1:39pm

Development Branch: 2019 Release 7.25

This release contains some significant updates to a number of modules.

The most obvious changes are updates to the default console L&F. The page and menu layouts are something of an homage to AltUI, without which openLuup would not exist in its current form. This is not to be confused with a replacement for AltUI… it is not. The console has always been about viewing and understanding some of the internals of the system and installed plugins, as an aid to debugging. It is not about presenting your devices in a convenient form for controlling your home, and pages are not dynamically updated as device variables change.

Again, taking another cue from AltUI, the Ace editor is used extensively to view internal structures and files in Lua, XML, and JSON formats. Ace may be dynamically downloaded from a Content Distribution Network (default is Cloudflare,) or loaded into a local folder which may be used offline. If neither is available, the system will default to simple HTML textarea elements.

Internally, the XML module has, once again, been refactored, this time to unify parsing and serialisation of XML, HTML, and SVG. Many device files needed by the basic system are generated directly at startup using the internal DOM (Domain Object Model) or JSON encoder. This sets the scene for a fully stand-alone system (without an attached Vera) which at the moment is only possible after downloading Vera device files.

Session cookies are used to persist a number of openLuup states, such as which room to view and sorting/filtering options. A total of three Lua Test windows allow parallel debugging of different snippets of code. A key point here is that the test code is actually persisted in the user_data.json file, and not the web browser, so different sessions from different browsers will actually share the same code, which is checkpointed with the rest of the user data (every hour, or at system reload.)

The underlying openLuup engine is little changed, apart from some preparatory work to improve the HTTP server, so I’m hoping that there are no bugs introduced which should affect the regular running of the system.

As ever, feedback welcomed.

akbooer · August 1, 2019, 5:15pm

Development Branch: 2019 Release 8.1

This release includes significant refactoring of the openLuup.http module:

openLuup.http now split into separate client & server modules
luup.inet.wget() now supports Basic and Digest authentication

Username/Password parameters may be defined in either the wget() parameter list or embedded in the URL.

akbooer · December 28, 2019, 1:15pm

Master Branch: 2019 Release 12.27

Roll-up of all 2019 incremental developments
much extended Console pages
very many incremental improvements

Thanks to all users/developers who have contributed to improving openLuup in so many different ways over the last year (and more.)

AK

akbooer · February 22, 2020, 12:55pm

Development Branch: 2020 Release 2.22

This development release brings a significant change in the way which VeraBridge handles devices which go missing on the remote Vera.

If, on restart, the bridge discovers that any remote devices are missing, they are not deleted from openLuup, but moved to Room 101 where they remain until manually deleted or, upon a later restart, reappear on the remote Vera.

This means, that these devices are not deleted from any local scenes which use them as triggers or actions, affording the possibility to update these scenes with current devices, should that be required. Thanks to @DesT for this suggestion.

The openLuup plugin has a new action EmptyRoom101, which will permanently remove any devices in the room. The room name may be changed to anything, but the id number is always 101.

Room 101 is implemented as a generic openLuup functionality for any bridge, which will include future implementations for other hardware (e.g. ZWay.)

As usual, you can update to this version, simply by typing development into the update box on the Plugins page.

akbooer · April 3, 2020, 9:57am

Development Branch: 2020 Release 4.2

Several openLuup console changes:

Plugins page links to GitHub repositories
App Store page gives direct access to Alt App Store (not via website)
Luup files browser with category filter
Command line tool
some menu layout changes to accommodate above

The App Store page (openLuup plugins only, obviously) provides a very responsive way of viewing published apps, and, again provides links to their respective GitHub respositories. One small caveat here, is that the database is not updated in real-time from the AllAppStore cloud, so if you publish a new version, it won’t immediately appear.

These changes are from internal development tools which I’ve had for a while and I find useful (particularly when developing plugins) so I thought I’d make them generally available.

akbooer · April 13, 2020, 2:15pm

Testing Branch: 2020 Release 4.12

It’s unusual for me to make a release to the testing branch, but this is a low-level change with potential impact (both good and bad.)

I have made some updates to the JSON module, which really is at the heart of everything. I have tested this extensively – over 100 separate unit tests and many different files – so all should be well, but fair warning.

A potentially huge benefit to plugins such as VeraBridge and ZWay is that the openLuup JSON module will use the C-library based cjson module, for decoding, if it is installed. The advantage of this is that it is, at least, 10x faster than the pure Lua openLuup module, so CPU overhead drops dramatically for the JSON-heavy Vera and ZWay APIs.

The openLuup module is, nonetheless, still used for all encode calls, because it sorts table parameters and prettyprints its output.

On a RPi, the cjson module may be installed already, but you can check it by doing the shell command

$ sudo apt-get install lua-cjson

I would appreciate any feedback on this change before it goes main stream.

Thanks,

AK

akbooer · April 14, 2020, 11:40am

Development Branch: 2020 Release 4.14

Incorporates updates to the openLuup.json module as introduced in the previous testing release. Thanks to @rafale77 for rising to the testing challenge! As a result there have been a few adjustments to the module, but just updating to this system should provide significant performance improvements for VeraBridge and ZWay plugin users, since JSON decoding is typically 15x faster when the cjson module is available.

In detail:

you need to install the cjson module (apt-get, lua-rocks, … depending on your system.) See MP: Lua CJSON for more details.
json.decode() will use cjson.decode() if available. The cjson module is strict about syntax, so some plugin files, which are a bit lax about where they put commas, will not be successfully parsed. In this event, the decoding automatically reverts to the openLuup Lua implementation, which handles this situation.
two additional tables, Lua and C, are available in the openLuup.json module, giving direct access to just the Lua or C implementations, if required. Really, this is just a development hook. Regular plugins should continue to use the openLuup.json module directly, as before, with no changes required to the plugin code.

{
  ABOUT = ABOUT,      -- usual documentation / author / copyright / etc...

  decode = function: 0x1fa10f8,   -- default decode (using cjson if present)
  encode = function: 0x1fc4ff0,   -- default encode 

  Lua = {              -- direct access to native Lua openLuup implementation
    decode = function: 0x1fc5060,
    encode = function: 0x1fc4ff0
    null = {}
  },
 
  C = {                -- direct access to C implementation, if present
    _NAME = "cjson",
    _VERSION = "2.1.0",
    decode = function: 0x1fb2b70,
    decode_invalid_numbers = function: 0x1fc4ef0,
    decode_max_depth = function: 0x1fb0438,
    encode = function: 0x1fb3158,
    encode_invalid_numbers = function: 0x1fc1fb0,
    encode_keep_buffer = function: 0x1fa2780,
    encode_max_depth = function: 0x1fa7e30,
    encode_number_precision = function: 0x1fa95f0,
    encode_sparse_array = function: 0x1fb5350,
    new = function: 0x1fc4f50,
    null = userdata: (nil)
  },
}

By way of example, here is the impact on my RPi system CPU load running the ZWay plugin:

akbooer · April 30, 2020, 7:53pm

Development Branch: 2020 Release 4.30

hot fix by @propHAed for VeraBridge remote action failures
update Ace editor to 1.4.11