LuaUPnP

Revision as of 22:26, 10 August 2009

This page explains how we are implementing our upcoming LuaUpNP feature in Vera. The document gets a bit technical so that users can comment on it and we can make adjustments as the feature comes closer to release. User feedback is appreciated and a sticky topic has been created in the 'Vera Developers' section of forum.micasaverde.com for comments.

This new feature is very significant for Vera because it expands the possibilities of Vera greatly, and makes it very simple to create really powerful automation systems. This one module provides infrared (IR) control for audio-visual (AV) devices, serial/USB/Ethernet control for any type of device, and a Lua scripting engine that can be used to do just about anything. And it's all based on UPnP, so anything you can create can be controlled from a standard UPnP control point.

Some background information: UPnP allows you to describe any type of device in a generic way, explaining what the device can do and how to control it. Then, there are a variety of UPnP control points, which are remote controls, such as a wall-mounted touch-panel, which are able to scan the network for UPnP devices, find out what those devices can do, and present you with a remote control.

Lua is a fast, lightweight scripting language for creating macros, code snippets, and even stand alone applications.

Our LuaUPnP solution consists of 3 parts: 1) The device builder web site; 2) the LuaUPnP engine; 3) any number of input-output (I/O) devices (infrared transmitters/receivers, RS232 ports, etc.).

Here is a general work flow showing how the user creates and uses devices:

Adding a basic infrared AV device or RS232 that already exists in the database

1. The user adds an infrared transmitter/receiver, such as plugging a Global Cache GC-100 into the LAN, or a USBUIRT into the extra USB port on Vera. The device is recognized and appears automatically on Vera's devices list. For communicating with a serial/RS232 device, the user can also connect a generic USB->RS232 (usb->serial) cable in Vera's spare USB port, or connect the device to one of the serial ports on a GC-100.

2. The user goes to the 'Devices' tab in Vera's 'Setup' user interface (UI), chooses the 'Add LuaUPnP device', and is presented with a web form to help locate the device. After picking the manufacturer, type of device, model #, etc., Vera asks to assign a room and choose the I/O device associated with it (e.g. the USBUIRT or the port on the GC-100 or the usb->serial device).

The device is now broadcast as a UPnP device and can be controlled from any UPnP control point. In Vera's 'Scenes' tab, an 'Add actions' button appears next to the device, allowing a choice of action(s) to send the device as part of the scene (i.e. 'TV on', 'Mute', etc.).

Adding a new, basic infrared AV device or RS232 that does not exist yet

Step 1 above is the same, and step 2 starts out the same. But, when he gets to pick the device, he chooses 'device not on the list'. Then:

1. The user fills in a form with basic information about the device, such as the manufacturer, model, type, etc. The user also picks how to send commands to the device: Infrared codes, RS232 data, network calls, etc., or he picks 'None' if this is purely a logic device that doesn't directly control any other device, such as a "Cold weather logic" device that may wait for sunset, and then check weather.com to see if the temperature is expected to drop below freezing, and if so, turn on the thermostat. This is a "logic" device because it's not directly controlling any device (the Z-Wave module is controlling the thermostat), it's merely providing logic. The user can also indicate if this device will be 'public' for other users to use too, or 'private', just for his Vera. This data is stored in an xml file that is a standard UPnP device description.

2. The user picks one or more of the standard service definitions (ie "TV", "CD Player", etc.). These have a list of actions, like "On", "Off", "Mute", etc. If there is no standard service definition, he has the option of creating a new one, in which case he adds all the actions his device will support "On", "Off", "Dim", etc., and for each action, adds any number of optional arguments. For example, "On" and "Off" may not have any arguments, but "Dim" takes a "Level" argument. This data is stored in an xml file that is a standard UPnP service description.

3. Now the user has a form that lists each action, and next to each action is a pull-down where the user can choose what to do in response to that action. The choices are: a) Send data to the device. There's an input box for the user to type in the data. The format of the data corresponds to the communication method picked above. If the communication is 'infrared', the user types an i/r code in the box. If it's rs232, he types in the data to send to the device. b) Run some Lua code. These can be mixed and matched. For example, this may be an infrared TV and basic commands, like "channel up" just require sending a code. But other commands, like "input #4" may require some logic to toggle inputs, so he wants to write Lua script. If the device is infrared, there's also a button "Learn the i/r code", which learns the code from an infrared learner.

4. The user also has the option of adding, for each action, Lua code to be run when: a) incoming data is received and the command is being executed, b) the command is completed, c) the command is aborted, d) the command times out. There are a couple other options for a command, such as specifying if the command should be aborted when another one of the same or other commands comes in before it's finished (ie if the user wants to turn the device off, but an 'on' command comes in before the device has turned 'off', abort the 'off').

After this is done, the user can add the device like described previously.

Writing the Lua Code

Whenever a command is received, it's assigned an internal job #. This is true whether or not there's any Lua script to be run. So, calling an action results in a Job, and the Job will either send the data (the i/r code, serial data, etc), or call the Lua script for that action if you wrote any.

For each action, there are 3 functions, or blocks of Lua script, you can create. 1) The _Run script, which is called when the action is received and needs to run. 2) The _Callback script, when the job is waiting for some data to come back. 3) The _Timeout script, when data was not received in time.

If the user wrote some _Run Lua script in step 3 above, it's called and past as arguments the the job number plus any parameters for the action. All Lua script for a device run in the same instance, so they can all share global variables.

The Lua script does whatever is needed in response to the command. It can call Lua functions as well as functions which LuaUPnP exports to the Lua engine. The Lua script can also set a global variable called "Status" to some free form text, like "sending command", or "device not responding".

When the Lua script exits, it returns 2 parameters: 1) the job status, which is: job_WaitingToStart, job_Error, job_Aborted, job_Done, or job_WaitingForCallback. 2) the second parameter is a delay, in seconds, for the job to timeout.

If the job status is job_Done, that is the normal situation when everything is ok. The timeout is not used.

If it's job_Abort or job_Error, the job is also marked as 'completed', but it ended in an error condition, or because the user aborted it. The timeout is not used.

If it's job_WaitingToStart, then the same Lua script will be called again after [timeout] seconds have passed. This is a way to say "I'm not ready to do this now, let me try again later".

If it's job_WaitingForCallback, then this means the script is waiting for some incoming data. For example, maybe the protocol requires sending something over the serial port and waiting for an "OK" to come back. If the job does not complete within [timeout] seconds, the _Timeout Lua script is called, which, if not defined, simply sets the status to "Timed out" and returns job_Error.

When data comes in through the i/o device (ie serial port, etc.), that data is first sent to the _Callback script of the job in the 'job_WaitingForCallback' state. The 'callback' code returns a) whether it processed the incoming data, b) the job status, and c) the timeout. If it returns 'false' to the first parameter, the framework will keep looking for other pending jobs that are in the 'job_WaitingForCallback' state, and if all of them return false, it goes to a generic 'Incoming data' script. If the job status returned is job_WaitingForCallback, then the job is still waiting and will be called again when data comes in.

The web engine takes care of creating the function names to shelter non-programmers from as much coding as possible. However, internally, all the script is stored in Lua functions. So, for example, if there's an On action, and the user adds _Run, _Timeout and _Callback script, there will be 3 Lua functions the framework creates: On_Run, On_Timeout and On_Callback. The user can also create with the web user interface a block of Lua script to run on startup (Startup), shutdown (Shutdown), as well as to run when data is received that wasn't handled by a job (IncomingData), and also an idle loop when the device is idle (IdleLoop).

So here is some pseudo code to explain the concept. Let's say that you want to add control for a TV with an RS232 interface. To turn the TV on you send the string: <ON> and when it's processed, it responds with <OK> if was processed, or <ERR> of it couldn't be processed. The TV may also send unsolicited data on the serial port when the status changes, such as <ON> when the TV is turned on.

You add the device, add the actions, "On", "Off", etc. Now when adding the implementation, you could take the really simple approach and next to the On action, choose "Send" from the pull down and just put <ON> in the input box. This would work. But, perhaps you want to write Lua script to be able to process the return values <OK> or <ERR> to know if the command went through, and maybe you want to be able to process incoming events, like <ON> so update the state of the UPnP device so if the user turns it on by hand, the status of the UPnP device changes to On. So, in this case you'd create Lua code for the _Run, _Callback, and _Timeout as follows:

_Run:

 Send("<ON>")
 Status="Waiting for device to respond"
 return job_WaitingForCallback,3

_Callback:

 if( IncomingData=="<OK>" )
   Status="Everything is ok"
   return true, job_Done
 elseif( IncomingData=="<ERR>" )
   Status="Error sending command"
   return true, job_Error
 else
   return false, job_WaitingForCallback

_Timeout:

 Status="Device not responding"
 return true, job_Error

And the generic IncomingData script looks like this:

 if( IncomingData.startswith("<VOLUME" )
   SetUPnPState("Volume",IncomingData.mid(6)
   return true

So, the user hits 'ON' on his UPnP control device. The _Run function is run for the 'ON' action, which sends <ON>, sets the status, and indicates we're waiting 3 seconds for the device to respond. Let's say that the user hit 'volume up' before that happens, so _Callback for the 'ON' action is called and Incoming data equals <VOLUME 50>. _Callback has no use for this, so it returns false, and indicates the job status is still 'waiting for callback'. If it wanted to increase the timeout another 3 seconds it could do: return false, job_WaitingForCallback, 3. But without the ,3, the timeout remains the same. Since it returned false, the framework will pass the volume string to the master IncomingData function which parses it and updates the UPNP state. Now, if, within 3 seconds the <OK> strings comes in, _Callback is called again and will set the job to 'Done', meaning the job won't get called again. If the 3 seconds passed without the job being 'Done', the _Timeout function is called, which in this case sets the job to an error state.

On Vera's device page you specify for the device the i/o device (serial port, infrared, etc.), and for devices like the serial port, the parameters like baud rate, etc. So in the Lua code you just call Send() with whatever you want to send. Also, in the device specification, you can specify delimiters for incoming data, so the framework knows what constitutes a 'chunk' of incoming data. In this protocol, all commands end in a line feed (\n), so <OK>\n<ON> is treated as 2 blocks of incoming data and handled accordingly. For more complex protocols you can provide a Lua script that is called with each incoming character that returns a value to indicate if it's a complete chunk, a bad chunk of data, or to just keep going.

Message Interceptors

This is what allows your Lua script to do something in response to other things that are happening and not just react to commands. You can have your Lua script get called when other devices receive an action, fire an event, and so on. We're using a flexible template-driven model so you can have a script called, for example, whenever any light switch changes state, or when a particular device is turned on, or when the temperature is above a certain threshold, or any time the temperature changes on a device, and so on.

Collaboration

When you create new devices online you can specify if they should be shared with the community, or are private to your device. If you choose not to use our online tools you can upload to Vera directly the 3 needed XML file: 1) device description, 2) service description, 3) implementation.

Compound devices

We are creating a visual web page within Vera to merge actions from different devices into one device. For example, let's say you have a PVR, a TV and a Receiver. All 3 will be exposed as UPnP devices. So with the UPnP control point you can pick the PVR as the device you want to control, and you'll see the actions the PVR supports. However, even though you're controlling the PVR, you'll probably also want to be able to control the volume, on, off, and input select on the receiver, as well as the on, off and maybe brightness on the TV. So in Vera's devices page we're adding a "Compound" button that shows you the actions for the PVR, let's you uncheck any actions you don't want to see on you control point, and let's you add other commands from other devices, like the on/off from the TV, so that when the control point shows the actions of the PVR, it will include the on/off for the tv, and volume for receiver, etc.