UI Notes
Here is some general information about getting information to display in a user interface:
UPnP messaging system
Vera uses UPnP for all the messaging, such as turning lights on and off. So Vera translates Z-Wave, Insteon, infrared codes, etc., into UPnP actions. If you have already implemented a UPnP stack and can send properly formated UPnP SOAP/XML requests, you can control everything using standard UPnP action invocation. If you did not, Vera provides a simple HTTP GET interface as well, which is discussed below.
In a nutshell UPnP works like this: You have UPnP devices, which are things that can be controlled and can report their state, like a light switch. And you have UPnP Control Points, which are the things that let a user control UPnP devices, like a touch-screen web pad. The Control Point does this by a) reading the device and descriptions for the UPnP device to see what it can do, and b) invoking actions on the UPnP device (ie telling it to do something), and c) reading and setting variables for the device which describe the current state and/or configuration settings for the device.
A UPnP device is defined in an XML document called a device description document. To see one, in Vera's setup UI, go to Devices, Luup plugins, Luup Files, and click 'view' file next to D_BinaryLight1.xml. Use Firefox since it has a built-in XML viewer. Notice the tag: <deviceType>urn:schemas-upnp-org:device:BinaryLight:1</deviceType>. This defines the type of device, namely an on/off light switch, which is what you use to know how to represent this device to a user. A device optionally implements one or more services. In this case the services: urn:upnp-org:serviceId:SwitchPower1, urn:micasaverde-com:serviceId:EnergyMetering1 and urn:micasaverde-com:serviceId:HaDevice1.
A service is a collection of actions which a device implement and which you can invoke from a UPnP Control Point, as well as a collection of variables which always have a current value. The variables describe the current state of the device and/or configuration settings. View the file S_SwitchPower1.xml to see a service. For example, the device urn:schemas-upnp-org:device:BinaryLight:1 implements the service urn:upnp-org:serviceId:SwitchPower1, which has the action SetTarget, which is how you turn the light on and off. The action takes a single argument newTargetValue which can be 0 to turn the light off and 1 to turn it on. The service also has the variable Status which is 0 or 1 depending on whether the light is currently on or off.
If you open the device description for a dimmable light D_DimmableLight1.xml you'll see that it implements the same service S_SwitchPower1.xml, since you can still turn it on and off, and it also implements the service urn:upnp-org:serviceId:Dimming1, which has an action to set the dim level (SetLoadLevelTarget).
To test UPnP calls you can use the Intel Device Spy utility available here. It should pick up Vera, show you all the devices, like light switches, thermostats, etc., let you expand the services and see the actions and variables. Click an action, like SetTarget, to specify the arguments and click 'Invoke' to run the action, such as turning on and off the light.
JSON vs. XML
The native UPnP protocol uses XML/SOAP. But there are also several times you will request data from Vera that has nothing to do with the UPnP spec. These will be done with data_request's as explained below, such as user_data, lu_status, etc. Internally Vera uses JSON as the native format, but Vera can convert to XML (details follow). In general when you want to view data in a human readable format, such as the device list, it's easiest to use the Firefox browser, and request Vera convert the data to XML. Firefox has a nice browser to display XML in human readable format. If you want to make the JSON data nicely formatted, you can use the web site jsonlint.com. In a UI application which will request data regularly from Vera we recommend retrieving everything in JSON format, not XML, because Vera's CPU is not that powerful and so continuous requests of data in XML format will cause a lot of CPU time to be devoted to converting JSON into XML.
Finding Vera on the local network
You can find Vera by using the traditional UPnP discovery process, or you can retrieve this URL:
http://findvera.com/locator.php
Each Vera, when it boots up, reports its internal IP address to the central findvera.com server, which tracks this along with the external IP address. locator.php shows all serial numbers and internal network IP addresses on the same external IP. For example, if http://findvera.com/locator.php returns the following:
7300 192.168.1.114 6907 192.168.1.102 7618 192.168.1.117
Then there are 3 Vera's on the same internal network (i.e. sharing the same external IP as your browser), with serial numbers 7300, 6907 and 7618, with the 3 IP internal (LAN) addresses shown.
You can also pass a FindVera username to locator.php and it will return the serial number and IP associated with it, such as:
http://findvera.com/locator.php?username=demovera
The HTTP interface
Vera listens on port 3480. Vera responds to UPnP actions the normal way, or by making simple HTTP requests. For example, to turn on a light you can either send the UPnP action "SetTarget" to the device using the normal UPnP mechanism, or you can simply put this URL in your web browser, or with wget, or similar:
http://[ipadress]:3480/data_request?id=lu_action&output_format=xml&DeviceNum=10&serviceId=urn:upnp-org:serviceId:SwitchPower1&action=SetTarget&newTargetValue=1
That URL will return the same results of the action as using the normal UPnP architecture. The output_format=xml on the URL means the data returned should be in XML format. You can instead use output_format=json
The service ID's, actions, and arguments are all defined by the UPnP forum. A list of ratified UPnP devices and services is available on the UPnP Forum. Mi Casa Verde added its own custom devices and services when there was no existing UPnP standard. A list of these can be found in the Luup_UPNP_Files header files.
Getting the system configuration
Either use the UPnP service/action:
urn:micasaverde-com:serviceId:HomeAutomationGateway1/GetUserData
or use the built-in request:
http://ipaddress:3480/data_request?id=user_data2&output_format=json (or xml)
This returns the system configuration file for Vera. It's assumed you understand the way Vera categorizes devices with Rooms, Devices, Scenes, etc. When creating a user interface for controlling Vera only, like on a mobile phone, most of the tags can be ignored, since usually you just need the list of rooms and devices.
Accessing Vera remotely through the findvera server
Native UPnP only works on a local area home network, and cannot be used over the Internet. The FindVera server provides a secure way to remotely access and control your Vera system using the HTTP interface without having to make any substantial changes to the UI. Everything you can do locally with Vera on port 3480, you can do remotely with findvera using the exact same syntax. You only need to ask the user for his username and password and pass them on the URL to the remote access server. For example, the above user_data is the syntax for local access on a home network. To retrieve the same URL over the internet with findvera use:
https://ra1.findvera.com/demovera/myvera123/3480/data_request?id=lu_status&output_format=json
assuming demovera is the username and myvera123 is the password that the user chose on the 'findvera' tab in his Vera setup page. Note that since the request is https, the username and password are encrypted because https encrypts the URL's as well as the contents of the page.
Building a control-only UI for Vera
If you just want to control Vera, and not modify the configuration, such as with a cell phone interface or web pad, the flow might possibly work like this:
1. For the initial setup, you need to ask whether the user wants to (a) access Vera remotely with the FindVera service, and if so, get the user's username/password; or (b) access Vera directly on the home network. You can use locator.php to show the Vera on the home network. You might want to do an auto-detect. For example, you could use locator.php to see if Vera exists locally, and if not, prompt the user for a FindVera username and password. You can use locator.php and pass the username to get the serial number and internal IP, and then automatically switch to direct access if locator.php (without the username) indicates that serial number is found on the local network. This would allow the user to automatically switch between local access with Wi-Fi (which is much faster of course) and remote access with FindVera and the mobile phone network.
2. Once the connection is established, retrieve the list of rooms, scenes, and devices from Vera (see user_data below). You can cache this data locally so you don't need to retrieve it and parse it over and over.
3. Present the user with the rooms, scenes, devices. In Vera's UI's we show first the rooms and then let the user pick a device or scene in the room. However this data could be presented graphically, such as on a floorplan. Your UI can have Vera store extra tags and data in the configuration database (ie user_data) to store things like coordinates for icons, outlines or rooms and so on.
4. If the user picks a scene you can run the scene with the following URL, passing the scene number as x, and it will return an OK if it went okay:
5. If the user picks a device, you use the category which is contained in the user_data to determine what type of controls to present the user. (See Luup UPNP Files for a list of the categories.) When the user chooses an action, you can send it like this:
For a list of common actions and device types see: Luup CP Mandatory Types
Some actions return immediately with an 'OK'. Other jobs may take time to complete and are processed asynchronously. For these asynchronous jobs you will get a job ID back. There are several ways to present this to the user. One possibility is to do what Vera does in the UI, namely to use lu_status (explained next) to find all active jobs and to put a 'job' icon next to each device to indicate the progress of the job. In this way the user isn't tracking a particular job per se, but just has visual feedback that a device is busy, and if the job icon(s) turn green, that everything went through okay. You could instead track the action individually using the job number that is returned, or use some other method, like making the icon for the device appear animated while it's busy, and turn green or red if the last command succeeded or failed.
6. Periodically poll the system with lu_status (explained below), like this:
http://ipaddress:3480/data_request?id=lu_status2&output_format=json
That returns the current values of the UPnP variables, which you can use to show the user current status. For example, if the variable Status is 1 for a switch, you can show the 'on' button in a selected state, while 0 shows it in an 'off' state. lu_status also shows any active jobs for a device.
user_data and lu_status
NOTE: On UI3 builds, user_data2 and user_data are the same, and so is lu_status2 and lu_status. However, on UI2 builds, user_data and lu_status have been deprecated, and you should user_data2 and lu_status2. Therefore, it's best to simply always use user_data2 instead of user_data, and lu_status2 instead of lu_status.
The user_data is the main configuration file that Vera uses to store all the rooms, devices, scenes, timers, events, users, and any other data about the system. You should request the user_data to get whatever data you need to present to the user, at a minimum, for example, you'll need the list of devices, their descriptions, and the categories (light switch, thermostat, etc.). You'll also want the list of scenes to present the user. To fetch the current user_data from Vera use the user_data2 request, like this:
http://[ip address]:3480/data_request?id=user_data2
That returns the data in JSON format, the native format Vera uses. To format it nicely for reading copy and paste it into jsonlint.com. You can also use the Firefox browser and append &output_format=xml to have Vera convert it to XML format so it shows up nicely in Firefox's xml viewer: http://[ip address]:3480/data_request?id=user_data2&output_format=xml
Although XML is easier to view with Firefox's human-readable viewer, for production use, use the JSON format because otherwise Vera has to convert JSON to XML for each request.
Notice there is a tag, like this: DataVersion="315587159" This is a timestamp of the last time the user_data was modified. If you pass the DataVersion which you have on the URL, Vera will return a page with the text NO_CHANGES if that is still the current version of user_data, like:
http://[ip address]:3480/data_request?id=user_data2&DataVersion=315587159
The user_data contains the state for all the devices. For example this indicates that the device id #17, called "Switch" is turned off (ie service SwitchPower1, variable Status is 0)
{
"devices": [
{
"id": 17,
"category": 3,
"room": 1,
"name": "Switch",
"ip": "",
"mac": "",
{
"service": "urn:upnp-org:serviceId:SwitchPower1",
"variable": "Status",
"value": "0"
}
}
]
}
These are the UPnP variables, which describe the state of the device according to the UPnP specs. When the variables change that information needs to be shown to the user, such as changing the icon for the Switch from off to on. If you are interfacing with Vera using the UPnP stack, you can subscribe to the device's events to receive notifications automatically when a device's state changes. However if you're using the web interface you will need to poll. You don't want to request and parse the full user_data each time you just want to see if devices states have changed. So there is also a lu_status request you can use like this:
http://[ip address]:3480/data_request?id=lu_status2
Again add &output_format=xml to see it in XML. This document contains only 3 pieces of information, which constantly change: 1) The state, or UPnP variables, for the device to indicate the device's current state. 2) The status of any active or recent jobs, such as if a light was successfully turned off. 3) The state of the system initialization, such as initialization of the plugin modules.
For #1, the UPnP variables, are simply a subset of the data in user_data. #2, the jobs, and #3, the system initialization, are generated live and are not stored in user_data. Thus lu_status is the appropriate request to poll Vera to get the up to date information about what's going on so it can be presented to the user.
The first tag is DataVersion. This is also essentially a timestamp for the device variables. If you call lu_status without passing a DataVersion on the URL, you will get the current value of all the UPnP variables for a device. Note that most of the UPnP variables do not ever change. For example, with a light switch, only the variable "SwitchPower:Status" changes, as the switch is turned off and on. There are many other variables, like "Capabilities" which are simply configuration data or settings the user modified. If you pass &DataVersion=1 on the URL, you will only see the UPnP variables which have changed since the Luup engine started. So, if you parse the initial UPnP variables when you first call user_data, you can call lu_status with DataVersion=1. Once you call lu_status the first time, store the DataVersion and on subsequent calls pass it on the URL and you will only get the UPnP variables which have changed since that last DataVersion. Regardless, you will also get the current state of jobs and system initialization.
On subsequent calls to lu_status2, you should also pass back in the LoadTime which you received on the prior call. You can also pass in a Timeout value, which means the call will block for up to that many seconds if nothing has changed, and will return immediately when something does. This way you can have infrequent polling, say every 60 seconds by putting a Timeout value of 60, but your UI will be immediately responsive since, when something changes, it returns immediately. You can also add a MinimumDelay, which means that the call will wait for at least that many millseconds. This is advisable so that in the event there are lots of continuous changes, such as a scene turning off many lights, you won't have lu_status2 calls returning non-stop consuming all the resources on both the Vera and the UI.
The jobs for a device will look like this:
"devices": [
{
"id": 1,
"Jobs": [
{
"id": "1",
"name": "job#1 :getnodes (0x00D068F0) P:10 S:4",
"icon": "",
"comments": "",
"status": "4"
}
]
},
The id is the job id number. When you run an action, like turning on a light, if it happens asynchronously, you'll get a job id number. All pending jobs for a device are shown by their id number. The name is just an internal name for the job. Icon is an optional keyword to indicate what sort of icon should be shown to the user. For example, 'getnodes' has no icon, meaning this type of job is generally not of interest to a user. If the icon was 'ON' or 'OFF' that would mean this job is busy turning on or off a light. Comments is notes about the job, if any. This may be shown to a user in a tool tip or similar, but is generally more technical than users would care to know. The status indicates the current state of the job, as shown:
job_None=-1, // no icon job_WaitingToStart=0, // gray icon job_InProgress=1, // blue icon job_Error=2, // red icon job_Aborted=3, // red icon job_Done=4, // green icon job_WaitingForCallback=5 // blue icon - Special case used in certain derived classes
If the status is 0, this means the job is queued but hasn't started. If it's 1 or 5 the job is being executed right now. Generally both 1 and 5 are presented to the user the same way as a 'busy', although technically 1 means Vera is actively talking to the device, and 5 means Vera is waiting for a reply from the device. Status codes 2, 3 and 4 mean the job has finished. 4 means the job went ok, and both 2 and 3 mean it failed. Technically 3 means the job was aborted by another job or an external process, like a user cancelling it, while 2 means it failed due to a problem talking to the device. In both cases this is usually presented the user the same way, as a failure.
The section with the initialization tasks is shown in the startup tag as shown:
"startup": {
"tasks": [
{
"id": 0,
"status": 2,
"type": "ZWave",
"comments": "No Z-Wave"
},
The startup tasks begin whenever the Luup engine is reset, such as after the user makes a configuration change. Ever plugin and module in the Luup engine does a startup sequence. The id is not really important. The status numbers are the same as for jobs, so 2 means the module failed to startup. The 'Type' is what you can use to key off of to know which module is being reported, such as "ZWave", and comments are just notes indicating the current state. For example, "No Z-Wave" means there's no Z-Wave dongle connected.
Once all the plugins have been initialized successfully the startup section will disappear completely from lu_status.
Note that lu_status2 also returns UserData_DataVersion, which is the dataversion from user_data. That way you don't need to poll user_data in parallel. Simply retrieve the user_data one time at startup, then poll lu_status, and, if UserData_DataVersion is different from the previous user_data you retrieved, then download user_data again.
Some notes on building a full replacement, including configuration, are available here UI_Notes_Replacement, however the syntax is not up to date with the Luup architecture.
Categories and device types
The following values are used in user_data to determine the types of devices. In general you can use the category instead of the devicetype since it's an integer and is thus faster to compare:
// Device Types
#define DEVICETYPE_HOME_AUTO_GATEWAY "urn:schemas-micasaverde-com:device:HomeAutomationGateway:1" #define DEVICETYPE_BINARY_LIGHT "urn:schemas-upnp-org:device:BinaryLight:1" #define DEVICETYPE_DIMMABLE_LIGHT "urn:schemas-upnp-org:device:DimmableLight:1" #define DEVICETYPE_THERMOSTAT "urn:schemas-upnp-org:device:HVAC_ZoneThermostat:1" #define DEVICETYPE_HUMIDITY_SENSOR "urn:schemas-micasaverde-com:device:HumiditySensor:1" #define DEVICETYPE_MULTI_IO "urn:schemas-micasaverde-com:device:MultiIO:1" #define DEVICETYPE_DOOR_LOCK "urn:schemas-micasaverde-com:device:DoorLock:1" #define DEVICETYPE_DOOR_SENSOR "urn:schemas-micasaverde-com:device:DoorSensor:1" #define DEVICETYPE_ZWAVE_NETWORK "urn:schemas-micasaverde-com:device:ZWaveNetwork:1" #define DEVICETYPE_INSTEON_NETWORK "urn:schemas-micasaverde-com:device:InsteonNetwork:1" #define DEVICETYPE_USB_UIRT "urn:schemas-micasaverde-com:device:USBUIRT:1" #define DEVICETYPE_TEMPERATURE_SENSOR "urn:schemas-micasaverde-com:device:TemperatureSensor:1" #define DEVICETYPE_POWER_METER "urn:schemas-micasaverde-com:device:PowerMeter:1" #define DEVICETYPE_MOTION_SENSOR "urn:schemas-micasaverde-com:device:MotionSensor:1" #define DEVICETYPE_LIGHT_SENSOR "urn:schemas-micasaverde-com:device:LightSensor:1" #define DEVICETYPE_IR_TRANSMITTER "urn:schemas-micasaverde-com:device:IrTransmitter:1" #define DEVICETYPE_WINDOW_COVERING "urn:schemas-micasaverde-com:device:WindowCovering:1" #define DEVICETYPE_GENERIC_IO "urn:schemas-micasaverde-com:device:GenericIO:1" #define DEVICETYPE_REMOTE_CONTROL "urn:schemas-micasaverde-com:device:RemoteControl:1" #define DEVICETYPE_COMBO_DEVICE "urn:schemas-micasaverde-com:device:ComboDevice:1" #define DEVICETYPE_CAMERA "urn:schemas-upnp-org:device:DigitalSecurityCamera:1" #define DEVICETYPE_SERIALPORT "urn:micasaverde-org:device:SerialPort:1" #define DEVICETYPE_SCENE_CONTROLLER "urn:schemas-micasaverde-com:device:SceneController:1" #define DEVICETYPE_ENERGY_CALCULATOR "urn:schemas-micasaverde-com:device:EnergyCalculator:1"
// IR Device Types
#define DEVICETYPE_TV "urn:schemas-micasaverde-com:device:tv:1" #define DEVICETYPE_CABLE "urn:schemas-micasaverde-com:device:cable:1" #define DEVICETYPE_SATELLITE "urn:schemas-micasaverde-com:device:satellite:1" #define DEVICETYPE_VIDEO_ACCESSORY "urn:schemas-micasaverde-com:device:videoaccessory:1" #define DEVICETYPE_VCR_DVR "urn:schemas-micasaverde-com:device:vcrdvd:1" #define DEVICETYPE_DVD_BLURAY "urn:schemas-micasaverde-com:device:dvdbluray:1" #define DEVICETYPE_RECEIVER "urn:schemas-micasaverde-com:device:receiver:1" #define DEVICETYPE_AMP "urn:schemas-micasaverde-com:device:amp:1" #define DEVICETYPE_CD "urn:schemas-micasaverde-com:device:cd:1" #define DEVICETYPE_MISC_HOME_CONTROL "urn:schemas-micasaverde-com:device:mischomecontrol:1" #define DEVICETYPE_AV_MISC "urn:schemas-micasaverde-com:device:avmisc:1" #define DEVICETYPE_VIRTUAL_DEVICE "urn:schemas-micasaverde-com:device:VirtualDevice:1"
// Device Categories
#define DEVICE_CATEGORY_INTERFACE 1 #define DEVICE_CATEGORY_DIMMABLE_LIGHT 2 #define DEVICE_CATEGORY_SWITCH 3 #define DEVICE_CATEGORY_SECURITY_SENSOR 4 #define DEVICE_CATEGORY_HVAC 5 #define DEVICE_CATEGORY_CAMERA 6 #define DEVICE_CATEGORY_DOOR_LOCK 7 #define DEVICE_CATEGORY_WINDOW_COV 8 #define DEVICE_CATEGORY_REMOTE_CONTROL 9 #define DEVICE_CATEGORY_IR_TX 10 #define DEVICE_CATEGORY_GENERIC_IO 11 #define DEVICE_CATEGORY_GENERIC_SENSOR 12 #define DEVICE_CATEGORY_SERIAL_PORT 13 #define DEVICE_CATEGORY_SCENE_CONTROLLER 14 #define DEVICE_CATEGORY_AV 15 #define DEVICE_CATEGORY_HUMIDITY 16 #define DEVICE_CATEGORY_TEMPERATURE 17 #define DEVICE_CATEGORY_LIGHT 18 #define DEVICE_CATEGORY_ZWAVE_INT 19 #define DEVICE_CATEGORY_INSTEON_INT 20 #define DEVICE_CATEGORY_POWER_METER 21
pseudo-code for a UI
Here is some pseudo code to show how the polling mechanism should be properly implemented. The parsing code is, of course, unique to your UI, so placeholder function names are shown here, like FetchAndParse. This shows the flow for handling the polling so you know you get the changes reported correctly.
variable int DataVersion_lustatus=0 variable int LoadTime=0 variable int DataVersion_userdata=0
- START
variable boolean FetchUserData=false;
if( DataVersion_userdata==0 ) // If we're on the first loop and haven't retrieved user_data, do it and don't do an lu_status
FetchUserData=true;
else
{
while(true)
{
newdata = FetchAndParse("lu_status2&DataVersion="+DataVersion_lustatus+"&LoadTime="+LoadTime);
if( newdata.IsEmpty()==false && newdata.isError()==false )
break; // If we got data, continue on and process it
else
Sleep 1; // If we didn't get anything, or if lu_status failed, just sleep 1 second and stay in the loop to try again
} // End of while loop
if( newdata.UserData_DataVersion!=DataVersion_userdata )
{
FetchUserData=true;
}
else // Only Parse if userdata is the same, otherwise we're going to reload user_data
{
ParseLuStatus(); // This won't run if user_data has changed
DataVersion_lustatus=newdata.DataVersion
}
} // end of else loop
if( FetchUserData==true )
{
UI.HourglassAndSuspend(true); // Put a 'busy' hourglass in the UI, display a message in the info panel "Updating data" and suspend all UI stuff since this may take a moment
newdata = FetchAndParse("user_data2");
if( newdata.IsEmpty()==false && newdata.isError()==false )
{
ParseUserData();
DataVersion_lustatus=newdata.DataVersion
LoadTime=newdata.LoadTime
DataVersion_userdata=newdata.DataVersion
}
else
UI.DisplayError("Failed to update data"); // Display in the info panel an error message that the user will see for 10 seconds or so
UI.HourglassAndSuspend(false); // Put a 'busy' hourglass in the UI, display a message in the info panel "Updating data" and suspend all UI stuff since this may take a moment
}
Sleep 1; // We know we just parsed user_data or lu_status, so sleep before looping goto START; // Loop again and start over
