Data Provider Catalog Plugin

Revision as of 22:24, 23 December 2008

This has a general overview of the request architecture. For a list of requests see Data_Provider_Catalog_Plugin_Requests

The Data_Provider_Catalog_Plugin runs as a plugin for the DCERouter. It acts as an aggregator for data sources. Other devices in the network can register with it by sending the command CMD_Register_Data_Provider and register to provide data, which is identified by a unique ID. This is intended to replace the DataGrid concept previously in LinuxMCE.

So, for example, the media plugin can register to provide a set of data with the ID "SongsByArtist", and a security module can provide a set of data with the ID "ListOfSecurityBreaches". These devices register with Data_Provider_Catalog_Plugin, which then becomes a single point for requesting data.

Request data from Data_Provider_Catalog_Plugin by sending it the message CMD_Request_Data_Provider (ID 971), which includes parameters where you can indicate what format you want the data in (ie XML, plain text, etc.), what input parameters are provided to the data source (ie "SongsByArtist" may require an input parameter of "Artist ID").

The Data_Provider_Catalog_Plugin is in the category 174 (DEVICECATEGORY_Data_Provider_Catalog_Plugins_CONST). One device that registers to provide general purpose data sources is Shared_JSON_Data_Provider_Plug, and one of the data ID's it registers is "alldata", which returns all the common user data in the system (ie list of devices, scenes, etc.). It provides it in 'json' format.

So to get this data you can use the MessageSend utility as follows: /usr/bin/MessageSend dcerouter -o -targetType category 0 174 1 971 10 alldata

The Data_Provider_Catalog_Plugin also includes a small http service which can be used to request data as well as to send messages which listens on port 3451. Previously when outside applications, like a php page, wanted to send messages they would do a system call to spawn the /usr/bin/MessageSend utility. This is inefficient because it makes a system call each time.

To request data, request "data_request" from the http server and include the parameters:

id=id of the data request
parameters=parameters to send on the data request
input_format=format of the parameters being sent
output_format= the desired output format of the data

for example the following URL returns the same list of 'alldata' described earlier:

http://192.168.80.1:3451/data_request?id=alldata&output_format=json

You can also use this to send messages by requesting "messagesend" and adding the parameters:

from=the originating device ID
to=the destination device id
type=the type of message
id=the message id
response=[r|o|c] for Return String, Output Parameters, and Confirmation of delivery. Default=don't wait for response
target=[c|t] meaning the 'to' id is not a device ID but rather a Category or Template of device. Default=to is a device id
[device data]=some other message parameter where device data is the number

For example, to request a video frame from a camera with the device id #20 you send the message type 1 (command) id 84 (get video frame) and optionally the parameter 60 and 61 (width and height of desired image). The command returns output parameters 19 and 20 where 19 contains the video image and 20 the format. So we want to send message with response=o because we want the request to block until the message is process by device 20 and we want to get back the output parameters. Here is the URL:

http://192.168.80.1:3451/messagesend?from=1&to=20&response=o&type=1&id=84&60=640&61=480

The response will be "OK" if the message succeeds, or "FAIL" if it didn't, followed by the values of the output parameters. Because output parameter #20 is defined as 'binary', the result is uuencoded and the U after the 20 indicates this. The above returns:

OK

[19=3]

jpg

[20U=18373]

[uu encoded video frame]

So parameter 19 is 3 bytes long ("jpg"), and 20 is UU encoded with 18373 characters.

Other plugins can register to provide certain types of data. For example, for each ip camera it manages Generic_IP_Camera_Manager registers the id "camera_control/device #". The parameters it recognizes are: get_video_frame, left, right, up, down, zoom_in, zoom_out

So the following makes the camera with device id #20 move right:

http://192.168.80.1:3451/data_request?id=camera_control/20&parameters=right

and the following returns a single video frame from the camera

http://192.168.80.1:3451/data_request?id=camera_control/20&parameters=get_video_frame

If you turn on the findvera.com remote access, you can also securely access your box's port 3451 remotely with to control it, fetch devices, etc. with this syntax:

https://[ra server]/[username]/[password]/3451/

Different installations use different remote access servers. The ra server for a given installation is defined in your system file: /etc/cmh/RA_Server. If you have an application that prompts the user for his username/password, such as a cell phone app, call this to get the ra server for a given username: https://findvera.com/get_server.php?username=[username]

So with a username of testvera9 and password of myvera123, the following: https://ra1.findvera.com/testvera9/myvera123/3451/data_request?id=simple_device_list&output_format=txt is the list of devices

you can remotely access port 80 like this: https://ra1.findvera.com/host-testvera2/cgi-bin/cmh/get_json.sh?file=template_data.json

Understanding the messages

The above assumes you understand the basic concept of LinuxMCE messages. The LinuxMCE site contains lots of documentation on messages, and there are user-friendly front-ends for navigating commands, command parameters, etc. The Vera system only uses a small subset of the full LinuxMCE commands. If you're not already familiar with LinuxMCE messaging, here's a quick primer.

Devices have a given "DeviceTemplate", which is the definition for that type of device. For example, "John's floor lamp" may be device #18, and have the device template #38, "Dimmable light". DeviceTemplates will implement certain commands, like a dimmable light implements command #192 "On". Commands take parameters. For example, the "Dim" command may take a parameter which is the value. DeviceTemplates also fire certain events. For example, a "motion sensor" may fire "Sensor Tripped". Events also have parameters, such as for "Sensor Tripped", there's a flag "Tripped" which can be 1 or 0 depending on if the event is that the sensor is detecting motion, or stopped sensing motion. DeviceTemplates also reference Data, or configuration parameters. For example, a "Dimmable Light" has a configuration parameter #12 "Port/Channel", which, for a Z-Wave device is the node id.

In LinuxMCE there are web pages to visually present all this data, and the data is stored in a mysql database. As an embedded system, Vera doesn't use a database, rather it stores all the data in .json files.

Note that you can also retrieve the raw json files directly using a cgi-bin: http://[ip address]/cgi-bin/cmh/get_json.sh?file=xx.json where xx is:
devices = the list of devices and all the device parameters
user_data = user specified data, like scenes, timers and events
template_data = static data defining the structure of commands, events, device templates, etc
timezone = the known time zones

so calling http://[ip address]/cgi-bin/cmh/get_json.sh?file=devices.json will return the raw json data file containing the list of devices, and this will be similar to the data returned by the call http://[ip address]:3451/data_request?id=alldata&output_format=json, except the former is returning the raw data from the file system, and the latter is using Data Provider Catalog Plugin's port to return the data.

The data file template_data has the master definition for Device Templates, commands, events, and data configuration.

Note that the devices have a hierarchy defined by the FK_Device_ControlledVia. Device 1 is the access point itself, the home. The child DCERouter, in this case device 2, is the message router. The user never needs to see the DCERouter or any of it's children presented on the device list because these are used internally.

Some devices are what we call "Interface devices", meaning they represent the interface that handles child devices. For example, device #9 is 'Z-Wave' and device #10 is 'Generic IP Camera Manager'. What the user cares about are the children of device #9 (the z-wave devices) and children of device #10 (the cameras). But the interface devices themselves likely wouldn't appear on the device tree you show the users. Note this on the device page.

Everything you do in the system is based on sending messages to devices. A message has a type. The types are:

     enum eMessageType 
     { MESSAGETYPE_COMMAND=1, MESSAGETYPE_EVENT=2, MESSAGETYPE_DATAPARM_CHANGE=3, MESSAGETYPE_REPLY=4,
       MESSAGETYPE_DATAPARM_REQUEST=5, MESSAGETYPE_LOG=6, MESSAGETYPE_SYSCOMMAND=7,
       MESSAGETYPE_REGISTER_INTERCEPTOR=8, MESSAGETYPE_MESSAGE_INTERCEPTED=9, MESSAGETYPE_EXEC_COMMAND_GROUP=10,
       MESSAGETYPE_START_PING=11, MESSAGETYPE_STOP_PING=12, MESSAGETYPE_PURGE_INTERCEPTORS=13,
       MESSAGETYPE_PENDING_TASKS=14 };

you generally just use message type #1, command. Messages have an ID, which is based on the message type. So, for a command (message type 1), id 192 is "Generic On".

You can get more info on the messages by looking in the template_data: http://[ip]/cgi-bin/cmh/get_json.sh?file=template_data.json

So, when you're looking at the devices, you'll note that the camera has device template #47. Look in template_data for the "DeviceTemplate" section and you'll see #47 is an "IP Camera".

To see what commands you can send an IP Camera, look in: DeviceTemplate_DeviceCommandGroup and see that device template 47, the IP camera, implements the DeviceCommandGroups: 26, 54, 68. ie:

"DeviceTemplate_DeviceCommandGroup_47_26": {
"FK_DeviceTemplate":"47",
"FK_DeviceCommandGroup":"26"
},

Now, looking in DeviceCommandGroup_Command we see command group #26 includes command #84 and 973:

"DeviceCommandGroup_Command_26_84": {
"FK_DeviceCommandGroup":"26",
"FK_Command":"84",
"Description":"Get's a picture from a specified surveilance camera"
},
"DeviceCommandGroup_Command_26_973": {
"FK_DeviceCommandGroup":"26",
"FK_Command":"973",
"Description":"Take a picture and store it in the archive"
},

to see more on those commands, look in the Command section:

"PK_Command_84": {
"Description":"Get Video Frame",
"FK_CommandCategory":"9"
},

So the command is called "Get Video Frame" and the CommandCategory is 9. We can see that from command #84 that it takes a couple parameters by looking in Command_CommandParameter:

"Command_CommandParameter_84_19": {
"FK_Command":"84",
"FK_CommandParameter":"19",
"Description":"The video frame",
"IsOut":"1"
},
"Command_CommandParameter_84_61": {
"FK_Command":"84",
"FK_CommandParameter":"61",
"Description":"Frame height",
"IsOut":"0"
},

Parameter #19 is an output paramter and contains the video frame, parameter #61 is an input parameter and contains the frame height, and there are other parameters too.

According to the section "CommandParameter", the description of the parameter #61 is Height

"PK_CommandParameter_61": {
"Description":"Height",
"FK_ParameterType":"2"
},

and the type of paramter is an int:

"PK_ParameterType_2": {
"Description":"int"
},

Now, look back at the above example for retrieving a video frame: http://192.168.80.1:3451/messagesend?from=1&to=20&response=o&type=1&id=84&60=640&61=480

You're sending a message of type 1 (a command) and id 84 (get video frame) from device #1 to device #20 (assuming #20 is an ip camera). We want to wait to get the output parameters (so response=o), and the parameter 61 (height) is 480.

Now the response, which includes [20U=18373], means parameter #19/#20 contain the image and type of image being returned.