I moved house a couple of months back to, what my partner likes to describe as, our "forever home". As such, I was keen to start looking into home monitoring / automation again as I knew things had progressed significantly since I last considered such things and I was keen to see what could be done. After lots of investigation I believe I have found the basis of a home automation solution which meets my current needs, should be extensible moving forward and which does not compromise the securty or privacy of my home.
This post is, I hope, the first in a series that discuss the various devices and software I use to monitor and automate my home.
Private by design
Firstly a word on privacy. While "smart devices" are everywhere nowadays, they almost all require connection to the internet and, often, a subscription to a "cloud" service. I wrote about this before and expressed my desire for smart devices to "drop the 'Inter' from IoT and expand the 'net' to become a 'Network of Things', or NoT" which could optionally be bridged to the internet if desired. To quote myself:
I've long loved the idea of home automation. From X10 and LightwaveRF through to modern Bluetooth and Wifi connected devices, I have played with dozens of technologies in search of home automation nirvana. But recently I have watched with growing bewilderment at the incredible number of "cloud-connected" home automation devices being released and the eagerness with which they're snapped up by naive consumers hungry to control everything from the carefree comfort of their iPhone.
You see, while you can buy a myriad of IoT devices off the shelf nowadays, they nearly all come with some form of "cloud-service" that is necessary in order for the device to work as sold. As the more wily of reader will no doubt be aware, this exposes your home network to innumerable security concerns, potential abuses and an external point of failure that cannot be closed/fixed without sacrificing some or all of the functionality of the new fangled device.
Since I wrote post, the market for internet connected home automation devices has exploded... as have the concerns around the privacy, security and functionality (or unwanted functionality) of the devices. I am not alone in being sufficiently concerned by these issues that I choose the devices I allow on my home network very carefully and monitor them closely.
In short, the list of "off-the-shelf" devices I would feel comfortable having on my home network is very short and, until recently, I was resigned to having to build these devices myself using single board computers or Wifi connected microcontrollers. That is until a family of devices came to my attention that originated from the most unlikely of places...
Mi Smart Home
China isn't the first country that comes to mind when you think about privacy yet, with the release of the "Mi Smart Home" family of devices, the Chinese electronics manufacturer Xiaomi Inc seems to have delivered a smart-device eco-system that is privacy-friendly... albeit somewhat tacitly as we will see below.
The Mi Smart family consists of a number of small, battery powered Zigbee devices such as temperature and humidity sensors, door / window sensors, movement sensors and various switches. Oh... and a curious 'cube' controller.
These devices connect to your internal Wifi network via a Gateway device. In addition to providing the Zigbee to Wifi bridge, this gateway also provides an ambient light sensor, a very useful RGB light and a umm... not so useful speaker. This is all packaged in a small, round, 30 gram device about 8 centimeters in diameter and 3 cenitmeters deep. Frustratingly the gateway only comes with the Chinese/Australian type I plug so you need an adapter to use it in the UK which adds significantly to the depth (although I've taken to using a convenient extension lead instead).
To set up the Wifi Gateway device and add sensors, you have to install the Xiaomi Mi Home app which, to be frank, is a privacy nightmare. The list of permissions it needs is quite incredible:
Version 5.1.1 can access:
- Device & app history
- retrieve running apps
- Identity
- find accounts on the device
- add or remove accounts
- Calendar
- read calendar events plus confidential information
- add or modify calendar events and send emails to guests without owners' knowledge
- Contacts
- find accounts on the device
- read your contacts
- modify your contacts
- Location
- approximate location (network-based)
- precise location (GPS and network-based)
- SMS
- read your text messages (SMS or MMS)
- receive text messages (SMS)
- send SMS messages
- Phone
- directly call phone numbers
- reroute outgoing calls
- read call log
- read phone status and identity
- write call log
- Photos / Media / Files
- read the contents of your USB storage
- modify or delete the contents of your USB storage
- Storage
- read the contents of your USB storage
- modify or delete the contents of your USB storage
- Camera
- take pictures and videos
- Microphone
- record audio
- Wi-Fi connection information
- view Wi-Fi connections
- Device ID & call information
- read phone status and identity
- Other
- download files without notification
- interact across users
- full licence to interact across users
- transmit infrared
- modify secure system settings
- read Home settings and shortcuts
- write Home settings and shortcuts
- view network connections
- create accounts and set passwords
- read battery statistics
- pair with Bluetooth devices
- access Bluetooth settings
- send sticky broadcast
- change network connectivity
- allow Wi-Fi Multicast reception
- connect and disconnect from Wi-Fi
- disable your screen lock
- control flashlight
- full network access
- change your audio settings
- control Near-Field Communication
- read sync settings
- run at startup
- draw over other apps
- use accounts on the device
- control vibration
- prevent device from sleeping
- modify system settings
- toggle sync on and off
- install shortcuts
- uninstall shortcuts
Scary huh! How can I possibly claim these devices are privacy friendly when you've basically just given a Chinese company permission to do pretty much anything it likes with your phone's hardware and data? Well, notice how I said you need the app to "set up the Wifi Gateway device and add sensors". Once they're setup you no longer need the app and, furthermore, once "local network functions" are enabled (more on this in a second) neither the gateway nor sensors need internet access to function.
So, how to go about using these sensors in a privacy friendly way?
Preparation
To get set up with these devices without compromising your privacy, you will need:
A dedicated VLAN and Wifi network for "smart home" devices
I recommend putting any 3rd party "smart" devices in an isolated environment within which you can easily enable or disable internet access. To do this I am using the VLAN feature of my Draytek 2860 router which involves creating a second VLAN on my network, enabling "Inter-LAN' routing so I could access the VLAN from my existing subnet and, finally, adding firewall rules to prevent devices on this VLAN from accessing the internet / other vlans.
A clean Android device
I had an old Android phone laying around on which I performed a hard-reset and wiped all user-data. With a clean device I could then install the app without worrying about sharing anything private with Xiaomi
Installation
First, install the Xiaomi Mi Home app on your clean Android device. On first run after installation you will be prompted for a region and asked to sign in. In order to use all the Mi Smart devices, you must select "Mainland China" for your region (this doesn't affect the language in the app) following which you can just create a new account to sign in.
Once the app is installed you can plug in a gateway. This results in a nice flashing yellow ring of light around the device... and a harsh female voice babbling Chinese at an almost intolerable volume; basically an audio and visual indication that the gateway is in "pairing" mode.
Go ahead and pair the gateway to the app following the walk-through here. At this point it's a good idea to also install any additional "sub devices" you have (i.e. the various Zigbee sensors) which can be done by following this walkthrough.
Finally, in order to use the gateway and sensors without the app and/or internet access, it is necessary to enable "local network functions". This can be done from the app by following the instructions here. Quite why Xiaomi decided to hide what its possible the killer feature of these devices behind a "secret" button I've no idea... fortunately it's an open secret and Xiaomi don't seem to be making any effort to conceal it further.
With all the above done, feel free to junk the app and disable internet access from the "smart device" subnet / ip range.
Usage
Once 'local network functions' have been enabled, each Gateway uses the multicast address 224.0.0.50 to broadcast UDP messages on port 9898 from the gateway and sub-devices. The gateway publishes a "heartbeat" messages every 10-15 seconds or so meaning you can easily determine everything is working by spinning up Wireshark (on a device connected to the "smart device" subnet) filtering out anything that isn't a UDP message (ip.proto == "udp") and looking for messages from the gateway IP address. You should eventually see something like this:
Once you can see these messages you can start interacting with the gateway and devices using various commands. For example, to get a list of the sub-devices from a gateway you can send a 'get_id_list' command using a UDP packet container this string (as ASCII encoded binary) to the gateway's IP address, again on port 9898:
{"cmd":"get_id_list"}
This will result in a get_id_list_ack response containing a list of sid (aka 'simple' id) values for devices registered with the gateway (including the gateway sensor/status itself) as follows:
{"cmd":"get_id_list_ack","sid":"7811dcb06972","token":"L0DI4IiFAvAgInyL","data":"[\"158d0001a200f5\",\"158d0001c1cdfb\"]"}
You're then able to retrieve the device status using the read command for each sid:
{"cmd":"read","sid":"7811dcb06972"}
Which will return a read_ack response containing device specific information in the 'data' property. For example, if you send a read command specifying the sid of the gateway you will receive something like the following:
{"cmd":"read_ack","model":"gateway","sid":"7811dcb06972","short_id":0,"data":"{\"rgb\":0,\"illumination\":1292,\"proto_version\":\"1.0.9\"}"}
Furthermore, when the status of a sensor changes (for example a door sensor opens or closes) a report message is broadcast as follows:
{"cmd":"report","model":"magnet","sid":"158d0001c1cdfb","short_id":56258,"data":"{\"status\":\"open\"}"}
Here's one I made earlier...
I used the above information to create a small application for listening to and interacting with the gateway and devices. Mostly out of interest, I used Microsoft's Orleans framework to create a console application which wraps up the interaction with various devices into strongly typed agents (also an agent approach seemed to nicely mirror the segregated nature of the devices themselves).
I didn't take this too far as I subsequently decided to use an "off-the-shelf" system for interacting with the Xiaomi devices (the subject of my next "smart home" post) but it's a decent proof-of-concept. I've published the source in a repository on Github; feel free to have a play and drop me a line with any questions you might have.
In the next post I'll disucss the off-the-shelf system I'm now using to privately interact with the Xiaomi devices.