jenswilly.dk


Blue Maestro sensor, openHAB and HomeKit

Since I’m already working on a device for showing the temperature reported from a Bluetooth sensor (read more) and I’ve also been doing some home automation stuff using openHAB, I thought it ought to be possible to get the openHAB server to report the temperature as well. It is. This is how it works:

  1. The BLE device reports the current temperature in BLE advertisement packages.
  2. The Raspberry Pi server runs a Python script as a service that performs BLE scanning and looks at the advertisement packages, identifying and parsing packages from the BLE device.
  3. Whenever a matching BLE advertisement package is found, the current temperature is extracted (along with humidity and dewpoint) and posted to an MQTT server.
  4. The openHAB server running on the same Raspberry Pi uses the MQTT Binding to connect to the MQTT server and things and items are configured to subscribe to the appropriate topics and expose the temperature and timestamp.
  5. The openHAB server also uses the HomeKit Add-on to publish the temperature as a HomeKit compatible temperature sensor so it's available on all my iOS/watchOS/macOS devices.

Awesome. But there were several steps along the path. Read on for the details and code...

The Python Script

The Blue Maestro sensors I’m using (including the Pebble and Tempo Disc) reports their data as part of BLE advertisement packages. So the devices work as BLE broadcasters. All data is broadcast in a manufacturer specific data field of the BLE advertisement packages.

Thus, it is not necessary to make a connection to the device and multiple clients can listen to the broadcast data at the same time.

Blue Maestro used to make the details of the protocol available in a document named “Temperature & Humidity Sensor API and Command Guide 2.4” but I can’t seem to find it anywhere on their site anymore. Fortunately, I downloaded it when it was available and you might find it somewhere on the Internet. A with only a little bit of package sniffing it's pretty easy to figure out.

I found psyciknz' repo on Github. This repository contains a bunch of Python scripts. Among them a script that performs BLE scanning using the Bluez Python package and looks for Blue Maestro broadcast advertisements and extracts all data fields.

I did need to make a couple of tweaks and fixes for the script to work as I wanted, but many thanks to psyciknz for making the initial work available. 🙇‍♂️

My current code is available on GitHub here: https://github.com/jenswilly/OpenHAB-Scripts. Note that the config.json file is encrypted using git-crypt (since it contains my MQTT broker credentials) so should make your own config file – the format is documented in the readme.md.

I have made the following changes:

  • Changes to the MQTT (using the paho-mqtt Python package) handling:
    • I don’t like sending any IoT data in plain text so the MQTT connection now uses TLS and username/password credentials. (I use the CloudMQTT broker.)
    • Instead of using the Homie format, I publish the raw values to individual MQTT topics specified in the configuration file
    • A single connection with a background network loop is now used
  • The config file parsing needed a little tweaking
  • Likewise for the logging
  • I removed most of the debug print and log statements since I didn’t want the system logs flooded
  • Removed all the files I didn’t need (yes, I didn’t need to keep the BlueTooth directory but I already entered that path in a bunch of scripts so…)
  • The script now sleeps (duration specified in config) only when a matching BLE device has been found and values have been published to the broker.

You should be able to run the script directly from the BlueTooth directory with:

sudo python mqtt.bluetooth.loop.py -c config.json

Note: you need to run it as sudo unless you can figure out how to grant permissions to your regular user to the BLE subsystem (this Stackoverflow reply describes the process but I never got it working).

There is a bunch of print statements so you should see output like this:

Configuration:
{u'MQTT': {u'USERNAME': u'(redacted)', u'HOST': u'm24.cloudmqtt.com', u'PASSWORD': u'(redacted)', u'TLS_CERT': u'/etc/ssl/certs/ca-certificates.crt', u'PORT': 22112, u'KEEPALIVE': 10}, u'topics': {u'battery': u'bluemaestro/battery', u'timestamp': u'bluemaestro/timestamp', u'dewpoint': u'bluemaestro/dewpoint', u'temp': u'bluemaestro/temp', u'humidity': u'bluemaestro/humidity'}, u'bluetooth': {u'frequency': 300, u'log': u'ble-log.txt'}}
Connecting to m24.cloudmqtt.com:22112
rc: 0
Date Time:   2019-06-28T20:50:58
number of beacons found 0
Date Time:   2019-06-28T20:50:59
number of beacons found 1
Temp: 24.7 C, humidity: 39.2%, dewpoint: 10.0 C, battery: 100.392156863%
mid: 1
mid: 2
mid: 3
mid: 4
mid: 5

Running as service

To install the Python script as a service that runs automatically upon startup, do the following (which is copied from this article by Diego Acuña):

  1. cd to the BlueTooth directory
  2. Edit the bluemaestro-mqtt.service file so the paths match your file locations (note: you need full paths to both the .py file and the config file)
  3. sudo cp bluemaestro-mqtt.service /lib/systemd/system/
  4. chmod +x mqtt.bluetooth.loop.py (I'm actually not quite sure this step is necessary but it is in the original article, so…)
  5. sudo systemctl daemon-reload
  6. sudo systemctl enable bluemaestro-mqtt.service
  7. sudo systemctl start bluemaestro-mqtt.service

You can verify if the service is running with systemctl status bluemaestro-mqtt.service. And if you have a BLE sensor within range you should also see data being published to the MQTT topics.

Thanks, Diego 🙇‍♂️

openHAB and MQTT

To use MQTT data sources in openHAB, you first need to enable the MQTT binding. In Paper UI, go to "Add-ons" → "Bindings" tab, find the "MQTT Binding" and click "INSTALL".

Next, you need to configure the broker connection and available channels. In a .things file, add the following:

Bridge mqtt:broker:cloudmqtt "MQTT Broker: CloudMQTT" [ host="something.cloudmqtt.com", port=22222, secure=true, username="username", password="password" ]

Edit the name, host, port, username and password to match your credentials. If your broker doesn't use TLS with username/password, read the documentation for more info about configuring the bridge. And you will have to tweak the Python script, too.

The openHAB log should show whether you are successfully connected:

...
12:59:22.616 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:broker:cloudmqtt' changed from OFFLINE to ONLINE
12:59:22.619 [INFO ] [smarthome.event.ThingUpdatedEvent    ] - Thing 'mqtt:broker:cloudmqtt' has been updated.
12:59:22.688 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:jwj_openhab_thing' changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE
...

MQTT channels

If the connection is good, next step is to define channels which can be linked to things. In a .things file, add the following:

Thing mqtt:topic:jwj_openhab_thing (mqtt:broker:cloudmqtt) {
Channels:
    Type number : temp_value "Outside temperature" [ stateTopic="bluemaestro/temperature", commandTopic="" ]
    Type datetime : temp_timestamp "Temperature timestamp" [ stateTopic="bluemaestro/timestamp", commandTopic="" ]
}

Adjust the topic names to match the ones in your config file. In this case I'm using only temperature and timestamp. I could also add channels for humidity, dewpoint and battery.

MQTT items

Then we need to add items for the channels. In an .items file, add the following:

Number MQTT_OutsideTemp "Temperature" <temperature> [ "CurrentTemperature" ] { channel="mqtt:topic:jwj_openhab_thing:temp_value" }
DateTime MQTT_OutsideTempTimestamp "[%1$tH:%1$tM]" <calendar> { channel="mqtt:topic:jwj_openhab_thing:temp_timestamp" }

Sitemap

Put the items in a sitemap by adding the following to a .sitemap file:

Text item=MQTT_OutsideTemp label="Outside temperature"
Default item=MQTT_OutsideTempTimestamp label="Outside temp. last updated"

Verify that no errors show up in the log when you save the files and the reload the UI and enjoy your success!

Leave a Reply