From Heatweb Wiki
Jump to: navigation, search

The Energy Saving Trust’s EMBED hub is an interactive online resource of building energy performance data, collated from energy monitoring field trials and various other studies. What EMBED offers

  • A single place to store and organise project information, documents and data.
  • In-depth monitoring data from past and current trials in a standard format.
  • Search and filter options for similar buildings and quick visualisation of large data sets.
  • Easy data sharing with other users for collaboration and dissemination.
  • Options to keep sensitive information and documents private and secure from other users.
  • A web API to extract data for your own online tools.
  • Secure online transfer of live monitoring data from any device using the Energy Saving Trust's open data format.

Taken from: http://www.energysavingtrust.org.uk/businesses/embed-building-performance-platform

Can the data be saved so that no-one other than the home owner knows where the data originated?

Yes, data can either be saved with full details of the property (this was the main requirement for retrofit for the future) or it can be saved anonymously so that householders could look at their own data. There are also different levels of access so that a landlord could see the data from all of their properties but a tenant would only see their property for example.

How much data can the system handle, and is it prepared for the inclusion of thousands of properties, each feeding data at up to 10 second resolution, from up to 10 sensors? Where are the limits?

Embed is a cloud-based solution so can be expanded almost infinitely as demand grows. It is a Cassandra database meaning scale is not a problem. It can accept and easily process very high volumes of high frequency data. It is also an open source platform, keen for contributors to get involved in the project.

Is EMBED free to use ?

Currently EMBED will be made available for free to projects that are happy to share data publicly. All public data are anonymised before being made available, some of this is automated but some of it relies on the data originator.

Potential uses for EMBED

  • Manufacturers who want performance data and alarms.
  • Landlords and ESCOs to extract meter data for billing.
  • Heat network managers, to monitor and alarm system inefficiencies.
  • Ofgem, to confirm RHI system efficiencies.
  • DECC, to monitor the effectiveness of research projects.
  • Installers, to commission systems.
  • Service engineers, to see whats up before arriving on site.
  • Occupants, to save money through improved management and efficiency, and have more effective backup.
  • As the database behind browser or app ENE3 Home Display Systems.

Thermal Integrations Intentions

As a manufacturer of HIUs, we are focused on supplying a superior product at a competitive price. We have had our systems installed alongside most billing systems on the market, and on a number of contracts we have been called upon to design and supply the network infrastructure required to get data from heat meters off-site, typically through Ethernet.

Our new generation of HIUs and Heat Banks make use of our IHIU Control Systems, a small Linux controller with Ethernet and Wifi connectivity, with M-Bus functionality in development. They are connected to a number of system sensors covering domestic hot water, central heating, and the primary operation. We fit Heat Meters into our HIUs, and as such, a host of sensor data and meter data is available from our system.

As detailed in the section above, the uses for this data are numerous, however we feel the potential for making use of a single database for domestic energy use can only be realised if the industry as a whole contributes to the data set. We have used remote sensor data for years now, in the study of how custom made heating systems perform, and on numerous occasions we have identified, and fixed, inefficiencies that would go unnoticed without the data. The knowledge is often transferable to improving systems of a similar type elsewhere. The use of common database and data visualisation tools, will allow the industry as a whole to ensure equipment runs as expected, rather that mysteriously costing significantly more to run as originally planned. Systems that perform to exceptional levels of performance can educate everyone.

It is in our interest to design systems to work with a standard non-commercial database as a minimum, and we feel it is in the interests of service companies to be able to access data from both heat meters and HIUs via a standard portal. It is certainly in the interests of our product users to have their data held in a single secure public database where they control access rather than an array of private companies.

It is our intention to be completely open about the process of connecting equipment to the EMBED system, and how data can then be visualised and analysed. This page will be updated and examples published as we progress.

See our own Compact 35 Pellet Boiler Test Rig system, used as a development and demonstration platform for advancing the control of pellet boilers, as well as testing Embed connectivity.



EMBED Webs Pages

To access the system you first need to register at www.getembed.com, and from there you can view public data.

Connecting an IHIU controller to EMBED

This is a record of the process involved in sending data to the EMBED site. We already have a fully functioning pellet boiler test rig in the factory that is providing a stream of sensor and operational data to our own servers, via an IHIU controller. The IHIU controller is itself a Linux web server, running PHP, so has all the required connectivity to talk to any web site. We aim to initially send data to our own server, and from then forwards the necessary data to EMBED. Once this is achieved, we will also get the IHIU talking directly to EMBED - which should be straightforward as both the IHIU and our web servers run using the same software infrastructure - PHP.


Programmes are areas within the EMBED system under which projects and properties are organised. They are assigned by the EMBED Administrator. For the purposes of this development, we have obtained a programme called Heatweb.


Projects are filed under programmes. They are created through the getembed.com website, once logged on. We have created a Project called MMSP, representing the Monitoring and Metering Service Package on offer by DECC and Ofgem for takers of the RHI (Renewable Heat Incentive). In brief, a small financial incentive is provided to install full monitoring alongside a renewable installation. Although the scheme has been running for over a year, there have been no takers - mainly because the task of setting up a monitoring system backed by online services, is something plumbers do not understand, and are not yet engaged with. If anything they are fearful of having customers phone them every time they spot something on a graph they don't understand. That and the fact there are no decent monitoring packages that do not cost hundreds of pounds. Linking a low cost general purpose controller to the EMBED database, is a perfect solution.

For some devices, such as pellet boilers, there is a need for manufacturer specific data to analyse data from installations. In our case, the ratio of auger rotation to volumetric delivery of pellets is a figure that can be determined through a test, or over time, but a simple connection to an online database would speed up and simplify the process. It also allows us to check that the boiler delivers pellets as expected by the manufacturer, and could give an indication of a faulty auger, or variations in fuel quality. As part of this project we will create a web page on our server to send the boiler specific data (once obtained) to the IHIU controller.

Question: Is there already a central public database of manufacturer/model specific data presented via an API ?

When inputting projects, you also need to provide a Project Code of your own choosing, as well as a description of the type of project, and the project itself.


Properties are filed under projects. They are created through the getembed.com website, once you have created a project. Our test rig is located in the Thermal Integration factory in Sudbury, Suffolk. This will be the property. We have given it a code of TIL1.


A Device is a piece of monitoring equipment that feeds back data. This is where it starts to get more involved. It is easiest to start by describing what we know about our device, or rather what data it provides:

  • Temperature and pressure from the boiler flow
  • Temperature and flow from the boiler return
  • 5x temperature readings from the buffer store
  • Run signals from the pump, auger motor, fan, and pellet delivery system
  • Percentage run time of auger (averaged over one minute)
  • Air velocity in flue (not yet connected)
  • Heat Meter data
  • Fault Codes

We have selected these inputs as our aim is to obtain via EMBED the heat meter readings, and a confirmation the pellet boiler is working to expected efficiencies. It should be noted that we could just as easily be monitoring the efficiency of a gas boiler, HIU, or entire district heating network. The basic idea at this stage is to send the data to an online database, where software can analyse it automatically, and alarm any variations from the norm.

From these raw values, we also wish to obtain some calculated values. These can be be calculated in the IHIU controller and then sent to EMBED, or potentially calculated on EMBED.

  • Calculated values for Power, and Total Generation - will act as a check to the heat meter data
  • Calculated valued for Volumetric Fuel Use - calculated from auger rate
  • Calculated values for Boiler Efficiency (averaged over 5, 30, and 60 minutes, and per day)

As sensors can be expensive, part of our aim is to ascertain the minimum number of sensors required to determine correct function. Temperature is very cheap to measure, followed by pressure, and between them a host of other problems can be determined. However, you need to first match patterns in the data to known events (such a pump failure), that in future could remove the need for a more expensive sensor (such as a relay to pick up pump signal).

The first round of testing highlighted the benefits of using properly calibrated heat meter data when calculating energy flow over using off the shelf sensors. A tiny inaccuracy in either flow or temperatures is exaggerated at low temperature differences, and hard to confirm without comparing to heat meter data.

The system now incorporates acts as an M-Bus Master and can read heat meter data via the M-Bus connections, or an RS232 connection. This data is MID approved, and required for MMSP.


Making a Connection to EMBED

Github EMBED examples.

The following python script (courtesy of the Embed developers) is used to send data. Note that the entity id is the 3rd id code in the url when on the property page of the Embed site.

The code is run by the command:

python /mnt/sda1/arduino/www/python/embed_upload_measurements.py username password
import sys
import requests
import time
import json
from requests.auth import HTTPBasicAuth
## Info on this script refer to a testing instance on getembed.com
## Embed url
#URL = "https://www.getembed.com/4/"
URL = "http://www.getembed.com/4/"
## Add entity and device ids for properties you want to upload measurements to:
## Example:
## [{"entity1": ["device1.1", "device1.2"]}, {"entity2": ["device2.1", "device2.2"]}]
entities = [
    {"d0a91ad4-dd99-4ad1-8b6f-ad97c54a6aae": ["b4149ec6-7264-4f5c-b62a-09d3baff7808"]}
measurements = {"measurements": [
        "value": "0.87",
        "timestamp": "2015-08-24T10:30:00Z",
        "type": "Pressure",
## To run if you want to check device info
def check_device(entity_id, device_id, user, pwd):
    "Check device exists and device info."
    auth = HTTPBasicAuth(user, pwd)
    url = URL + "entities/" + entity_id + "/devices/" + device_id
    r = requests.get(url=url, auth=auth)
    print r.status_code
    print r.json()
## To run to upload measurements device per device
# NOTE: Update measurements content and make sure "type" matches current sensor type
def post_measurements(entity_id, device_id, user, pwd):
    "Post measurements for a device."
    auth = HTTPBasicAuth(user, pwd)
    url = URL + "entities/" + entity_id + "/devices/" + device_id + "/measurements/"
    data = json.dumps(measurements)
        r = requests.post(url=url, data=data, auth=auth)
        print r.status_code, r.reason
        if r.status_code == 202:
            print "Measurements uploaded for device ", device_id, " in property ", entity_id
    except requests.ConnectionError as e:
        print "Connection error. ", e
    except requests.ConnectTimeout as e:
        print "Connection timed out. ", e
## To run to upload measurements to several devices
def upload_all_measurements(user, pwd):
    "Add measurements for multiple devices."
    for entity in entities:
        for device in entity:
            post_measurements(entity, device, user, pwd)
if __name__ == "__main__":
    post_measurements("d0a91ad4-dd99-4ad1-8b6f-ad97c54a6aae", "b4149ec6-7264-4f5c-b62a-09d3baff7808", sys.argv[1], sys.argv[2])
    # check_device("45f93880-df89-4565-acd1-6a1d6c22792c", "d434ec0e-210f-4937-9873-6c42eddd3936", sys.argv[1], sys.argv[2])

This code results in the following response:


The Output on Embed

One a successful connection has been made, one can view the output on the Embed website in various ways, including detailed graphs.