Forecasting & scheduling

Once FlexMeasures contains data (see Posting data), you can enjoy its forecasting and scheduling services. Let’s take a look at how FlexMeasures users can access information from these services, and how you (if you are hosting FlexMeasures yourself) can set up the data science queues for this.

If you want to learn more about the actual algorithms used in the background, head over to Algorithms.

Maintaining the queues

Note

If you are not hosting FlexMeasures yourself, skip right ahead to How forecasting jobs are queued or Getting power forecasts (prognoses).

Here we assume you have access to a Redis server and configured it (see Redis).

Start to run one worker for each kind of job (in a separate terminal):

flexmeasures run-worker --queue forecasting
flexmeasures run-worker --queue scheduling

You can also clear the job queues:

flexmeasures clear-queue --queue forecasting
flexmeasures clear-queue --queue scheduling

When the main FlexMeasures process runs (e.g. by flexmeasures run), the queues of forecasting and scheduling jobs can be visited at http://localhost:5000/tasks/forecasting and http://localhost:5000/tasks/schedules, respectively (by admins).

When forecasts and schedules have been generated, they should be visible at http://localhost:5000/analytics.

How forecasting jobs are queued

A forecasting job is an order to create forecasts based on measurements. A job can be about forecasting one point in time or about forecasting a range of points.

In FlexMeasures, forecasting jobs are created by the server when new power, weather or price data arrives through the API (see Posting data). So technically, you don’t have to do anything to keep fresh forecasts.

The decision which horizons to forecast is currently also taken by FlexMeasures. For power data, FlexMeasures makes this decision depending on the asset resolution. For instance, a resolution of 15 minutes leads to forecast horizons of 1, 6, 24 and 48 hours. For price data, FlexMeasures chooses to forecast prices forward 24 and 48 hours These are decent defaults, and fixing them has the advantage that scheduling scripts (see below) will know what to expect. However, horizons will probably become more configurable in the near future of FlexMeasures.

Historical forecasts

There might be reasons to add forecasts of past time ranges. For instance, for visualisation of past system behaviour and to check how well the forecasting models have been doing on a longer stretch of data.

If you host FlexMeasures yourself, we provide a CLI task for adding forecasts for whole historic periods. This is an example call:

flexmeasures add forecasts --from_date 2020-01-02 --to_date 2020-6-30 --horizon_hours 6  --resolution 60 --asset-id 2

Here, forecasts are being computed for asset 2, with one horizon (6 hours) and a resolution of 60 minutes. This is half a year of data, so it will take a while. You can also queue this work to workers (see above) with the additional --as-job parameter (though in general we’d advise to dispatch this work in smaller chunks).

How scheduling jobs are queued

In FlexMeasures, a scheduling job is an order to plan optimised actions for flexible devices. It usually involves a linear program that combines a state of energy flexibility with forecasted data to draw up a consumption or production plan ahead of time.

We already learned about the postUdiEvent endpoint in Posting flexibility states, where we saw how to post a state of flexibility (in this case, the state of charge of a battery at a certain point in time).

This endpoint can also be used to request a future state of charge (using soc-with-target in the entity address).

As an example, consider the same UDI event as we saw earlier (in Posting flexibility states), but with an additional target value.

{
    "type": "PostUdiEventRequest",
    "event": "ea1.2021-01.io.flexmeasures.company:7:10:204:soc-with-targets",
    "value": 12.1,
    "datetime": "2015-06-02T10:00:00+00:00",
    "unit": "kWh",
    "targets": [
        {
            "value": 25,
            "datetime": "2015-06-02T16:00:00+00:00"
        }
    ]
}

Here we have described the state of charge at 10am to be 12.1. In addition, we requested that it should be 25 at 4pm. For instance, this could mean that a car should be charged at 90% at that time.

Now here is a task that requires some scheduling. If FlexMeasures receives this UDI Event, a scheduling job will be made and put into the queue. In turn, the forecasting job creates a proposed schedule. We’ll look a bit deeper into those further down in Getting schedules (control signals);

Note

Even without a target state of charge, FlexMeasures will create a scheduling job. The flexible device can then be used with more freedom to reach the system objective (e.g. buy power when it is cheap, store it, and sell back when it’s expensive).

Getting power forecasts (prognoses)

Prognoses (the USEF term used for power forecasts) are used by FlexMeasures to determine the best control signals to valorise on balancing opportunities.

You can access forecasts via the FlexMeasures API at GET /api/v2_0/getPrognosis. Getting them might be useful if you want to use prognoses in your own system, or to check their accuracy against meter data, i.e. the realised power measurements. The FlexMeasures UI also lists forecast accuracy, and visualises prognoses and meter data next to each other.

A prognosis can be requested for a single asset at the getPrognosis endpoint, at a URL looking like this:

https://company.flexmeasures.io/api/<version>/getPrognosis

This example requests a prognosis for 24 hours, with a rolling horizon of 6 hours before realisation.

{
    "type": "GetPrognosisRequest",
    "connection": "ea1.2021-01.io.flexmeasures.company:fm1.1",
    "start": "2015-01-01T00:00:00+00:00",
    "duration": "PT24H",
    "horizon": "PT6H",
    "resolution": "PT15M",
    "unit": "MW"
}

Getting schedules (control signals)

We saw above how FlexMeasures can create optimised schedules with control signals for flexible devices. You can access the schedules via the GET /api/v2_0/getDeviceMessage endpoint. The URL then looks like this:

https://company.flexmeasures.io/api/<version>/getDeviceMessage

Control signals can be queried by UDI event for up to 1 week after the UDI event was posted (ask your host if you need to keep them around longer). This example of a request body shows that we want to look up a control signal for UDI event 203 (which was posted previously, see Posting flexibility states).

{
    "type": "GetDeviceMessageRequest",
    "event": "ea1.2021-01.io.flexmeasures.company:7:10:203:soc"
}

The following example response indicates that FlexMeasures planned ahead 45 minutes for this battery. The list of consecutive power values represents the target consumption of the battery (negative values for production). Each value represents the average power over a 15 minute time interval.

{
    "type": "GetDeviceMessageResponse",
    "event": "ea1.2021-01.io.flexmeasures.company:7:10:203",
    "values": [
        2.15,
        3,
        2
    ],
    "start": "2015-06-02T10:00:00+00:00",
    "duration": "PT45M",
    "unit": "MW"
}

How to interpret these control signals?

One way of reaching the target consumption in this example is to let the battery start to consume with 2.15 MW at 10am, increase its consumption to 3 MW at 10.15am and decrease its consumption to 2 MW at 10.30am.

However, because the targets values represent averages over 15-minute time intervals, the battery still has some degrees of freedom. For example, the battery might start to consume with 2.1 MW at 10.00am and increase its consumption to 2.25 at 10.10am, increase its consumption to 5 MW at 10.15am and decrease its consumption to 2 MW at 10.20am. That should result in the same average values for each quarter-hour.