Toy example: Introduction and setup

This page is a starting point of a series of tutorials that will help you get practical experience with FlexMeasures.

Let’s walk through an example from scratch! We’ll …

  • install FlexMeasures

  • create an account

  • load hourly prices

What do you need? Your own computer, with one of two situations: either you have Docker or your computer supports Python 3.8+, pip and PostgresDB. The former might be easier, see the installation step below. But you choose.

Below are the flexmeasures CLI commands we’ll run, and which we’ll explain step by step. There are some other crucial steps for installation and setup, so this becomes a complete example from scratch, but this is the meat:

# setup an account with a user and an energy market (ID 1)
$ flexmeasures add toy-account
# load prices to optimise the schedule against
$ flexmeasures add beliefs --sensor 1 --source toy-user prices-tomorrow.csv --timezone Europe/Amsterdam

Okay, let’s get started!

Note

You can copy the commands by hovering on the top right corner of code examples. You’ll copy only the commands, not the output!

Install Flexmeasures and the database

If docker is running on your system, you’re good to go. Otherwise, see here.

We start by installing the FlexMeasures platform, and then use Docker to run a postgres database and tell FlexMeasures to create all tables.

$ docker pull lfenergy/flexmeasures:latest
$ docker pull postgres
$ docker network create flexmeasures_network

After running these commands, we can start the Postgres database and the FlexMeasures app with the following commands:

$ docker run --rm --name flexmeasures-tutorial-db -e POSTGRES_PASSWORD=fm-db-passwd -e POSTGRES_DB=flexmeasures-db -d --network=flexmeasures_network postgres:latest
$ docker run --rm --name flexmeasures-tutorial-fm --env SQLALCHEMY_DATABASE_URI=postgresql://postgres:fm-db-passwd@flexmeasures-tutorial-db:5432/flexmeasures-db --env SECRET_KEY=notsecret --env FLEXMEASURES_ENV=development --env LOGGING_LEVEL=INFO -d --network=flexmeasures_network -p 5000:5000 lfenergy/flexmeasures

To establish the FlexMeasures database structure, execute:

$ docker exec flexmeasures-tutorial-fm bash -c "flexmeasures db upgrade"

Note

A tip on Linux/macOS ― You might have the docker command, but need sudo rights to execute it. alias docker='sudo docker' enables you to still run this tutorial.

Now - what’s very important to remember is this: The rest of this tutorial will happen inside the flexmeasures-tutorial-fm container! This is how you hop inside the container and run a terminal there:

$ docker exec -it flexmeasures-tutorial-fm bash

To leave the container session, hold CTRL-D or type “exit”.

To stop the containers, you can type

$ docker stop flexmeasures-tutorial-db
$ docker stop flexmeasures-tutorial-fm

To start the containers again, do this (note that re-running the docker run commands above deletes and re-creates all data!):

$ docker start flexmeasures-tutorial-db
$ docker start flexmeasures-tutorial-fm

Note

For newer versions of MacOS, port 5000 is in use by default by Control Center. You can turn this off by going to System Preferences > Sharing and untick the “Airplay Receiver” box. If you don’t want to do this for some reason, you can change the host port in the docker run command to some other port, for example 5001. To do this, change -p 5000:5000 in the command to -p 5001:5000. If you do this, remember that you will have to go to localhost:5001 in your browser when you want to inspect the FlexMeasures UI.

Note

Got docker-compose? You could run this tutorial with 5 containers :) ― Go to Seeing it work: Running the toy tutorial.

Add some structural data

The data we need for our example is both structural (e.g. a company account, a user, an asset) and numeric (we want market prices to optimize against).

Let’s create the structural data first.

FlexMeasures offers a command to create a toy account with a battery:

$ flexmeasures add toy-account --kind battery

Generic asset type `solar` created successfully.
Generic asset type `wind` created successfully.
Generic asset type `one-way_evse` created successfully.
Generic asset type `two-way_evse` created successfully.
Generic asset type `battery` created successfully.
Generic asset type `building` created successfully.
Generic asset type `process` created successfully.
Creating account Toy Account ...
Toy account Toy Account with user toy-user@flexmeasures.io created successfully. You might want to run `flexmeasures show account --id 1`
Adding transmission zone type ...
Adding NL transmission zone ...
Created day-ahead prices
The sensor recording day-ahead prices is day-ahead prices (ID: 1).
Created <GenericAsset None: 'toy-battery' (battery)>
Created discharging
Created <GenericAsset None: 'toy-solar' (solar)>
Created production
The sensor recording battery discharging is discharging (ID: 2).
The sensor recording solar forecasts is production (ID: 3).

And with that, we’re done with the structural data for this tutorial!

If you want, you can inspect what you created:

$ flexmeasures show account --id 1

===========================
Account Toy Account (ID: 1)
===========================

Account has no roles.

All users:

ID  Name      Email                     Last Login    Last Seen    Roles
----  --------  ------------------------  ------------  -----------  -------------
1  toy-user  toy-user@flexmeasures.io  None          None         account-admin

All assets:

ID  Name           Type     Location
----  -----------  -------  -----------------
2  toy-building   building  (52.374, 4.88969)
3  toy-battery    battery   (52.374, 4.88969)
4  toy-solar      solar     (52.374, 4.88969)

$ flexmeasures show asset --id 2

=========================
Asset toy-building (ID: 2)
=========================

Type      Location           Attributes
-------   -----------------  ----------------------------
building  (52.374, 4.88969)

====================================
Child assets of toy-building (ID: 2)
====================================

Id       Name               Type
-------  -----------------  ----------------------------
3        toy-battery        battery
4        toy-solar          solar

No sensors in asset ...

$ flexmeasures show asset --id 3

==================================
Asset toy-battery (ID: 3)
Child of asset toy-building (ID: 2)
==================================

Type     Location           Attributes
-------  -----------------  ----------------------------
battery  (52.374, 4.88969)  capacity_in_mw: 0.5
                            min_soc_in_mwh: 0.05
                            max_soc_in_mwh: 0.45
                            sensors_to_show: [1, [3, 2]]

====================================
Child assets of toy-battery (ID: 3)
====================================

No children assets ...

All sensors in asset:

ID  Name         Unit    Resolution    Timezone          Attributes
----  -----------  ------  ------------  ----------------  ------------
2  discharging  MW      15 minutes    Europe/Amsterdam

Yes, that is quite a large battery :)

Note

Obviously, you can use the flexmeasures command to create your own, custom account and assets. See CLI Commands. And to create, edit or read asset data via the API, see Version 3.0.

We can also look at the battery asset in the UI of FlexMeasures (in Docker, the FlexMeasures web server already runs, on your PC you can start it with flexmeasures run). Visit http://localhost:5000/ (username is “toy-user@flexmeasures.io”, password is “toy-password”):

https://github.com/FlexMeasures/screenshots/raw/main/tut/toy-schedule/asset-view-dashboard.png

Note

You won’t see the map tiles, as we have not configured the MAPBOX_ACCESS_TOKEN. If you have one, you can configure it via flexmeasures.cfg (for Docker, see Configuration and customization).

Add some price data

Now to add price data. First, we’ll create the CSV file with prices (EUR/MWh, see the setup for sensor 1 above) for tomorrow.

$ TOMORROW=$(date --date="next day" '+%Y-%m-%d')
$ echo "Hour,Price
$ ${TOMORROW}T00:00:00,10
$ ${TOMORROW}T01:00:00,11
$ ${TOMORROW}T02:00:00,12
$ ${TOMORROW}T03:00:00,15
$ ${TOMORROW}T04:00:00,18
$ ${TOMORROW}T05:00:00,17
$ ${TOMORROW}T06:00:00,10.5
$ ${TOMORROW}T07:00:00,9
$ ${TOMORROW}T08:00:00,9.5
$ ${TOMORROW}T09:00:00,9
$ ${TOMORROW}T10:00:00,8.5
$ ${TOMORROW}T11:00:00,10
$ ${TOMORROW}T12:00:00,8
$ ${TOMORROW}T13:00:00,5
$ ${TOMORROW}T14:00:00,4
$ ${TOMORROW}T15:00:00,4
$ ${TOMORROW}T16:00:00,5.5
$ ${TOMORROW}T17:00:00,8
$ ${TOMORROW}T18:00:00,12
$ ${TOMORROW}T19:00:00,13
$ ${TOMORROW}T20:00:00,14
$ ${TOMORROW}T21:00:00,12.5
$ ${TOMORROW}T22:00:00,10
$ ${TOMORROW}T23:00:00,7" > prices-tomorrow.csv

This is time series data, in FlexMeasures we call “beliefs”. Beliefs can also be sent to FlexMeasures via API or imported from open data hubs like ENTSO-E or OpenWeatherMap. However, in this tutorial we’ll show how you can read data in from a CSV file. Sometimes that’s just what you need :)

$ flexmeasures add beliefs --sensor 1 --source toy-user prices-tomorrow.csv --timezone Europe/Amsterdam
Successfully created beliefs

In FlexMeasures, all beliefs have a data source. Here, we use the username of the user we created earlier. We could also pass a user ID, or the name of a new data source we want to use for CLI scripts.

Note

Attention: We created and imported prices where the times have no time zone component! That happens a lot. FlexMeasures can localize them for you to a given timezone. Here, we localized the data to the timezone of the price sensor - Europe/Amsterdam - so the start time for the first price is 2022-03-03 00:00:00+01:00 (midnight in Amsterdam).

Let’s look at the price data we just loaded:

$ flexmeasures show beliefs --sensor 1 --start ${TOMORROW}T00:00:00+01:00 --duration PT24H

Beliefs for Sensor 'day-ahead prices' (ID 1).
Data spans a day and starts at 2022-03-03 00:00:00+01:00.
The time resolution (x-axis) is an hour.
┌────────────────────────────────────────────────────────────┐
│       ▗▀▚▖                                                 │
│      ▗▘  ▝▖                                                │
│      ▞    ▌                                                │
│     ▟     ▐                                                │ 15EUR/MWh
│    ▗▘     ▝▖                                      ▗        │
│   ▗▘       ▚                                    ▄▞▘▚▖      │
│   ▞        ▐                                  ▄▀▘   ▝▄     │
│ ▄▞          ▌                                ▛        ▖    │
│▀            ▚                               ▐         ▝▖   │
│             ▝▚            ▖                ▗▘          ▝▖  │ 10EUR/MWh
│               ▀▄▄▞▀▄▄   ▗▀▝▖               ▞            ▐  │
│                      ▀▀▜▘  ▝▚             ▗▘             ▚ │
│                              ▌            ▞               ▌│
│                              ▝▖          ▞                ▝│
│                               ▐         ▞                  │
│                                ▚      ▗▞                   │ 5EUR/MWh
│                                 ▀▚▄▄▄▄▘                    │
└────────────────────────────────────────────────────────────┘
           5            10            15           20
                     ██ day-ahead prices

Again, we can also view these prices in the FlexMeasures UI:

https://github.com/FlexMeasures/screenshots/raw/main/tut/toy-schedule/sensor-data-prices.png

Note

Technically, these prices for tomorrow may be forecasts (depending on whether you are running through this tutorial before or after the day-ahead market’s gate closure). You can also use FlexMeasures to compute forecasts yourself. See Forecasting & scheduling.