Detailed documentation of all internal modules

Todo

This is not yet complete, see issue #52

app

flexmeasures.app.create(env: Optional[str] = None, path_to_config: Optional[str] = None)flask.app.Flask

Create a Flask app and configure it.

Set the environment by setting FLASK_ENV as environment variable (also possible in .env). Or, overwrite any FLASK_ENV setting by passing an env in directly (useful for testing for instance).

A path to a config file can be passed in (otherwise a config file will be searched in the home or instance directories)

data.models

exception flexmeasures.data.models.ModelException

utils

Utilities for the FlexMeasures project.

utils.calculations

Calculations

flexmeasures.utils.calculations.integrate_time_series(s: pandas.core.series.Series, s0: float, decimal_precision: Optional[int] = None)pandas.core.series.Series

Integrate time series of length n and closed=”left” (representing a flow) to a time series of length n+1 and closed=”both” (representing a stock), given a starting stock s0. The unit of time is hours: i.e. the stock unit is flow unit times hours (e.g. a flow in kW becomes a stock in kWh). Optionally, set a decimal precision to round off the results (useful for tests failing over machine precision).

>>> s = pd.Series([1, 2, 3, 4], index=pd.date_range(datetime(2001, 1, 1, 5), datetime(2001, 1, 1, 6), freq=timedelta(minutes=15), closed="left"))
>>> integrate_time_series(s, 10)
    2001-01-01 05:00:00    10.00
    2001-01-01 05:15:00    10.25
    2001-01-01 05:30:00    10.75
    2001-01-01 05:45:00    11.50
    2001-01-01 06:00:00    12.50
    Freq: D, dtype: float64
>>> s = pd.Series([1, 2, 3, 4], index=pd.date_range(datetime(2001, 1, 1, 5), datetime(2001, 1, 1, 7), freq=timedelta(minutes=30), closed="left"))
>>> integrate_time_series(s, 10)
    2001-01-01 05:00:00    10.0
    2001-01-01 05:30:00    10.5
    2001-01-01 06:00:00    11.5
    2001-01-01 06:30:00    13.0
    2001-01-01 07:00:00    15.0
    dtype: float64

utils.time_utils

flexmeasures.utils.time_utils.as_server_time(dt: datetime.datetime)datetime.datetime

The datetime represented in the timezone of the FlexMeasures platform.

flexmeasures.utils.time_utils.decide_resolution(start: Optional[datetime.datetime], end: Optional[datetime.datetime])str

Decide on a practical resolution given the length of the selected time period. Useful for querying or plotting.

flexmeasures.utils.time_utils.ensure_local_timezone(dt: Union[pandas._libs.tslibs.timestamps.Timestamp, datetime.datetime], tz_name: str = 'Europe/Amsterdam')Union[pandas._libs.tslibs.timestamps.Timestamp, datetime.datetime]

If no timezone is given, assume the datetime is in the given timezone and make it explicit. Otherwise, if a timezone is given, convert to that timezone.

flexmeasures.utils.time_utils.forecast_horizons_for(resolution: Union[str, datetime.timedelta])Union[List[str], List[datetime.timedelta]]

Return a list of horizons that are supported per resolution. Return values or of the same type as the input.

flexmeasures.utils.time_utils.freq_label_to_human_readable_label(freq_label: str)str

Translate pandas frequency labels to human-readable labels.

flexmeasures.utils.time_utils.get_timezone(of_user=False)

Return the FlexMeasures timezone, or if desired try to return the timezone of the current user.

flexmeasures.utils.time_utils.localized_datetime(dt: datetime.datetime)datetime.datetime

Localise a datetime to the timezone of the FlexMeasures platform.

flexmeasures.utils.time_utils.localized_datetime_str(dt: datetime.datetime, dt_format: str = '%Y-%m-%d %I:%M %p')str

Localise a datetime to the timezone of the FlexMeasures platform. Hint: This can be set as a jinja filter, so we can display local time in the app, e.g.: app.jinja_env.filters[‘datetime’] = localized_datetime_filter If no datetime is passed in, use server_now() as basis.

flexmeasures.utils.time_utils.naive_utc_from(dt: datetime.datetime)datetime.datetime

Return a naive datetime, that is localised to UTC if it has a timezone.

flexmeasures.utils.time_utils.naturalized_datetime_str(dt: Optional[datetime.datetime])str

Naturalise a datetime object.

flexmeasures.utils.time_utils.resolution_to_hour_factor(resolution: Union[str, datetime.timedelta])float

Return the factor with which a value needs to be multiplied in order to get the value per hour, e.g. 10 MW at a resolution of 15min are 2.5 MWh per time step.

Parameters

resolution – timedelta or pandas offset such as “15T” or “1H”

flexmeasures.utils.time_utils.server_now()datetime.datetime

The current time of the FlexMeasures platform. UTC time, localized to the FlexMeasures timezone.

flexmeasures.utils.time_utils.tz_index_naively(data: Union[pandas.core.frame.DataFrame, pandas.core.series.Series, pandas.core.indexes.datetimes.DatetimeIndex])Union[pandas.core.frame.DataFrame, pandas.core.series.Series, pandas.core.indexes.datetimes.DatetimeIndex]

Turn any DatetimeIndex into a tz-naive one, then return. Useful for bokeh, for instance.

ui.utils.view_utils


ui.utils.plotting_utils

flexmeasures.ui.utils.plotting_utils.build_palette()Tuple[List[str], str, str]

Choose a color palette, and also single out our primary, forecasting and scheduling colors

flexmeasures.ui.utils.plotting_utils.compute_legend_height(legend)float

Adapted from: https://github.com/bokeh/bokeh/blob/master/bokehjs/src/lib/models/annotations/legend.ts

flexmeasures.ui.utils.plotting_utils.create_graph(data: pandas.core.frame.DataFrame, unit: str = 'Some unit', title: str = 'A plot', x_label: str = 'X', y_label: str = 'Y', legend_location: Union[str, Tuple[float, float]] = 'top_right', legend_labels: Tuple[str, Optional[str], Optional[str]] = ('Actual', 'Forecast', 'Schedules'), x_range: Optional[bokeh.models.ranges.Range1d] = None, forecasts: Optional[pandas.core.frame.DataFrame] = None, schedules: Optional[pandas.core.frame.DataFrame] = None, show_y_floats: bool = False, non_negative_only: bool = False, tools: Optional[List[str]] = None, sizing_mode: str = 'scale_width')bokeh.plotting.figure.Figure

Create a Bokeh graph. As of now, assumes x data is datetimes and y data is numeric. The former is not set in stone.

Parameters
  • data – the actual data. Expects column name “event_value” and optional “belief_horizon” and “source” columns.

  • unit – the (physical) unit of the data

  • title – Title of the graph

  • x_label – x axis label

  • y_label – y axis label

  • legend_location – location of the legend

  • legend_labels – labels for the legend items

  • x_range – values for x axis. If None, taken from series index.

  • forecasts – forecasts of the data. Expects column names “event_value”, “yhat_upper” and “yhat_lower”.

  • schedules – scheduled data. Expects column name “event_value”.

  • hover_tool – Bokeh hover tool, if required

  • show_y_floats – if True, y axis will show floating numbers (defaults False, will be True if y values are < 2)

  • non_negative_only – whether or not the data can only be non-negative

  • tools – some tools for the plot, which defaults to [“box_zoom”, “reset”, “save”].

Returns

a Bokeh Figure

flexmeasures.ui.utils.plotting_utils.create_hover_tool(y_unit: str, resolution: datetime.timedelta, as_beliefs: bool = False)bokeh.models.tools.HoverTool

Describe behaviour of default tooltips (we could also return html for custom tooltips)

Uses from_py_func, a deprecated function since bokeh==1.1 https://docs.bokeh.org/en/latest/docs/releases.html?highlight=from_py_func

flexmeasures.ui.utils.plotting_utils.decide_plot_resolution(data: pandas.core.frame.DataFrame)datetime.timedelta

Finding out which resolution to use: prefer resolution in data, otherwise from session (which is based on the session’s time period)

flexmeasures.ui.utils.plotting_utils.get_latest_power_as_plot(asset: flexmeasures.data.models.assets.Asset, small: bool = False)Tuple[str, str]

Create a plot of an asset’s latest power measurement as an embeddable html string (incl. javascript). First returned string is the measurement time, second string is the html string.

flexmeasures.ui.utils.plotting_utils.highlight(fig: bokeh.plotting.figure.Figure, x_start: Any, x_end: Any, color: str = '#FF3936', redirect_to: Optional[str] = None)

Add a box highlight to an area above the x axis. If a redirection URL is given, it can open the URL on double-click (this assumes datetimes are used on x axis!). It will pass the year, month, day, hour and minute as parameters to the URL.

flexmeasures.ui.utils.plotting_utils.make_datasource_from(data: pandas.core.frame.DataFrame, resolution: datetime.timedelta)bokeh.models.sources.ColumnDataSource

Make a bokeh data source, which is for instance useful for the hover tool.

flexmeasures.ui.utils.plotting_utils.make_range(index: pandas.core.indexes.datetimes.DatetimeIndex, other_index: Optional[pandas.core.indexes.datetimes.DatetimeIndex] = None)Optional[bokeh.models.ranges.Range1d]

Make a 1D range of values from a datetime index or two. Useful to share axis among Bokeh Figures.

flexmeasures.ui.utils.plotting_utils.replace_source_with_label(data: pandas.core.frame.DataFrame)pandas.core.frame.DataFrame

Column “source” is dropped, and column “label” is created. The former column should contain DataSource objects, while the latter will contain only strings.

flexmeasures.ui.utils.plotting_utils.separate_legend(fig: bokeh.plotting.figure.Figure, orientation: str = 'vertical')bokeh.plotting.figure.Figure

Cuts legend out of fig and returns a separate legend (as a Figure object). Click policy doesn’t work on the new legend. Requires bokeh==1.0.2 for solution from https://groups.google.com/a/continuum.io/forum/#!topic/bokeh/BJRhWlnmhWU Open feature request to share a legend across plots: https://github.com/bokeh/bokeh/issues/7607

ui.views

flexmeasures.ui.views.dashboard.dashboard_view()

Dashboard view. This is the default landing page for the platform user. It shows a map with the location and status of all of the user’s assets, as well as a breakdown of the asset types in the user’s portfolio. Assets for which the platform has identified upcoming balancing opportunities are highlighted.

flexmeasures.ui.views.portfolio.data_or_zeroes(df: pandas.core.frame.DataFrame, start, end, resolution)pandas.core.frame.DataFrame

Making really sure we have the structure to let the plots not fail

flexmeasures.ui.views.portfolio.get_flex_action_hour(h: int)datetime.datetime

get the next hour from now on

flexmeasures.ui.views.portfolio.portfolio_view()

Portfolio view. By default, this page shows live results (production, consumption and market data) from the user’s portfolio. Time windows for which the platform has identified upcoming balancing opportunities are highlighted. The page can also be used to navigate historical results.

flexmeasures.ui.views.portfolio.rename_event_value_column_to_resource_name(df_dict: Dict[str, pandas.core.frame.DataFrame])Dict[str, pandas.core.frame.DataFrame]

Replace the column name “event_source” with the resource name, for each resource in the dictionary.

flexmeasures.ui.views.portfolio.stack_df(df: pandas.core.frame.DataFrame)pandas.core.frame.DataFrame

Stack columns of df cumulatively, include bottom

flexmeasures.ui.views.control.control_view()

Control view. This page lists balancing opportunities for a selected time window. The user can place manual orders or choose to automate the ordering process.

flexmeasures.ui.views.analytics.analytics_data_view(content, content_type)

Analytics view as above, but here we only download data. Content can be either source or metrics. Content-type can be either CSV or JSON.

flexmeasures.ui.views.analytics.analytics_view()

Analytics view. Here, four plots (consumption/generation, weather, prices and a profit/loss calculation) and a table of metrics data are prepared. This view allows to select a resource name, from which a models.Resource object can be made. The resource name is kept in the session. Based on the resource, plots and table are labelled appropriately.

flexmeasures.ui.views.analytics.determine_resolution(data: pandas.core.frame.DataFrame, forecasts: Optional[pandas.core.frame.DataFrame] = None, schedules: Optional[pandas.core.frame.DataFrame] = None)str

Determine the resolution to be displayed under the plot. We try to get it from the DataFrame’s meta data, or guess from the actual data. Lastly, we try the session. If nothing can be found this way, the resulting string is “?”

flexmeasures.ui.views.analytics.filter_for_past_data(data)

Make sure we only show past data, useful for demo mode

flexmeasures.ui.views.analytics.filter_forecasts_for_limited_time_window(data)

Show forecasts only up to a limited horizon

flexmeasures.ui.views.analytics.get_data_and_metrics(query_window: Tuple[datetime.datetime, datetime.datetime], resolution: str, show_consumption_as_positive: bool, showing_individual_traces_for: str, selected_resource: flexmeasures.data.services.resources.Resource, selected_market, selected_sensor_type, assets)Tuple[Dict[str, pandas.core.frame.DataFrame], Dict[str, float], str, flexmeasures.data.models.weather.WeatherSensor]

Getting data and calculating metrics for them

flexmeasures.ui.views.analytics.make_power_figure(resource_display_name: str, data: pandas.core.frame.DataFrame, forecast_data: Optional[pandas.core.frame.DataFrame], schedule_data: Optional[pandas.core.frame.DataFrame], show_consumption_as_positive: bool, shared_x_range: bokeh.models.ranges.Range1d, tools: Optional[List[str]] = None, sizing_mode='scale_width')bokeh.plotting.figure.Figure

Make a bokeh figure for power consumption or generation

flexmeasures.ui.views.analytics.make_prices_figure(data: pandas.core.frame.DataFrame, forecast_data: Union[None, pandas.core.frame.DataFrame], shared_x_range: bokeh.models.ranges.Range1d, selected_market: flexmeasures.data.models.markets.Market, tools: Optional[List[str]] = None, sizing_mode='scale_width')bokeh.plotting.figure.Figure

Make a bokeh figure for price data

flexmeasures.ui.views.analytics.make_revenues_costs_figure(resource_display_name: str, data: pandas.core.frame.DataFrame, forecast_data: pandas.core.frame.DataFrame, show_consumption_as_positive: bool, shared_x_range: bokeh.models.ranges.Range1d, selected_market: flexmeasures.data.models.markets.Market, tools: Optional[List[str]] = None, sizing_mode='scale_width')bokeh.plotting.figure.Figure

Make a bokeh figure for revenues / costs data

flexmeasures.ui.views.analytics.make_weather_figure(selected_resource: flexmeasures.data.services.resources.Resource, data: pandas.core.frame.DataFrame, forecast_data: Union[None, pandas.core.frame.DataFrame], shared_x_range: bokeh.models.ranges.Range1d, weather_sensor: flexmeasures.data.models.weather.WeatherSensor, tools: Optional[List[str]] = None, sizing_mode='scale_width')bokeh.plotting.figure.Figure

Make a bokeh figure for weather data