flexmeasures.data.schemas.sensors

Functions

flexmeasures.data.schemas.sensors.floor_bdf_event_starts(bdf: BeliefsDataFrame, event_resolution: timedelta) → BeliefsDataFrame

Classes

class flexmeasures.data.schemas.sensors.QuantityOrSensor(*args, **kwargs)

__init__(*args, **kwargs): Deprecated class. Use VariableQuantityField instead.

Represents a quantity string like ‘1 EUR/MWh’.

fields: dict[str, Field]: Dictionary mapping field_names -> Field objects

class flexmeasures.data.schemas.sensors.RepurposeValidatorToIgnoreSensorsAndLists(original_validator, *, error: str | None = None)

Validator that executes another validator (the one you initialize it with) only on non-Sensor and non-list values.

__init__(original_validator, *, error: str | None = None)

Schema for uploading a file with sensor data. This one describes only the file upload part, not the sensor itself. See SensorDataFileSchema for the full schema.

fields: dict[str, Field]: Dictionary mapping field_names -> Field objects

class flexmeasures.data.schemas.sensors.SensorDataFileRequestSchema(*args, source_user: User | None = None, **kwargs)

Validate a sensor data upload without parsing or resampling its files.

fields: dict[str, Field]: Dictionary mapping field_names -> Field objects

post_load(fields, **kwargs): Process the deserialized and validated fields. Remove the ‘sensor’ and ‘files’ fields, and add the ‘data’ field containing a list of BeliefsDataFrames.

class flexmeasures.data.schemas.sensors.SensorDataFileSchema(*args, source_user: User | None = None, **kwargs)

__init__(*args, source_user: User | None = None, **kwargs)

fields: dict[str, Field]: Dictionary mapping field_names -> Field objects

post_load(fields, **kwargs): Process the deserialized and validated fields. Remove the ‘sensor’ and ‘files’ fields, and add the ‘data’ field containing a list of BeliefsDataFrames.

validate_unit(data, **kwargs): Validate unit compatibility with the sensor’s unit.

validate_uploaded_files(files: list[FileStorage], **kwargs): Validate the deserialized fields.

class flexmeasures.data.schemas.sensors.SensorIdField(asset: GenericAsset | None = None, unit: str | Quantity | None = None, *args, **kwargs)

Field that deserializes to a Sensor and serializes back to an integer.

__init__(asset: GenericAsset | None = None, unit: str | Quantity | None = None, *args, **kwargs)

_deserialize(value: Any, attr, data, **kwargs) → Sensor: Turn a sensor id into a Sensor.

_serialize(value: Sensor, attr, obj, **kwargs) → int: Turn a Sensor into a sensor id.

class flexmeasures.data.schemas.sensors.SensorReference(sensor: Sensor, source_types: list[str] | None = None, exclude_source_types: list[str] | None = None, sources: list[DataSource] | None = None, source_account: list[Account] | None = None)

A sensor reference that wraps a Sensor with optional source filters for belief queries.

Exposes the same unit, id, and event_resolution properties as a plain Sensor, so code that reads those properties works without modification. The source filters are passed through to TimedBelief.search in get_series_from_quantity_or_sensor().

__init__(sensor: Sensor, source_types: list[str] | None = None, exclude_source_types: list[str] | None = None, sources: list[DataSource] | None = None, source_account: list[Account] | None = None) → None

property event_resolution: timedelta: Event resolution of the underlying sensor.

property id: int: ID of the underlying sensor.

property unit: str: Unit of the underlying sensor.

Sensor reference with optional source filters.

class Meta

fields: dict[str, Field]: Dictionary mapping field_names -> Field objects

class flexmeasures.data.schemas.sensors.SensorSchema(*args, **kwargs)

Sensor schema with validations.

class Meta

model: alias of Sensor

opts: LoadInstanceMixin.Opts = <flask_marshmallow.sqla.SQLAlchemySchemaOpts object>

Base sensor schema.

Here we include all fields which are implemented by timely_beliefs.SensorDBMixin All classes inheriting from timely beliefs sensor don’t need to repeat these. In a while, this schema can represent our unified Sensor class.

When subclassing, also subclass from ma.SQLAlchemySchema and add your own DB model class, e.g.:

class Meta:
model = Asset

fields: dict[str, Field]: Dictionary mapping field_names -> Field objects

set_default_timezone(data, **kwargs): Set the default timezone to the server timezone only for a full load (POST, not PATCH).

timezone_validator(): Validate timezone, suggesting the closest match if possible or the server default otherwise.

class flexmeasures.data.schemas.sensors.TimeSeriesOrSensor(*args, **kwargs)

__init__(*args, **kwargs): Deprecated class. Use VariableQuantityField instead.

List of time series segments.

fields: dict[str, Field]: Dictionary mapping field_names -> Field objects

class flexmeasures.data.schemas.sensors.TimedEventSchema(timezone: str | None = None, event_resolution: timedelta | None = None, value_validator: Validator | None = None, to_unit: str | None = None, default_src_unit: str | None = None, return_magnitude: bool = True, *args, **kwargs)

A time period (or single point) with a value.

Parameters:: timezone – Optionally, set a timezone to be able to interpret nominal durations.

check_time_window(data, **kwargs)

Checks whether a complete time interval can be derived from the timing fields.

The data is updated in-place, guaranteeing that the ‘start’ and ‘end’ fields are filled out.

fields: dict[str, Field]: Dictionary mapping field_names -> Field objects

class flexmeasures.data.schemas.sensors.VariableQuantityField(to_unit, *args, default_src_unit: str | None = None, return_magnitude: bool = False, timezone: str | None = None, event_resolution: timedelta | None = None, value_validator: Validator | None = None, additional_sensor_units: list[str] | None = None, **kwargs)

__init__(to_unit, *args, default_src_unit: str | None = None, return_magnitude: bool = False, timezone: str | None = None, event_resolution: timedelta | None = None, value_validator: Validator | None = None, additional_sensor_units: list[str] | None = None, **kwargs)

Field for validating, serializing and deserializing a variable quantity.

A variable quantity can be represented by a sensor, time series or fixed quantity.

# todo: Sensor should perhaps deserialize already to sensor data

NB any value validators passed are only applied to Quantities. For example, value_validator=validate.Range(min=0) will raise a ValidationError in case of negative quantities, but will let pass any sensor that has recorded negative values.

Parameters:

to_unit –
Unit to which the sensor, time series or quantity should be convertible. - Sensors are checked for convertibility, but the original sensor is returned,

so its values are not yet converted.
- Time series and quantities are already converted to the given unit.
- Units starting with ‘/’ (e.g. ‘/MWh’) lead to accepting any value, which will be converted to the given unit. For example, a quantity of 1 EUR/kWh with to_unit=’/MWh’ is deserialized to 1000 EUR/MWh.
default_src_unit – What unit to use in case of getting a numeric value. Does not apply to time series or sensors. In case to_unit is dimensionless, default_src_unit defaults to dimensionless; as a result, numeric values are accepted.
return_magnitude – In case of getting a time series, whether the result should include the magnitude of each quantity, or each Quantity object itself.
timezone – Only used in case a time series is specified and one of the timed events in the time series uses a nominal duration, such as “P1D”.
additional_sensor_units – Additional sensor units (besides those convertible to to_unit) that are accepted for sensor references. Use this only for units that are dimensionally incompatible with to_unit but contextually meaningful — for example, ["%"] allows sensors with a percentage unit for fields where the conversion requires an external capacity factor (such as soc-max). The actual unit conversion must be handled downstream by the caller. Do not use this as a general-purpose unit allowlist.

_deserialize(value: dict[str, int] | list[dict] | str, attr, data, **kwargs) → Sensor | list[dict] | Quantity

Deserialize value. Concrete Field classes should implement this method.

Parameters:

value – The value to be deserialized.
attr – The attribute/key in data to be deserialized.
data – The raw input data passed to the Schema.load <marshmallow.Schema.load>.
kwargs – Field-specific keyword arguments.

Raises:

ValidationError – In case of formatting or validation failure.

Returns:

The deserialized value.

Changed in version 3.0.0: Added **kwargs to signature.

_deserialize_dict(value: dict[str, Any]) → Sensor | SensorReference

Deserialize a sensor reference to a Sensor or SensorReference.

Returns a plain Sensor when no source filter keys are present (backward compatible), and a SensorReference when any of source-types, exclude-source-types, sources, or source-account are provided.

_deserialize_list(value: list[dict]) → list[dict]: Deserialize a time series to a list of timed events.

_deserialize_numeric(value: Real, attr, data, **kwargs) → Quantity: Try to deserialize a numeric value to a Quantity, using the default_src_unit.

_deserialize_source_filters(value: dict[str, Any]) → tuple[list[str] | None, list[str] | None, list[DataSource] | None, list[Account] | None]

Deserialize and validate source filter fields from a sensor-reference dict.

Returns (source_types, exclude_source_types, sources, source_account).

_deserialize_str(value: str) → Quantity: Deserialize a string to a Quantity.

_get_original_unit(serialized_variable_quantity: str | list[dict] | dict, deserialized_variable_quantity: Quantity | list[dict] | Sensor | SensorReference) → str: Obtain the original unit from the still serialized variable quantity.

_get_unit(variable_quantity: Quantity | list[dict] | Sensor | SensorReference) → str

Obtain the unit from the (deserialized) variable quantity.

>>> VariableQuantityField("MW")._get_unit(ur.Quantity("3 kWh"))
'kWh'
>>> VariableQuantityField("/MW")._get_unit([{'value': ur.Quantity("3 kEUR/MWh")}, {'value': ur.Quantity("0 EUR/kWh")}])
'kEUR/MWh'

_serialize(value: Sensor | SensorReference | Series | Quantity, attr, obj, **kwargs) → str | dict[str, Any]

Serializes value to a basic Python datatype. Noop by default. Concrete Field classes should implement this method.

Example:

class TitleCase(Field):
    def _serialize(self, value, attr, obj, **kwargs):
        if not value:
            return ""
        return str(value).title()

Parameters:

value – The value to be serialized.
attr – The attribute or key on the object to be serialized.
obj – The object the value was pulled from.
kwargs – Field-specific keyword arguments.

Returns:

The serialized value

get_obj_type(obj): Returns name of the schema during dump() calls, given the object being dumped.