flexmeasures.data.services.utils

Functions

flexmeasures.data.services.utils.get_asset_or_sensor_from_ref(asset_or_sensor: dict)

Fetch Asset or Sensor object described by the asset_or_sensor dictionary. This dictionary needs to contain the class name and row id.

We currently cannot simplify this by just passing around the object instead of the class name: i.e. the function arguments need to be serializable as job parameters.

Examples:

>> get_asset_or_sensor({“class” : “Asset”, “id” : 1})

Asset(id=1)

>> get_asset_or_sensor({“class” : “Sensor”, “id” : 2})

Sensor(id=2)

flexmeasures.data.services.utils.get_asset_or_sensor_ref(asset_or_sensor: Asset | Sensor) → dict
flexmeasures.data.services.utils.get_or_create_model(model_class: Type[GenericAsset | GenericAssetType | Sensor], **kwargs) → GenericAsset | GenericAssetType | Sensor

Get a model from the database or add it if it’s missing.

For example: >>> weather_station_type = get_or_create_model( >>> GenericAssetType, >>> name=”weather station”, >>> description=”A weather station with various sensors.”, >>> )

flexmeasures.data.services.utils.get_scheduler_instance(scheduler_class: Type[Scheduler], asset_or_sensor: Asset | Sensor, scheduler_params) → Scheduler

Get an instance of a Scheduler adapting for the previous Scheduler signature, where a sensor is passed, to the new one where the asset_or_sensor is introduced.

flexmeasures.data.services.utils.hash_function_arguments(args, kwags)

Combines the hashes of the args and kargs

The way to go to do h(x,y) = hash(hash(x) || hash(y)) because it avoid the following:

  1. h(x,y) = hash(x || y), might create a collision if we delete the last n characters of x and we append them in front of y. e.g h(“abc”, “d”) = h(“ab”, “cd”)

  2. we don’t want to sort x and y, because we need the function h(x,y) != h(y,x)

  3. extra hashing just avoid that we can’t decompose the input arguments and track if the same args or kwarg are called several times. More of a security measure I think.

source: https://crypto.stackexchange.com/questions/55162/best-way-to-hash-two-values-into-one

flexmeasures.data.services.utils.job_cache(queue: str)

To avoid recomputing the same task multiple times, this decorator checks if the function has already been called with the same arguments. Input arguments are hashed and stored as Redis keys with the values being the job IDs input_arguments_hash:job_id).

The benefits of using redis to store the input arguments over a local cache, such as LRU Cache, are: 1) It will work in distributed environments (in computing clusters), where multiple workers will avoid repeating

work as the cache will be shared across them.

  1. Cached calls are logged, which means that we can easily debug.

  2. Cache will still be there on restarts.

Arguments :param queue: name of the queue

flexmeasures.data.services.utils.make_hash_sha256(o)

SHA256 instead of Python’s hash function because apparently, python native hashing function yields different results on restarts. Source: https://stackoverflow.com/a/42151923

flexmeasures.data.services.utils.make_hashable(o)

Function to create hashes for dictionaries with nested objects Source: https://stackoverflow.com/a/42151923