flexmeasures.data.utils
Utils around the data models and db sessions
Functions
- flexmeasures.data.utils.get_data_source(data_source_name: str, data_source_model: str | None = None, data_source_version: str | None = None, data_source_type: str = 'script') DataSource
Make sure we have a data source. Create one if it doesn’t exist, and add to session. Meant for scripts that may run for the first time.
- flexmeasures.data.utils.save_to_db(data: BeliefsDataFrame | BeliefsSeries | list[BeliefsDataFrame | BeliefsSeries], bulk_save_objects: bool = False, save_changed_beliefs_only: bool = True) str
Save the timed beliefs to the database.
Note: This function does not commit. It does, however, flush the session. Best to keep transactions short.
We make the distinction between updating beliefs and replacing beliefs.
# Updating beliefs
An updated belief is a belief from the same source as some already saved belief, and about the same event, but with a later belief time. If it has a different event value, then it represents a changed belief. Note that it is possible to explicitly record unchanged beliefs (i.e. updated beliefs with a later belief time, but with the same event value), by setting save_changed_beliefs_only to False.
# Replacing beliefs
A replaced belief is a belief from the same source as some already saved belief, and about the same event and with the same belief time, but with a different event value. Replacing beliefs is not allowed, because messing with the history corrupts data lineage. Corrections should instead be recorded as updated beliefs. Servers in ‘play’ mode are exempt from this rule, to facilitate replaying simulations.
- Parameters:
data – BeliefsDataFrame (or a list thereof) to be saved
bulk_save_objects – if True, objects are bulk saved with session.bulk_save_objects(), which is quite fast but has several caveats, see: https://docs.sqlalchemy.org/orm/persistence_techniques.html#bulk-operations-caveats
save_changed_beliefs_only – if True, unchanged beliefs are skipped (updated beliefs are only stored if they represent changed beliefs) if False, all updated beliefs are stored
- Returns:
status string, one of the following: - ‘success’: all beliefs were saved - ‘success_with_unchanged_beliefs_skipped’: not all beliefs represented a state change - ‘success_but_nothing_new’: no beliefs represented a state change