# Batcontrol Documentation > Optimize your electricity costs by re-charging your PV battery when electricity is cheap and there is not enough solar power available. Batcontrol controls a PV battery inverter based on dynamic electricity prices, solar production forecasts, and consumption patterns. It charges from the grid when prices are low and preserves stored energy when prices are high. It supports Fronius Gen24 inverters (HTTP API and Modbus TCP) and any inverter via MQTT bridge. Tariff providers include Tibber, aWATTar, evcc, EnergyForecast.de, and static zone tariffs. Solar forecast sources include Forecast.Solar, Solar-Prognose.de, evcc, and Home Assistant Solar Forecast ML. Peak shaving distributes PV battery charging across the day using forecasts, and works with both dynamic and static tariffs. # Getting Started # Installation Batcontrol can be run as a Docker container, a Home Assistant add-on, or directly from a local Python environment. ## What You Need Collect the following before you start configuring. ### Inverter access - Local IP address or hostname of the inverter on your network - Customer or technician login credentials ### Electricity tariff (one of the following) - **Dynamic provider** (Tibber, aWATTar, evcc, EnergyForecast.de): API key if required by the provider - **Static tariff**: your electricity price in €/kWh — sufficient for peak shaving without dynamic charging ### Solar forecast (one of the following) - **Forecast.Solar** (default, free, no account required): GPS coordinates of your PV installation, roof azimuth (−90 = East, 0 = South, 90 = West), tilt angle in degrees (0 = horizontal, 90 = vertical), and system size in kWp - **Solar-Prognose.de**: API key - **evcc**: a running evcc instance with solar forecast data - **Home Assistant Solar Forecast ML**: a running Home Assistant instance with the Solar Forecast ML integration and the entity ID of the forecast sensor ### Consumption forecast Your annual electricity consumption in kWh. The default load profile covers typical household patterns, so this single number is enough to get started. ## Configuration File Docker (plain or Compose) and local Python installs use a `batcontrol_config.yaml` in the `config/` directory. If none is present when the Docker container starts for the first time, batcontrol copies a sample configuration that uses a dummy inverter. Home Assistant add-on configuration is done via the add-on UI (no local YAML file). Once you are ready, edit `config/batcontrol_config.yaml` and set your inverter type, credentials, tariff provider, and PV details. See [Main Configuration](https://mastr.github.io/batcontrol/configuration/batcontrol-configuration/index.md) and the other configuration pages for a full reference. ## Docker This is the recommended way to run batcontrol for most users. ### Setup ``` mkdir -p ./config ./logs ``` ### Plain Docker ``` docker run -d \ --name batcontrol \ -v /path/to/config:/app/config \ -v /path/to/logs:/app/logs \ mastr950/batcontrol:latest ``` ### Docker Compose Create a `docker-compose.yml`: ``` services: batcontrol: image: mastr950/batcontrol:latest volumes: - ./config:/app/config - ./logs:/app/logs restart: unless-stopped ``` Then start the container: ``` docker compose up -d ``` ### Timezone By default the container uses UTC. Set the `TZ` environment variable to match your local time zone, which affects log timestamps and time-based scheduling. Plain Docker: ``` docker run -d \ --name batcontrol \ -v /path/to/config:/app/config \ -v /path/to/logs:/app/logs \ -e TZ=Europe/Berlin \ mastr950/batcontrol:latest ``` Docker Compose: ``` services: batcontrol: image: mastr950/batcontrol:latest volumes: - ./config:/app/config - ./logs:/app/logs environment: - TZ=Europe/Berlin restart: unless-stopped ``` ## Home Assistant Add-on Add the [batcontrol_ha_addon](https://github.com/MaStr/batcontrol_ha_addon) repository to your Home Assistant add-on store. Configuration is done through the add-on UI rather than a YAML file. ## Local Python Requires Python 3.13 and [uv](https://docs.astral.sh/uv/). ``` git clone https://github.com/MaStr/batcontrol.git cd batcontrol uv venv --python 3.13 --allow-existing source .venv/bin/activate uv pip install . ``` Place your `batcontrol_config.yaml` in the `config/` directory, then run: ``` python -m batcontrol ``` ## Inverter Preparation Before the first run, verify the following: - Confirm your inverter login credentials (customer or technician access both work). - If you have previously run any third-party tools that use Modbus, or executed Modbus commands directly, disable Modbus and restart the inverter before starting batcontrol. Batcontrol will enable the Solar.API on first run (local network only) and save the current battery configuration, which it restores on shutdown. ## Next Steps After the first successful start, work through this in order: 1. **Check the logs.** Confirm batcontrol connects to your inverter and enters the control loop without errors. The log shows the current mode, the next decision, and the price/forecast values used. 1. **Switch to your real inverter.** Edit `config/batcontrol_config.yaml`, change `inverter.type` from `dummy` to `fronius_gen24` (or `fronius-modbus`), and add your inverter address and credentials. 1. **Set your annual consumption.** Update `consumption_forecast.csv.annual_consumption` with your actual value in kWh. This scales the default load profile to your household. 1. **Configure your tariff provider.** If you use a dynamic provider (Tibber, aWATTar, etc.), set the correct type and API key. For a flat rate, use `tariff_zones` with a single zone price — batcontrol will still run peak shaving. 1. **Review battery charge limits.** The defaults (`max_charging_from_grid_limit: 89%`, `always_allow_discharge_limit: 90%`) are conservative starting values. Adjust them to match your usage patterns after a few days of observation. 1. **Enable peak shaving** (optional). If your PV installation frequently reaches full battery charge before midday, enable peak shaving to spread the charging and keep buffer capacity for the afternoon. Requires `battery_control.type: next` and `peak_shaving.enabled: true`. See [Peak Shaving](https://mastr.github.io/batcontrol/features/peak-shaving/index.md). 1. **Connect to Home Assistant** (optional). Enable the MQTT API to get real-time state, price, and forecast data as Home Assistant entities, and to override battery limits at runtime. See [MQTT API](https://mastr.github.io/batcontrol/integrations/mqtt-api/index.md). 1. **Connect evcc** (optional). If you charge an electric vehicle, the evcc integration can hold battery discharge while the car is charging. See [evcc Connection](https://mastr.github.io/batcontrol/integrations/evcc-connection/index.md). ## Uninstalling Batcontrol saves your inverter's original battery settings and restores them on a clean shutdown. To uninstall safely: 1. If batcontrol is currently running, wait until it finishes a cycle and goes to sleep, then shut it down. This ensures the saved inverter settings are restored. 1. If batcontrol is not running and you suspect it crashed during a run, restart it, let it complete one cycle and sleep, then shut it down cleanly. 1. Verify that the battery control schedule in your inverter's local web UI looks correct after shutdown. 1. Remove the Docker container or the local installation directory. 1. Disable the Solar API in your inverter's web UI if no other software requires it. Note: the Fronius Wattpilot wallbox requires the Solar API to be enabled. ## Development Setup For working on the batcontrol source code, use an editable install: ``` git clone https://github.com/MaStr/batcontrol.git cd batcontrol uv venv --python 3.13 --allow-existing source .venv/bin/activate uv pip install --editable '.[test]' ``` Run the test suite: ``` ./run_tests.sh # or python -m pytest tests/ # with coverage: python -m pytest tests/ --cov=src/batcontrol ``` # How Batcontrol Works Batcontrol is an intelligent battery management system that optimizes your home energy storage by predicting energy prices, solar production, and consumption patterns. It automatically controls your battery inverter to minimize electricity costs. ## Core Principle The fundamental idea is simple: **charge your battery when electricity is cheap and use stored energy when electricity is expensive**. However, the implementation requires sophisticated forecasting and decision-making logic. ## System Architecture ### Main Components 1. **Forecasting Engines**: Gather predictions for prices, solar production, and consumption 1. **Logic Engine**: Calculates optimal battery control decisions 1. **Inverter Interface**: Controls your battery inverter 1. **External APIs**: Integrates with MQTT, evcc, and Home Assistant ### Operation Cycle Batcontrol runs in a continuous loop, evaluating conditions **every 3 minutes**: ``` 1. Fetch Forecasts → 2. Calculate Optimal Strategy → 3. Control Inverter → 4. Wait 3 Minutes → Repeat ``` ## Forecasting System Batcontrol combines three critical forecasts to make intelligent decisions: ### 1. Electricity Price Forecast - **Source**: Dynamic tariff providers (Tibber, aWATTar, evcc, energyforecast.de, static tariff zones) — see [Dynamic Tariff Provider](https://mastr.github.io/batcontrol/configuration/dynamic-tariff-provider/index.md) - **Data**: Hourly electricity prices for the next 24-48 hours - **Purpose**: Identifies when electricity is cheapest for charging ### 2. Solar Production Forecast - **Source**: Solar forecast APIs (Forecast.Solar, Solarprognose, evcc, HomeAssistant Solar Forecast ML) — see [Solar Forecast](https://mastr.github.io/batcontrol/configuration/solar-forecast/index.md) - **Data**: Expected PV generation based on weather predictions - **Configuration**: Your PV system specifications (kWp, orientation, location) - **Purpose**: Predicts available solar energy ### 3. Consumption Forecast - **Source**: Load profile CSV file scaled to your annual consumption, or your actual historical data via the HomeAssistant API — see [Consumption Forecast](https://mastr.github.io/batcontrol/configuration/consumption-forecast/index.md) - **Data**: Expected household energy usage patterns - **Purpose**: Estimates energy demand throughout the day ### Net Consumption Calculation The key metric is **Net Consumption = Consumption - Solar Production**: - **Positive values**: You need energy from grid/battery - **Negative values**: You have surplus solar energy ## Decision Logic Based on the forecasts and current battery state, batcontrol puts your inverter into one of four modes: ### Mode 10: DISCHARGE ALLOWED (Normal Operation) - **When**: Energy is currently expensive OR battery is sufficiently charged - **Behavior**: - Battery can discharge to meet household demand - Surplus solar energy charges battery (respects `max_pv_charge_rate`) - Excess energy feeds into grid - **Always Active**: When SOC > `always_allow_discharge_limit` (typically 90%) ### Mode 8: LIMIT BATTERY CHARGE RATE (Peak Shaving) - **Requires**: version 0.8.0+ , Logic type `next` must be selected in `battery_control.type` - **When**: Peak shaving is enabled, PV is producing, and the battery should not fill up too quickly - **Behavior**: - Battery **discharge is allowed** (handles household demand normally) - PV charging is **rate-limited** so the battery fills gradually instead of hitting 100% early in the day - Excess PV energy that cannot be stored at the limited rate feeds into the grid - **Purpose**: Preserve free battery capacity so the battery can absorb maximal solar energy later in the day, avoiding unnecessary grid feed-in during midday PV peaks ### Mode 0: AVOID DISCHARGE (Energy Saving) - **When**: Prices are rising and stored energy will be more valuable later - **Behavior**: - Battery does not discharge - Grid provides additional energy needed - Direct solar consumption continues normally - **Purpose**: Preserve battery energy for expensive periods ### Mode -1: CHARGE FROM GRID (Active Charging) - **When**: Current prices are significantly lower than future prices - **Behavior**: - Battery charges from grid at configured rate (`max_grid_charge_rate`) - Charging stops at `max_charging_from_grid_limit` (typically 89%) - **Calculation**: Estimates required energy for upcoming expensive hours - **Efficiency**: Accounts for 10-20% charge/discharge losses ## Price-Based Decision Making ### Key Parameters **`min_price_difference`** (absolute): Minimum price difference in Euro to justify grid charging - Example: 0.05 means current price must be ≥5 cents lower than future price **`min_price_difference_rel`** (relative): Percentage-based price difference threshold - Example: 0.10 means current price must be ≥10% lower than future price - Helps avoid inefficient charging during high-price periods **Final threshold**: `max(min_price_difference, current_price × min_price_difference_rel)` ### Advanced Price Logic **Expert Mode** offers additional refinements: - **Price Rounding**: Configurable precision for price comparisons - **Softened Charging**: Earlier charging with relaxed price requirements - **Charge Rate Multiplier**: Compensates for charging inefficiencies ## Battery Management ### SOC (State of Charge) Limits **`always_allow_discharge_limit`** (default: 90%) - Above this SOC, battery always discharges freely - Prevents over-charging and ensures availability **`max_charging_from_grid_limit`** (default: 80%) - Maximum SOC for grid charging - Must be lower than discharge limit to prevent oscillation **`min_recharge_amount`** (default: 100Wh) (since 0.5.3) - Minimum amount of energy that is needed to be recharged before starting to recharge. - That prevents ditching between discharge + recharge on an increasing price situation. ### Safety Constraints The system validates configuration to prevent problematic behavior: - `always_allow_discharge_limit` must be > `max_charging_from_grid_limit` - If violated, `max_charging_from_grid_limit` is automatically lowered ## External Integrations ### evcc Integration - **Purpose**: Coordinate with electric vehicle charging - **Function**: Can lock battery discharge when car is charging - **Configuration**: Monitors charging status and adjusts battery limits ### MQTT/Home Assistant - **Real-time Data**: Battery state, prices, forecasts - **Remote Control**: Override modes and parameters - **Auto-Discovery**: Automatic Home Assistant entity creation ## Error Handling ### Forecast Failures - **Timeout**: If APIs fail for >10 minutes, defaults to discharge mode - **Fallback Strategy**: Ensures battery remains usable during outages - **Recovery**: Automatically resumes normal operation when APIs return ### Configuration Validation - **Runtime Checks**: Validates parameter relationships - **Automatic Corrections**: Adjusts conflicting settings - **Comprehensive Logging**: Detailed operation logs for troubleshooting ## Configuration Flow 1. **Hardware Setup**: Configure inverter connection and credentials 1. **Location Setup**: PV system specifications and geographic location 1. **Consumption Profile**: Annual usage and load pattern 1. **Price Provider**: Dynamic tariff API configuration 1. **Battery Limits**: Discharge and charging thresholds 1. **Fine-Tuning**: Expert parameters and external integrations ## Battery Configuration Visualization This diagram illustrates the relationship between the key battery SOC limits and how they control charging/discharging behavior. # Configuration This is the main control logic configuration: ``` timezone: Europe/Berlin #your time zone. not optional. time_resolution_minutes: 60 # Time resolution for forecasts: 15 (quarter-hourly) or 60 (hourly). Default: 60 loglevel: debug logfile_enabled: true log_everything: false # if false debug messages from fronius.auth and urllib3.connectionpool will be suppressed max_logfile_size: 200 #kB logfile_path: logs/batcontrol.log ``` ## Time Resolution (with 0.6.0) ``` time_resolution_minutes: 60 ``` This parameter controls the time resolution for all forecasts (solar production, consumption, and electricity prices). Valid values are: * **60** (default) - Hourly intervals, backward compatible, lower memory usage * **15** - Quarter-hourly intervals, higher accuracy for dynamic tariffs, 4x more data points **Recommendation**: Use **15 minutes** if your dynamic tariff provider offers quarter-hourly prices (e.g., some Tibber or energyforecast.de plans). Use **60 minutes** for standard hourly tariffs or if you want to minimize resource usage. **Technical Details**: - 15-min mode: 192 intervals per 48 hours (~8 KB per forecast) - 60-min mode: 48 intervals per 48 hours (~2 KB per forecast) - All forecast providers automatically adapt to the configured resolution - MQTT topics publish data at the configured interval ## Timezone This parameter is used to calculate the correct time for your location, as some datasources deliver UTC based timeslots. Valid values are [tz based(wikipedia)](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). ## Logfile ### Logpath ``` logfile_path: logs/batcontrol.log ``` Describes where logfiles are stored. The path can be relative or absolute. ### Log Level ``` loglevel: debug ``` Increases or decreases the verbosity of log messages. Valid entries are - error - warning - info - debug The recommended settings are `info` and `debug`. To reduce the noise in the default setup, we introduced ``` log_everything: false ``` Setting this to `true`, the logmessage from Fronius authentication logic + HTTP-Requests are visible in the logfile. These are very verbose messages, which is the reason to only enable it for debugging purposes. ### Enable / Disable logfile ``` logfile_enabled: true ``` This parameter is used to enable a pyhsical logfile. Console out is still active if this value is set to `false`. This can be useful in docker-based environments. ### Logsize ``` max_logfile_size: 200 #Kb ``` Amount of logsize bevore a logswitch is applied. The logs switches from log.1 to log.2 and back. Each file will be the size of `max_logfile_size`. This is used to avoid a filling up disk. ## Batcontrol alogrithm configuration ``` battery_control: min_price_difference: 0.05 min_price_difference_rel: 0.10 always_allow_discharge_limit: 0.90 max_charging_from_grid_limit: 0.89 min_grid_charge_soc: 0.55 # optional: grid-charge to this SoC before expensive slots min_recharge_amount: 100 ``` Details about the Price configuration can be found on [price difference calculation](https://mastr.github.io/batcontrol/features/price-difference-calculation/index.md) page. `always_allow_discharge_limit` & `max_charging_from_grid_limit` is explained [here](https://mastr.github.io/batcontrol/getting-started/how-batcontrol-works/index.md). `min_grid_charge_soc` is optional. When set as a ratio, for example `0.55`, batcontrol grid-charges toward this target when charging is economical. Leave it unset to keep the default behavior. To also preserve this target as reserved energy during cheap/pre-expensive windows, enable the expert option `preserve_min_grid_charge_soc`. If `min_grid_charge_soc` is higher than `max_charging_from_grid_limit`, grid charging cannot reach the configured minimum SoC target. Batcontrol will log a warning in this case; increase `max_charging_from_grid_limit` or lower `min_grid_charge_soc` so the settings correlate. `min_recharge_amount` controls the minimum amount of Wh is needed to be recharged before batcontrol activates battery charging. ## Battery Control Expert Tuning Parameters ``` battery_control_expert: charge_rate_multiplier: 1.1 soften_price_difference_on_charging: false soften_price_difference_on_charging_factor: 5 round_price_digits: 4 production_offset_percent: 1.0 preserve_min_grid_charge_soc: false ``` These expert parameters allow fine-tuning of Batcontrol's behavior. See [Battery Control Expert](https://mastr.github.io/batcontrol/features/battery-control-expert/index.md) for detailed explanations of each parameter: | Parameter | Type | Default | Description | | -------------------------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------- | | `charge_rate_multiplier` | float | 1.1 | Multiplier for calculated charge rate to compensate for charging inefficiencies | | `soften_price_difference_on_charging` | boolean | false | Enable earlier charging based on more relaxed price difference calculations | | `soften_price_difference_on_charging_factor` | integer | 5 | Factor to soften price difference requirements when enabled | | `round_price_digits` | integer | 4 | Decimal places for price rounding in comparisons | | `production_offset_percent` | float | 1.0 | Multiplier to adjust solar production forecast (1.0 = no change, 0.8 = 80%, etc.) | | `preserve_min_grid_charge_soc` | boolean | false | Also preserve `min_grid_charge_soc` as reserved battery energy during cheap/pre-expensive windows | ## General options ### max_grid_charge_rate This is the upper limit to charge the battery from grid. Value is WATT. This value should not be above the limit of your inverter. Default: ``` max_grid_charge_rate: 5000 ``` ### max_pv_charge_rate This limits the used amount of PV to charge the battery. Value is WATT. With adding `#` in front of the value, this limit is not set and will push all PV into the battery. Default: ``` #max_pv_charge_rate: 3000 (Disabled) ``` ## Resilient Wrapper Options (since 0.7.0) These options enable graceful handling of temporary inverter outages (e.g., during firmware upgrades or network interruptions). ### enable_resilient_wrapper Enable or disable the resilient wrapper for graceful outage handling. When enabled, a temporary inverter failure makes batcontrol skip the current control cycle and retry on the next scheduled run, instead of terminating. No decisions are made on stale data - the inverter is simply read again next cycle. This helps batcontrol survive brief connection losses (e.g. firmware upgrades) without exiting. Errors before the first successful control command still fail fast, so configuration mistakes are caught at startup. Why not just let batcontrol crash and rely on the container restart policy? Without the wrapper, every inverter outage terminates the process, and `restart: unless-stopped` brings it straight back up. During a multi-minute outage this turns into a tight restart loop, and **each restart re-fetches the price and solar forecasts from their providers**. Repeated cold starts can therefore run into provider rate limits (e.g. Awattar/Tibber for prices, Forecast.Solar/SolarPrognose for solar), which can leave batcontrol without fresh data even after the inverter recovers. Keeping the process alive and skipping cycles avoids hammering both the inverter and the data providers. Default: ``` enable_resilient_wrapper: false ``` ### outage_tolerance_minutes The maximum duration (in minutes) to tolerate inverter outages before terminating. While the inverter is unreachable, each control cycle is skipped. If communication is not restored within this window, batcontrol gives up and exits with an error. Default: ``` outage_tolerance_minutes: 24 # 24 minutes ``` ## mqtt This enables the MQTT inverter driver, which allows integration with any battery/inverter system via MQTT topics. This is a generic bridge that works with any external system that can publish battery status and receive control commands over MQTT. For detailed documentation, see [MQTT Inverter](https://mastr.github.io/batcontrol/integrations/mqtt-inverter/index.md). ``` inverter: type: mqtt capacity: 10000 # Battery capacity in Wh (required) min_soc: 5 # Minimum SoC % (default: 5) max_soc: 100 # Maximum SoC % (default: 100) max_grid_charge_rate: 5000 # Maximum charge rate in W (required) cache_ttl: 120 # Cache TTL for SOC values in seconds (default: 120) ``` **Key Features:** - Generic MQTT-based integration for any inverter - Uses batcontrol's shared MQTT connection - Supports Home Assistant auto-discovery - Real-time status updates and command control - No vendor-specific protocols required ## fronius_gen24 This enables the Fronius GEN24 inverter. ``` inverter: type: fronius_gen24 #currently only fronius_gen24 supported address: 192.168.0.XX # the local IP of your inverter. needs to be reachable from the machine that runs batcontrol user: customer #customer or technician lowercase only!! password: YOUR-PASSWORD # max_grid_charge_rate: 5000 # Watt fronius_inverter_id: '1' # Optional: ID of the inverter in Fronius API (default: '1') (ab 0.5.6) fronius_controller_id: '0' # Optional: ID of the controller in Fronius API (default: '0') (ab 0.5.6) ``` ### Additional Parameters (since 0.5.6) - **fronius_inverter_id**: Optional parameter to specify the inverter ID in the Fronius API. Default is '1'. - **fronius_controller_id**: Optional parameter to specify the controller ID in the Fronius API. Default is '0'. ## fronius-modbus This enables the Fronius Modbus TCP inverter backend. It controls a Fronius GEN24/BYD battery through SunSpec storage-control registers and does not require inverter web-login credentials. Enable Modbus TCP in the Fronius inverter web UI before using this backend. ``` inverter: type: fronius-modbus address: 192.168.0.XX # Local IP/host of your inverter port: 502 # Optional, default: 502 unit_id: 1 # Optional, default: 1 capacity: 10000 # Required: battery capacity in Wh max_grid_charge_rate: 5000 # Required: maximum grid charge rate in W min_soc: 5 # Optional, default: 5 max_soc: 100 # Optional, default: 100 revert_seconds: 0 # Optional, default: 0 ``` ### Backup / emergency-power systems For systems with backup or emergency-power support, batcontrol should run from a UPS/USV-backed power source. If the public grid fails while restrictive Modbus battery flags are active, batcontrol must remain powered so it can react and reset the Modbus flags. Optional backup-mode safety settings: ``` backup_mode_safety_enabled: true meter_unit_id: 200 # Optional, default: 200 ``` With backup-mode safety enabled, restrictive battery-control writes are only sent while the grid is detected as available. If grid status is off-grid, unknown, or unreadable, batcontrol restores allow-discharge mode instead. ### Notes - Do not run multiple tools that write Fronius battery-control Modbus registers at the same time. - If you previously changed battery-control registers with another tool, stop that tool and restart the inverter before running batcontrol. ## dummy This option is for testing purposes only \*\*\* Sample needs to be added \*\*\* Currently following data providers are available: - tibber - awattar - Two-Tariff Providers (e.g. Octopus) - evcc - energyforecast.de You can chose one and need to adjust the configuration. ## tibber You need to get an API Key from https://developer.tibber.com/ , which looks like `Zz-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx`. After obtaining this key, use the following configuration: ``` utility: type: tibber apikey: YOUR-PASSWORD ``` ## awattar batcontrol provides two different awattar types: - `awattar_de` for German aWATTar - `awattar_at` for Austrian aWATTar Please choose the corresponding version. For aWATTar you can use this configuration: ``` utility: type: awattar_de vat: 0.19 # 19% VAT fees: 0.015 # Depends on you Netzendgeld markup: 0.03 # Depends on you aWATTar contract ``` The calculation is `( marketprice/1000*(1+markup) + fees ) * (1+vat)` ## Multi-Zone Tariff Providers (e.g. Octopus, Two-Tariff) (since 0.7.0) If your energy provider offers distinct tariff zones (e.g. day/night rates, peak/off-peak), you can configure batcontrol to optimize battery usage accordingly. The `tariff_zones` provider supports up to **3 different price zones**. ## `tariff_zones` — Static & multi-zone tariff The `tariff_zones` provider uses a locally configured, fixed price schedule instead of an external API. It supports **1, 2, or 3 zones**: | Zones configured | Behaviour | | ------------------------- | ---------------------------------------- | | **1 zone** (static price) | One flat price for every hour of the day | | **2 zones** | Classic peak / off-peak split | | **3 zones** | Peak / shoulder / off-peak | > **Since v0.8.0:** `tariff_zone_2` / `zone_2_hours` are **optional**. In earlier versions zone 2 was mandatory. Configurations written for the old 2-zone-only behaviour keep working unchanged. This makes it suitable for: - Users **without a dynamic tariff** who still want peak-shaving (see issue [#318](https://github.com/MaStr/batcontrol/issues/318)). - Users with a fixed day / night tariff. - Users with a fixed three-period (HT/NT/shoulder) tariff. ### Static price mode (single zone) > Available since **v0.8.0**. Set only `tariff_zone_1`. `zone_1_hours` is optional and defaults to all 24 hours. ``` utility: type: tariff_zones tariff_zone_1: 0.30 # EUR/kWh incl. VAT/fees, applied to every hour ``` ### Two zones (peak / off-peak) ``` utility: type: tariff_zones tariff_zone_1: 0.2733 # peak price zone_1_hours: 7-22 tariff_zone_2: 0.1734 # off-peak price zone_2_hours: 0-6,23 ``` ### Three zones (peak / shoulder / off-peak) ``` utility: type: tariff_zones tariff_zone_1: 0.30 zone_1_hours: 9-16 tariff_zone_2: 0.15 zone_2_hours: 0-6,23 tariff_zone_3: 0.22 zone_3_hours: 7-8,17-22 ``` ### Configuration reference | Key | Required | Description | | --------------- | --------------------------------------------------- | ---------------------------------------------- | | `type` | yes | Must be `tariff_zones` | | `tariff_zone_1` | **yes** | Price for zone 1 in EUR/kWh (incl. VAT & fees) | | `zone_1_hours` | optional in single-zone mode; required otherwise | Hours assigned to zone 1 | | `tariff_zone_2` | optional since v0.8.0 (paired with `zone_2_hours`) | Price for zone 2 | | `zone_2_hours` | optional since v0.8.0 (paired with `tariff_zone_2`) | Hours assigned to zone 2 | | `tariff_zone_3` | optional (paired with `zone_3_hours`) | Price for zone 3 | | `zone_3_hours` | optional (paired with `tariff_zone_3`) | Hours assigned to zone 3 | #### Hour syntax `zone_*_hours` accepts flexible formats (may be mixed): - Single integer: `5` - Comma-separated list: `0,1,2,3` - Inclusive range: `0-5` → `[0, 1, 2, 3, 4, 5]` - Mixed: `0-5,6,7` - YAML list with any of the above: `[7, 8, '17-22']` #### Validation rules - `tariff_zone_1` is always required. All prices must be positive. - If `tariff_zone_2` is set, `zone_2_hours` **must** also be set (and vice versa). Same rule applies to zone 3. - **Every hour 0–23 must be assigned to exactly one zone** — no gaps, no overlaps. If you configure `zone_1_hours` explicitly in single-zone mode, it must cover the full day (it is **not** auto-extended); omit it to get the default `0-23`. - `zone_1_hours` may only be omitted when zones 2 and 3 are also omitted (single-zone / static mode, v0.8.0+). ### Charging behaviour tips The charge rate is not evenly distributed across low-price hours by default. - For **more even charging** across low-price hours, enable `soften_price_difference_on_charging` and set `max_grid_charge_rate` to a modest value (e.g. battery capacity ÷ low-price hours). - For a **late charging start** (optimise efficiency, keep the battery at high SOC for less time), disable `soften_price_difference_on_charging`. In pure single-zone (static) mode, prices never differ between hours, so price-based scheduling has no effect — use it together with peak-shaving (see [#271](https://github.com/MaStr/batcontrol/issues/271)) or fixed charging-window settings. ## evcc If you are running evcc, it can be used to fetch the price information from this endpoint. The configuration for this is ``` utility: type: evcc url: http://evcc.local:7070/api/tariff/grid ``` You may need to adjust hostname + port for your setup. If evcc is running under HomeAssistant, you should use either `http://homeassistant:7070/api/tariff/grid` or `http://:7070/api/tariff/grid` ## energyforecast.de (0.5.6) [energyforecast.de](https://www.energyforecast.de) provides a calculated forecast for upcoming prices. Dayahead prices are populated at 14:00 GMT+2, which is after the lunch-drop in prices and prevents a good energy calculation. Based on different values, energyforecast.de calculates a price expectation with a median of 3 cent of. batcontrol uses the 48h forecast only and it is not possible to activate the 96 hour forecast. You need to setup VAT, markup (+ % on energy price) and fees (Netzentgeld) in the configuration. We are not using the calculation provided by energyforecast.de. If you like to use this forecast type, please create a login at [energyforecast.de](https://www.energyforecast.de) to aquire an API key. ``` utility: type: energyforecast apikey: xxxxxxxxx vat: 0.19 # 19% VAT fees: 0.15 # Depends on you Netzendgeld markup: 0.00 # Depends on you aWATTar contract ``` To enable the paid 96h forecast, use type: energyforecast_96 ## Dynamic network fees (§14a EnWG) Batcontrol can add time-of-use network fees (NT/ST/HT zones according to §14a EnWG) on top of the energy prices. This only applies to providers that calculate fees locally: **awattar** and **energyforecast**. All-inclusive providers (tibber, evcc, tariff_zones) already deliver final prices and do not need this. The fee data (NET prices, excluding VAT) is fetched from [dyn-net.batcontrol.software](https://dyn-net.batcontrol.software/) and added to the raw energy price before VAT is applied. Data is cached for 12 hours, as tariffs change at most quarterly. Configure it as a top-level block (next to `utility`): ``` dynamic_network_fees: enabled: true # Set to true to activate dynamic network fees country: de # Country code: de, at, ch operator: syna # Network operator ID (e.g. syna, westnetz, ggv, ewr-netz) # url: https://dyn-net.batcontrol.software/api/ # Optional: override for self-hosted instance ``` | Key | Required | Description | | ---------- | -------- | -------------------------------------------------------------------------------------------------------------------- | | `enabled` | yes | Master switch, defaults to `false` | | `country` | yes | Country code (`de`, `at`, `ch`) | | `operator` | yes | Your network operator ID — see [dyn-net.batcontrol.software](https://dyn-net.batcontrol.software/) for available IDs | | `url` | no | Override the API endpoint, e.g. for a self-hosted instance | The following providers are currently available: - [Forecast Solar](https://forecast.solar/) - [Solarprognose.de](https://www.solarprognose.de) - since 0.5.0 - local evcc instance - since 0.5.3 - [HomeAssistant Solar Forecast ML](https://zara-toorox.github.io/) - since 0.7.0 Multiple Installations can be entered, like this: ``` solar_forecast_provider: fcsolarapi pvinstallations: - name: Haus #name lat: 48.4334480 lon: 8.7654968 declination: 32 #inclination toward horizon 0..90 0=flat 90=vertical (e.g. wallmounted) azimuth: -90 # -90:East, 0:South, 90:West -180..180 kWp: 15.695 # power in kWp - name: Garage #... further installations lat: 48.4334480 lon: 8.7654968 declination: 32 azimuth: 87 kWp: 6.030 ``` The **name** must be a unique value. If a solar forecast provider is not available, batcontrol is running on cached values. It stops working if less then 12 hours of forecast are available. That should be enough to overcome outages. ## Forecast.Solar (Default) [Forecast Solar](https://forecast.solar/) allows a limited amount of free requests with no subscription or account. The minimum configuration block is: ``` - name: Haus #name lat: 48.4334480 lon: 8.7654968 declination: 32 #inclination toward horizon 0..90 0=flat 90=vertical (e.g. wallmounted) azimuth: -90 # -90:East, 0:South, 90:West -180..180 kWp: 15.695 # power in kWp ``` In addtion you can register and use an api key with adding: ``` - name: Haus #name lat: 48.4334480 lon: 8.7654968 declination: 32 #inclination toward horizon 0..90 0=flat 90=vertical (e.g. wallmounted) azimuth: -90 # -90:East, 0:South, 90:West -180..180 kWp: 15.695 # power in kWp api: ffff-ffff-fff-ffff ``` If you have an obstructed horizon, you can add a horizon modifier: ``` - name: Haus #name lat: 48.4334480 lon: 8.7654968 declination: 32 #inclination toward horizon 0..90 0=flat 90=vertical (e.g. wallmounted) azimuth: -90 # -90:East, 0:South, 90:West -180..180 kWp: 15.695 # power in kWp horizon: 30,30,30,0,0,0 # leave empty for default PVGIS horizon, only modify if solar array is shaded by trees or houses ``` ## Solarprognose.de Solarprognose offers a free tier for installations below 10KW. Currently, larger tiers are available for free, but this may change. The provider is asking for donations. You need to register on their website and enter you installation. With using the provided API key, you can run batcontrol with following configuration: ``` solar_forecast_provider: solarprognose pvinstallations: - name: Haus #name apikey: 44k4j5j5j5j5j6j6j6j6j6j6j6j6j6j6j6j6 ``` This configuration delivers the forecast for the first defined location. The API provider asks to add `project: ` as an additional parameter, that he can contact a person in case of issues. In addition you can change the algorithm using: ``` pvinstallations: - name: Haus #name apikey: 44k4j5j5j5j5j6j6j6j6j6j6j6j6j6j6j6j6 algorithm: own-v1 # (Default is 'mosmix') ``` If you run multiple installations with you account or want to split up forecasts for reasons, you can use the ITEM and ID syntax. - item: - token: For further details see: [API description](https://www.solarprognose.de/web/de/solarprediction/page/api) ## Local evcc instance evcc is able to collect its own PV forecast, which can be obtained via REST API. batcontrol can make use of that. ``` solar_forecast_provider: evcc-solar pvinstallations: - name: Haus #name url: http://evcc.local:7070/api/tariff/solar ``` If evcc is running under HomeAssistant, you should use either `http://homeassistant:7070/api/tariff/solar` or `http://:7070/api/tariff/solar` ## HomeAssistant Solar Forecast ML The [HomeAssistant Solar Forecast ML](https://zara-toorox.github.io/) integration (available via HACS) provides machine learning-based solar forecasts directly from your HomeAssistant instance. This provider requires the HACS integration to be installed first. Use the evcc based sensor, which might be additionally enabled in the SolarML Addon. Sensor name is `sensor.solar_forecast_ml_evcc_solar_prognose` . If you have startup issues, define sensor_unit `Wh`. **Minimum Requirements:** - batcontrol version: 0.7.2 - HomeAssistant addon minimum version: V16.2.0 The minimum configuration is: ``` solar_forecast_provider: homeassistant-solar-forecast-ml pvinstallations: - name: HA Solar ML Forecast base_url: ws://homeassistant.local:8123 # Your HomeAssistant URL api_token: eyJ... # Long-lived access token from HA Profile entity_id: sensor.solar_forecast_ml_evcc_solar_prognose # Forecast sensor entity ``` If you're running batcontrol in a HomeAssistant addon, use `ws://homeassistant:8123` as the base_url. For standalone installations, use your HomeAssistant IP or hostname. ### Optional Parameters You can customize the behavior with additional parameters: ``` pvinstallations: - name: HA Solar ML Forecast base_url: ws://homeassistant:8123 api_token: eyJ... entity_id: sensor.solar_forecast_ml_prognose_nachste_stunde sensor_unit: auto # Options: 'auto' (default, auto-detect), 'Wh', or 'kWh' cache_ttl_hours: 24.0 # Cache duration in hours (default: 24.0) ``` The `sensor_unit` parameter: - `auto` (default): Automatically detects the unit from the sensor - `Wh`: If you know your sensor reports in Wh - `kWh`: If you know your sensor reports in kWh Setting the explicit unit (`Wh` or `kWh`) can speed up startup by skipping auto-detection. ## Adjusting Production Forecasts ### production_offset_percent If your actual solar production systematically differs from the forecast (e.g., winter snow coverage, panel degradation, or consistently higher performance), you can adjust the entire forecast using the `production_offset_percent` parameter in the `battery_control_expert` section: ``` battery_control_expert: production_offset_percent: 0.8 # Use 80% of the forecast (20% reduction) ``` This multiplier is applied to all forecasted values: - `1.0` = no adjustment (default) - `0.8` = 80% of forecast (useful for winter/snow conditions) - `1.1` = 110% of forecast (for systems that consistently outperform) For detailed information, see [Battery Control Expert - production_offset_percent](https://mastr.github.io/batcontrol/features/battery-control-expert/#adjust-solar-production-forecast). # Consumption Forecast Batcontrol uses consumption forecasting to predict your household's energy usage for optimal battery management. This helps the system make smart decisions about when to charge or discharge your battery based on expected consumption patterns. ## Available Forecast Providers Batcontrol supports two methods for consumption forecasting: 1. **CSV** - Static load profile based on typical consumption patterns 1. **HomeAssistant API** - (Since 0.5.4) Dynamic forecast based on your actual historical consumption data ______________________________________________________________________ ## 1. CSV-Based Forecast The CSV method uses a predefined load profile file with typical consumption patterns. This is the default and simplest option. ### Configuration (Since 0.5.0) ``` consumption_forecast: type: csv csv: annual_consumption: 4500 # Total consumption in kWh per year load_profile: load_profile.csv # Name of the load profile file in config folder ``` ### Configuration (Before 0.5.0) ``` consumption_forecast: annual_consumption: 4500 # Total consumption in kWh per year load_profile: load_profile.csv # Name of the load profile file in config folder ``` ### CSV File Format The CSV file must be placed in the `config/` folder and contain the following fields: ``` month,weekday,hour,energy ``` **Field Definitions:** - `month`: 1-12 (January = 1, December = 12) - `weekday`: 0-6 (Monday = 0, Sunday = 6) - `hour`: 0-23 (midnight = 0, 11 PM = 23) - `energy`: Consumption in Wh (Watt-hours) ### Example CSV Entry ``` 1,0,8,350 ``` This means: In January, on Monday, at 8 AM, the consumption is 350 Wh. ### How CSV Scaling Works When batcontrol loads the CSV profile, it: 1. Calculates the total annual consumption from the load profile 1. Compares it to your configured `annual_consumption` 1. Scales all hourly values proportionally to match your actual consumption **Example log output:** ``` INFO [FC Cons] The annual consumption of the applied load profile is 3225.29 kWh INFO [FC Cons] The hourly values from the load profile are scaled with a factor of 1.40 to match the annual consumption of 4500 kWh ``` ### Default Load Profile If no load profile is specified, batcontrol uses `default_load_profile.csv` as a fallback. ______________________________________________________________________ ## 2. HomeAssistant API-Based Forecast The HomeAssistant API method provides **dynamic consumption forecasting** based on your actual historical consumption data. This is the most accurate method as it learns from your real usage patterns. ### How It Works 1. **Connects to HomeAssistant** via WebSocket API 1. **Fetches historical data** from configured time periods (e.g., last 7, 14, 21 days) 1. **Calculates weighted averages** for each hour of the week 1. **Generates forecasts** for up to 48 hours ahead 1. **Caches results** to minimize API calls ### Prerequisites - HomeAssistant instance accessible from batcontrol - Long-term statistics enabled for your consumption sensor - HomeAssistant Long-Lived Access Token ### Configuration ``` consumption_forecast: type: homeassistant-api homeassistant_api: base_url: ws://homeassistant.local:8123 # Your HomeAssistant URL apitoken: YOUR_LONG_LIVED_ACCESS_TOKEN # Long-Lived Access Token entity_id: sensor.energy_consumption # Entity ID with consumption data sensor_unit: auto # Options: 'auto', 'Wh', or 'kWh' (since 0.5.7) history_days: "-7;-14;-21" # Days to look back (negative values) history_weights: "1;1;1" # Weight for each history period (1-10) cache_ttl_hours: 48.0 # Cache duration in hours multiplier: 1.0 # Forecast adjustment multiplier ``` ### Configuration Parameters #### Required Parameters | Parameter | Description | Example | | ----------- | --------------------------------------------------------------- | ------------------------------- | | `base_url` | HomeAssistant URL (ws is correct) | `ws://homeassistant.local:8123` | | `apitoken` | Long-Lived Access Token from HomeAssistant | `eyJ0eXAiOiJKV1Qi...` | | `entity_id` | Entity ID tracking consumption (must have long-term statistics) | `sensor.energy_consumption` | #### Optional Parameters | Parameter | Default | Description | | ----------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `sensor_unit` | `auto` | **Since 0.5.7**: Sensor unit configuration. Options: `'auto'` (auto-detect), `'Wh'`, or `'kWh'`. Set to `'Wh'` or `'kWh'` to skip auto-detection (faster startup, recommended for large HA installations). | | `history_days` | `"-7;-14;-21"` | List of day offsets to fetch historical data. Negative values = days in the past. | | `history_weights` | `"1;1;1"` | Weight for each history period (1-10). Higher = more influence. Must match length of `history_days`. | | `cache_ttl_hours` | `48.0` | How long to cache computed statistics (in hours) | | `multiplier` | `1.0` | Global multiplier for all forecast values. Use `1.1` for +10%, `0.9` for -10% | ### Getting a HomeAssistant Access Token 1. Open HomeAssistant web interface 1. Click on your profile (bottom left) 1. Scroll down to **"Long-Lived Access Tokens"** 1. Click **"Create Token"** 1. Give it a name (e.g., "Batcontrol") 1. Copy the token (you won't be able to see it again!) ### Sensor Unit Configuration (Since 0.5.7) The `sensor_unit` parameter controls how batcontrol detects the unit of measurement for your consumption sensor. **Options:** - `auto` (default) - Automatically detect unit by querying HomeAssistant - `Wh` - Sensor reports in Watt-hours (no conversion needed) - `kWh` - Sensor reports in Kilowatt-hours (values multiplied by 1000) **When to use explicit configuration (`Wh` or `kWh`):** - **Large HomeAssistant installations** with many entities (faster startup, avoids "message too big" errors) - **Performance optimization** - skips the auto-detection query on every startup - **Consistent behavior** - eliminates the need to fetch all entity states **How to check your sensor's unit:** 1. Open HomeAssistant → Developer Tools → States 1. Find your entity (e.g., `sensor.energy_consumption`) 1. Check the `unit_of_measurement` attribute 1. Set `sensor_unit` accordingly in your configuration **Example:** ``` consumption_forecast: type: homeassistant-api homeassistant_api: base_url: ws://homeassistant.local:8123 apitoken: YOUR_TOKEN entity_id: sensor.energy_consumption sensor_unit: kWh # Explicit configuration - faster startup! ``` **Note:** If you're unsure, leave it as `auto` (default). Batcontrol will automatically detect the correct unit. ### Entity Requirements The entity you specify must: - Be a **sensor** entity - Track **cumulative energy consumption** (in Wh) - Have **long-term statistics enabled** - Provide **hourly statistics** via the HomeAssistant recorder **Good entity examples:** - `sensor.energy_consumption` - `sensor.house_energy_total` - `sensor.grid_import_total` **Not suitable:** - Instantaneous power sensors (W) - Entities without statistics - Non-energy entities ### Example Sensor Configuration Using SunSpec Integration - install SunSpec via HACS and configure with the IP from your inverter (see the SunSpec documentation if further informations are needed) - Create a template helper as sensor: Settings -> Devices & Services -> Helper -> Create helper -> Template -> Sensor - Name: e.g. `sensor.energy_consumption` or `sensor.house_energy_total` - State: ``` {{ states('sensor.smartmeter_ac_meter_total_watt_hours_imported') | float - states('sensor.smartmeter_ac_meter_total_watt_hours_exported') | float + (states('sensor.inverter_mppt_module_0_lifetime_energy') | float + states('sensor.inverter_mppt_module_1_lifetime_energy') | float + states('sensor.inverter_mppt_module_3_lifetime_energy') | float - states('sensor.inverter_mppt_module_2_lifetime_energy') | float) }} ``` - Unit of measurement: `kWh` - Device class: `Energy` - State class: `Total` ### How History Weights Work The `history_weights` parameter allows you to give more importance to recent data vs. older data. **Example 1: Equal weighting** ``` history_days: "-7;-14;-21" history_weights: "1;1;1" ``` All three weeks have equal influence (33.3% each). **Example 2: Recent data preferred** ``` history_days: "-7;-14;-21" history_weights: "3;2;1" ``` - Last week: 50% influence (3/6) - Two weeks ago: 33% influence (2/6) - Three weeks ago: 17% influence (1/6) **Example 3: Short-term forecast** ``` history_days: "-1;-2;-3" history_weights: "3;2;1" ``` Uses only the last 3 days for very dynamic forecasting. ### Multiplier for Forecast Adjustment The `multiplier` parameter allows you to globally adjust all forecast values: - `1.0` = No adjustment (default) - `1.1` = Increase forecast by 10% - `0.9` = Decrease forecast by 10% - `1.2` = Increase forecast by 20% **Use cases:** - You know consumption will increase (e.g., guests coming, new appliances) - You want to be more conservative with battery discharge - Seasonal adjustments without changing historical data ### Caching Behavior To minimize load on HomeAssistant: - Computed statistics are **cached** for `cache_ttl_hours` - Cache stores consumption values per weekday/hour combination - Cache is automatically refreshed when data is missing - Cache survives batcontrol restarts (in-memory cache) **Cache key format:** `"weekday_hour"` (e.g., `"0_14"` = Monday 14:00) ### WebSocket Communication The HomeAssistant forecaster uses the modern **WebSocket API** for efficient communication: 1. Establishes WebSocket connection 1. Authenticates with access token 1. Fetches hourly statistics using `recorder/statistics_during_period` 1. Processes and caches results 1. Reuses connection for multiple requests when possible This is more efficient than the REST API for frequent data fetches. ______________________________________________________________________ ## Testing Your Configuration ### Test Script Batcontrol includes a test script to verify your HomeAssistant configuration: ``` cd batcontrol/scripts python test_homeassistant_forecast.py ``` Edit the configuration section in the script: ``` HOMEASSISTANT_URL = "ws://homeassistant.local:8123" HOMEASSISTANT_TOKEN = "YOUR_LONG_LIVED_ACCESS_TOKEN" ENTITY_ID = "sensor.energy_consumption" HISTORY_DAYS = [-7, -14, -21] HISTORY_WEIGHTS = [3, 2, 1] ``` The script will: - Connect to HomeAssistant - Fetch historical data - Generate a 24-hour forecast - Display results in a formatted table with statistics ______________________________________________________________________ ## Troubleshooting ### CSV Method **Problem:** "The annual consumption of the applied load profile is X kWh" **Solution:** This is just informational. The profile will be automatically scaled to match your `annual_consumption` setting. **Problem:** "No load profile specified, using default" **Solution:** Specify a valid `load_profile` filename in your configuration. ### HomeAssistant API Method **Problem:** "Authentication failed" **Solution:** - Verify your access token is correct - Check if the token has been revoked in HomeAssistant - Create a new Long-Lived Access Token **Problem:** "ConnectionClosedError: sent 1009 (message too big)" or "websockets.exceptions.ConnectionClosedError: frame exceeds limit" **Solution (Since 0.5.7):** This error occurs when your HomeAssistant instance has many entities (sensors, lights, automations, etc.) and the response exceeds the WebSocket size limit during auto-detection. **Quick Fix:** Set the `sensor_unit` parameter explicitly to skip auto-detection: ``` consumption_forecast: type: homeassistant-api homeassistant_api: base_url: ws://homeassistant.local:8123 apitoken: YOUR_TOKEN entity_id: sensor.energy_consumption sensor_unit: kWh # or 'Wh' depending on your sensor ``` **How to determine your sensor unit:** 1. Open HomeAssistant → Developer Tools → States 1. Find your entity (e.g., `sensor.energy_consumption`) 1. Check the `unit_of_measurement` attribute 1. Use `kWh` if it shows "kWh", or `Wh` if it shows "Wh" **Technical details:** Batcontrol 0.5.7+ uses a 4MB WebSocket frame limit (up from 1MB) and allows you to skip the auto-detection query entirely by configuring the sensor unit explicitly. **Problem:** "No statistics data returned for entity" **Solution:** - Verify the entity exists in HomeAssistant - Check if long-term statistics are enabled for this entity - Wait for HomeAssistant to collect at least one hour of statistics - Check HomeAssistant logs for recorder issues **Problem:** "Connection refused" **Solution:** - Verify `base_url` is correct and accessible from batcontrol - Check if HomeAssistant is running - Verify network connectivity - Check firewall rules **Problem:** "Length of history_days must match history_weights" **Solution:** Ensure both lists have the same number of elements: ``` history_days: "-7;-14;-21" # 3 elements history_weights: "3;2;1" # 3 elements ``` **Problem:** "History weights must be between 1 and 10" **Solution:** Use only values from 1 to 10 in `history_weights`. **Problem:** Empty or incomplete forecast **Solution:** - Check if HomeAssistant has enough historical data (at least 7 days recommended) - Verify the entity is recording data continuously - Check cache TTL - try reducing it temporarily - Enable DEBUG logging to see detailed fetch information **Problem:** Forecast values are too small or too large **Solution (Since 0.5.7):** Batcontrol automatically detects whether your sensor reports in Wh or kWh and applies the correct conversion. If values are incorrect: 1. **Check auto-detection:** Let batcontrol auto-detect (default `sensor_unit: auto`) 1. **Verify sensor unit:** Check your sensor's `unit_of_measurement` in HomeAssistant 1. **Set explicitly:** If auto-detection fails, set `sensor_unit` manually: ``` sensor_unit: kWh # if sensor reports in kWh # or sensor_unit: Wh # if sensor reports in Wh ``` **Legacy workaround (before 0.5.7):** If auto-detection is not available, use the `multiplier` parameter: ``` multiplier: 1000 # Convert kWh to Wh (if sensor reports in kWh) ``` **Note:** Batcontrol expects all consumption values in Wh (Watt-hours) internally. ______________________________________________________________________ ## Comparison: CSV vs. HomeAssistant API | Feature | CSV | HomeAssistant API | | -------------------- | ------------------------- | ------------------------------- | | **Accuracy** | Generic patterns | Based on your actual usage | | **Setup Complexity** | Simple | Moderate (requires HA setup) | | **Maintenance** | Manual updates needed | Automatic learning | | **Dependencies** | None | HomeAssistant + Long-term stats | | **Flexibility** | Low (static profile) | High (adapts to changes) | | **Performance** | Fast (local file) | Cached (WebSocket API) | | **Best For** | Testing, consistent usage | Real-world scenarios | ______________________________________________________________________ ## Recommendations - **Start with CSV** for initial testing and setup - **Switch to HomeAssistant API** once you have historical data for accurate forecasting - Use **recent history weighting** (e.g., `[3, 2, 1]`) for more responsive forecasts - Set `cache_ttl_hours` to `24-48` hours for good balance between accuracy and API load - Use **multiplier** for temporary adjustments rather than changing configuration frequently - Monitor logs to ensure forecasts are being generated correctly ______________________________________________________________________ ## Advanced Tips ### Seasonal Adjustments For seasonal changes, consider: - Using shorter `history_days` periods (e.g., `-7, -14` instead of `-7, -14, -21`) - Adjusting the `multiplier` seasonally - Creating different CSV profiles for different seasons ### Multiple Consumption Points If you have multiple consumption sensors, you can: - Create a template sensor in HomeAssistant that combines them - Use the combined sensor's entity_id in batcontrol configuration ### Debugging Enable DEBUG logging to see detailed information: ``` logging.basicConfig(level=logging.DEBUG) ``` Look for: - WebSocket connection messages - Statistics fetch results - Cache hit/miss events - Weighted average calculations ______________________________________________________________________ ## Example Configurations ### Example 1: Simple Setup (CSV) ``` consumption_forecast: type: csv csv: annual_consumption: 4500 load_profile: load_profile.csv ``` ### Example 2: HomeAssistant with Equal Weights ``` consumption_forecast: type: homeassistant-api homeassistant_api: base_url: ws://192.168.1.100:8123 apitoken: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9... entity_id: sensor.house_energy_total sensor_unit: auto # Auto-detect (default) history_days: "-7;-14;-21" history_weights: "1;1;1" cache_ttl_hours: 48.0 multiplier: 1.0 ``` ### Example 3: HomeAssistant with Explicit Unit (Recommended for Large Installations) ``` consumption_forecast: type: homeassistant-api homeassistant_api: base_url: ws://homeassistant.local:8123 apitoken: your_token_here entity_id: sensor.energy_consumption sensor_unit: kWh # Explicit unit - faster startup, no auto-detection needed history_days: "-7;-14;-21" history_weights: "3;2;1" # Recent week has most influence cache_ttl_hours: 24.0 multiplier: 1.1 # Increase forecast by 10% ``` ### Example 4: Short-term Dynamic Forecast ``` consumption_forecast: type: homeassistant-api homeassistant_api: base_url: ws://homeassistant.local:8123 apitoken: your_token_here entity_id: sensor.energy_consumption sensor_unit: Wh # Explicit unit for optimal performance history_days: "-1;-2;-3" # Only last 3 days history_weights: "5;3;2" # Yesterday has most weight cache_ttl_hours: 12.0 # Shorter cache multiplier: 1.0 ``` ______________________________________________________________________ ## Related Documentation - [Batcontrol Configuration](https://mastr.github.io/batcontrol/configuration/batcontrol-configuration/index.md) - [How Batcontrol Works](https://mastr.github.io/batcontrol/getting-started/how-batcontrol-works/index.md) - [MQTT API](https://mastr.github.io/batcontrol/integrations/mqtt-api/index.md) - [Solar Forecast](https://mastr.github.io/batcontrol/configuration/solar-forecast/index.md) # Features # Peak Shaving ## Why Peak Shaving? Most PV systems produce peak power around midday. Without any intervention the battery charges as fast as possible and is full well before the afternoon. Once the battery is full, all surplus PV energy is fed into the grid -- often at the lowest grid prices of the day. For newer installations feed-in compensation may be very small or zero, so this exported energy is essentially wasted. Peak shaving solves this by **limiting the PV-to-battery charge rate** so the battery fills gradually over the course of the day. The goal is to reach full capacity only by a configurable target hour (e.g. 14:00). This way the battery absorbs as much solar energy as possible and grid feed-in during midday peaks is minimised. ## Status The algorithm is in the status "experimental", which is the reason why it is only available in the logic type `next`. After collecting enough experience with that feature, it will move into `default` eventually. ## Prerequisites Peak shaving was introduced with 0.8.0 and is only available with the **`next` logic type**. Set this in the `battery_control` section of your configuration: ``` battery_control: type: next # Required -- 'default' does not include peak shaving ``` The `default` logic type does not support peak shaving at all. Enabling peak shaving without switching to `next` has no effect. ## Configuration Add a `peak_shaving` block at the **top level** of your configuration file (not nested under `battery_control`): ``` peak_shaving: enabled: false mode: combined # 'time' | 'price' | 'combined' allow_full_battery_after: 14 # Hour (0-23) -- battery should be full by this hour price_limit: 0.05 # Euro/kWh -- slots at or below this price are "cheap" ``` ### Parameter Reference | Parameter | Type | Default | Description | | -------------------------- | ------ | ---------- | ---------------------------------------------------------------------- | | `enabled` | bool | `false` | Master switch for peak shaving | | `mode` | string | `combined` | Algorithm mode (see below) | | `allow_full_battery_after` | int | `14` | Target hour (0-23) by which the battery should be full | | `price_limit` | float | *none* | Price threshold in Euro/kWh. Required for modes `price` and `combined` | ### MQTT Runtime Control All four parameters can be changed at runtime via MQTT without restarting batcontrol: | Topic | Accepts | Description | | -------------------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------- | | `{base}/peak_shaving/enabled/set` | `true` / `false` | Enable or disable peak shaving | | `{base}/peak_shaving/allow_full_battery_after/set` | int 0-23 | Change the target hour | | `{base}/peak_shaving/mode/set` | `time` / `price` / `combined` | Change the algorithm mode | | `{base}/peak_shaving/price_limit/set` | float | Change the price threshold in EUR/kWh; send `-1` to disable the price component | Runtime changes are temporary and are not written back to the configuration file. ## The `allow_full_battery_after` Target Hour This parameter controls when the battery is **allowed** to be 100% full: - **Before this hour:** the PV charge rate may be limited (depending on mode). - **At or after this hour:** no limit is applied -- the battery charges as fast as possible. The target hour applies globally to **all three modes**. Set it to the hour by which your PV system typically produces enough to fill the battery. For many Central European systems `14` (2 PM) is a good starting point; adjust based on your panel orientation and local conditions. ## Modes Peak shaving offers three modes that control which algorithm components are active: ### `time` -- Time-Based Only Distributes the remaining free battery capacity evenly over the slots between now and `allow_full_battery_after`, using a **counter-linear ramp**. The allowed charge rate starts low and increases as the target hour approaches, which mirrors the typical PV generation curve that rises towards midday. `price_limit` is **not required** for this mode. **Formula:** ``` slots_remaining = n (slots until allow_full_battery_after) pv_surplus = sum of max(production - consumption, 0) per remaining slot If pv_surplus > free_capacity: wh_current_slot = 2 * free_capacity / (n * (n + 1)) charge_limit = wh_current_slot / interval_hours ``` **Example** (free capacity = 2000 Wh, 1 h intervals): | Hours to target | Allowed charge rate | | --------------- | ------------------- | | 8 | 55 W | | 4 | 200 W | | 2 | 666 W | | 1 | 2000 W (full rate) | If the expected PV surplus does not exceed the free capacity, no limit is applied -- the battery can absorb everything anyway. ### `price` -- Price-Based Only Reserves free battery capacity for upcoming **cheap-price** slots where PV is still producing. A slot is "cheap" when its price is at or below `price_limit`. `price_limit` is **required** for this mode. Only slots within the **production window** are considered. The production window ends at the first forecast slot where PV production is zero. This prevents reserving capacity for a cheap slot at e.g. 03:00 that would never produce any solar energy. **Before the cheap window:** 1. Sum the expected PV surplus during cheap slots to get the target reserve. 1. Calculate how much additional charging is allowed: `additional_allowed = free_capacity - target_reserve`. 1. If `additional_allowed <= 0`: block PV charging entirely (rate = 0). 1. Otherwise: spread `additional_allowed` evenly over the slots before the cheap window. **Inside the cheap window:** - If total PV surplus during cheap slots exceeds free capacity, spread `free_capacity` evenly over cheap slots so the battery fills gradually. - If surplus fits in free capacity, no limit is applied. ### `combined` -- Both Active (Default) Both the time-based and price-based components run in parallel. The **stricter (lower non-negative) limit wins**. This is the most conservative and generally recommended mode. `price_limit` is **required** for the price component. If `price_limit` is not set, the price component is disabled and `combined` falls back to **time-only** behaviour — batcontrol logs a warning at startup in this case. Set a numeric `price_limit` or change the mode to `time` to silence the warning. ## Charge Limit and Minimum Charge Rate The calculated charge limit is applied via **Mode 8** (`LIMIT_BATTERY_CHARGE_RATE`). In this mode the inverter caps PV-to-battery charging at the given wattage while still allowing the battery to discharge normally. A minimum charge rate of **500 W** is enforced: any computed limit between 1 W and 499 W is raised to 500 W to avoid inefficient low-power charging. A limit of exactly **0 W** (block charging completely) is kept as-is and is not raised. The charge limit is published via MQTT: | Topic | Type | Retained | Description | | ---------------------------------- | ---- | -------- | ---------------------------------------------------- | | `{base}/peak_shaving/charge_limit` | int | No | Current charge limit in W (-1 = inactive / no limit) | ## When Peak Shaving is Skipped Peak shaving is automatically bypassed in the following situations: | Condition | Reason | | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | No PV production (nighttime) | Nothing to limit | | Past `allow_full_battery_after` hour | Target reached, charge freely | | Battery in `always_allow_discharge` region (high SOC) | Battery is nearly full anyway | | Force-charge from grid active (Mode -1) | Grid charging takes priority | | Discharge not allowed | Battery is being preserved for expensive hours -- limiting PV would be counterproductive | | evcc is actively charging the EV | The EV already consumes excess PV | | EV connected in PV mode (evcc) | evcc will absorb surplus PV when its threshold is reached | | `price_limit` not configured | Price component cannot operate; `combined` falls back to time-only, `price` is effectively inactive | ## evcc Interaction When an EV charger is managed by [evcc](https://mastr.github.io/batcontrol/integrations/evcc-connection/index.md): - **EV actively charging** (`charging=true`): peak shaving is disabled because the EV is already consuming excess PV energy. - **EV connected in PV mode** (`connected=true` AND `mode=pv`): peak shaving is disabled because evcc will naturally absorb surplus PV once its threshold is reached. - **EV disconnects or mode changes**: peak shaving is automatically re-enabled. ## Home Assistant Auto-Discovery When MQTT auto-discovery is enabled, the following Home Assistant entities are created automatically: | Entity | Type | Description | | ----------------------------- | ------------- | ------------------------------- | | Peak Shaving Enabled | Switch | Enable/disable peak shaving | | Peak Shaving Allow Full After | Number (0-23) | Set the target hour | | Peak Shaving Charge Limit | Sensor (W) | Current calculated charge limit | ## Self-Correction The charge limit is recalculated every evaluation cycle (typically every 3 minutes). If clouds reduce PV production significantly, the free capacity stays higher at the next cycle and the counter-linear ramp automatically produces a higher allowed rate. This means the system self-corrects without manual intervention, though there is no intra-cycle adjustment. ## Quick-Start Examples **Simple time-based setup** -- spread charging until 14:00, no price awareness: ``` battery_control: type: next peak_shaving: enabled: true mode: time allow_full_battery_after: 14 ``` **Price-aware combined setup** -- reserve capacity for cheap slots below 5 ct/kWh: ``` battery_control: type: next peak_shaving: enabled: true mode: combined allow_full_battery_after: 14 price_limit: 0.05 ``` # Adjust the charging rate on a charging event - Config option: `charge_rate_multiplier` - Default: 1.1 When Batcontrol reaches the lowest point in the price curve, it determines which future hours exceed a minimum price threshold. For those hours, it retrieves the required Wh from the load profile, and those Wh are then recharged. ``` charge_rate = required_recharge_energy/remaining_time charge_rate * charge_rate_multiplier = final_charge_rate if final_charge_rate < 500W then charge_rate = 500W ``` Example: We are currently at a price of €0.30. We plan to recharge for two hours during which the price will exceed €0.35, and we will need 600 Wh and 400 Wh respectively. The battery is empty, so we recharge a total of 1,000 Wh. Because there are conversion losses, we use the charge_rate_multiplier to increase the charging power: `1,000 Wh * 1.1 = 1,100 W` With a multiplier value of 1.0, you may observe that the charging power ramps up over the course of the hour because the setpoints are reached too slowly. At a value of 1.5, the target is reached more quickly and then charging slows steadily as the hour progresses. # Adjust Charging pricepoint - Config options: - `soften_price_difference_on_charging` ; true / false - Enable / Disable - `soften_price_difference_on_charging_factor` ; 5 - Default: disabled By default, Batcontrol checks: ``` found_lower_price = future_price <= current_price ``` This means that any future price that is lower or equal to the current price (current_price) is considered cheaper, and Batcontrol adjusts its evaluation period accordingly. The result is, that **only** at the lowest point, batcontrol searches for possible price-targets, which needs to be covered by energy stored in the battery. ### How the Soften Mechanism Works When you enable `soften_price_difference_on_charging` (by setting it to True), Batcontrol modifies the current price by a fraction ( `soften_price_difference_on_charging_factor` ) of `min_price_difference`: ``` modified_price = current_price - min_price_difference / self.soften_price_difference_on_charging_factor found_lower_price = future_price <= modified_price ``` **Key idea:** The code requires the future price to be lower than a slightly reduced threshold (modified_price) before treating it as “cheaper.” In other words, batcontrol will only stop evaluating (the current price window) if the future price is truly lower by some margin—rather than just marginally lower. ### Example Let's assume: - `current_price = 0.30 €/kWh` - `min_price_difference = 0.02 €/kWh` - `soften_price_difference_on_charging = True` - `soften_price_difference_on_charging_factor = 2.0` Then ``` modified_price = current_price - min_price_difference / self.soften_price_difference_on_charging_factor = 0.30 - (0.02 / 2.0) = 0.30 - 0.01 = 0.29 ``` Now, a future price will only be considered “cheaper” if it's **less than or equal** to `€0.29/kWh`. If the next hour’s price is `€0.295/kWh` (which is indeed lower than `€0.30/kWh`), Batcontrol **will not** count it as cheap enough to interrupt the current evaluation period—because `€0.295` is still higher than \`€0.29. Without `soften_price_difference_on_charging`, Batcontrol would see €0.295 as cheaper than €0.30, so it does not evaluate for chasing a marginally lower price. By introducing this “softening” factor, we allow batcontrol to decide earlier how much energy needs to be charged. This helps in scenarios where the **battery can not be charged to maximum within one hour**. It ensures Batcontrol waits for a more significant price drop before adjusting its strategy, too. The **downside** is, that not each cent of saving is achieved. # Granularity in price calculations: - Config option: `round_price_digits` - Default: 4 ## round_price_digits **Config option**: `round_price_digits`\ **Default**: `4` This option defines how many decimal places the algorithm uses when comparing electricity prices. By default, Batcontrol rounds prices to 4 decimal places (e.g., `0.30345` becomes `0.3035`). Since Batcontrol relies on precise comparisons (e.g., "greater than" or "less than") to determine when to charge or discharge the battery, the number of decimal places can significantly affect how it perceives small price differences. ### Why Does It Matter? - **Higher Precision (more decimal places)**\ If you increase `round_price_digits`, Batcontrol will consider smaller price differences. This can lead to more frequent adjustments if tiny price variations cause the algorithm to switch between charging and not charging. - **Lower Precision (fewer decimal places)**\ If you decrease `round_price_digits`, Batcontrol ignores subtle fluctuations below that rounding threshold. As a result, the algorithm behaves more conservatively and won't react to very small price movements. ### Example - **4 decimal places (default)** - A price of `0.30345 €/kWh` is rounded to `0.3035 €/kWh`. - A price of `0.30344 €/kWh` is rounded to `0.3034 €/kWh`. - The difference (`0.3035 - 0.3034`) is `0.0001 €/kWh`. - **2 decimal places** - Both `0.30345 €/kWh` and `0.30344 €/kWh` become `0.30 €/kWh`. - The difference disappears, so Batcontrol sees them as the same price. By adjusting `round_price_digits`, you can fine-tune how sensitive Batcontrol is to minor variations in the price curve. If your tariff data is very precise and you want every tiny fluctuation to matter, use more decimal places. If you prefer a more stable, less reactive charging strategy, you can reduce the decimal precision. # Adjust Solar Production Forecast - Config option: `production_offset_percent` - Default: 1.0 ## production_offset_percent (since 0.6.1) **Config option**: `production_offset_percent` **Default**: `1.0` This option allows you to adjust the solar production forecast by a percentage multiplier. This is particularly useful for scenarios where your actual solar production differs systematically from the forecast—such as during winter months when solar panels may be partially covered with snow, or when dust/dirt affects panel efficiency. ### How It Works The `production_offset_percent` parameter multiplies the entire solar production forecast. For example: - `1.0` = 100% of the forecast (no adjustment, default behavior) - `0.8` = 80% of the forecast (20% reduction, useful for winter/snow conditions) - `1.2` = 120% of the forecast (20% increase) - `0.5` = 50% of the forecast (50% reduction) The adjusted forecast is then used throughout Batcontrol's decision-making logic, affecting: - Charging recommendations based on solar production - Battery discharge decisions - Grid charging evaluations ### Use Cases **Winter Mode (Snow Coverage)** ``` battery_control_expert: production_offset_percent: 0.7 # Reduce forecast to 70% during winter ``` If panels are covered with snow, actual production may be 20-30% lower than the forecast. Using `0.7` helps Batcontrol make more conservative charging decisions and avoid over-discharging the battery when the predicted solar energy doesn't materialize. **Panel Degradation or Dirt** ``` battery_control_expert: production_offset_percent: 0.95 # 5% reduction for degraded panels ``` Over time, solar panels degrade slightly. If your panels are particularly dirty or aged, you can apply a small reduction factor. **High-Efficiency Summer** ``` battery_control_expert: production_offset_percent: 1.05 # 5% increase during peak summer ``` In optimal conditions, your system might consistently outperform forecasts. A slight increase can help maximize battery charging from available solar energy. ### Example Impact Assume: - Forecasted solar production: 5,000 W - `production_offset_percent: 0.8` (winter mode) Then: - **Adjusted forecast**: 5,000 W × 0.8 = 4,000 W - Batcontrol will base its decisions on 4,000 W instead of 5,000 W This conservative approach prevents Batcontrol from over-planning battery discharge based on overly optimistic solar forecasts. ### Logging When `production_offset_percent` differs from `1.0`, Batcontrol logs the adjustment: ``` Production forecast adjusted by 80% (factor: 0.8) ``` This helps you verify that your offset setting is being applied correctly. ## Understanding `min_price_difference` and the Relative Approach In [Issue #84](https://github.com/muexxl/batcontrol/issues/84), the idea arose to make the parameter `min_price_difference` more flexible. Originally, it was a fixed absolute value, like `€0.03`, meaning Batcontrol would only consider future prices “high enough” if they exceeded the current price by at least `€0.03`. However, when prices vary a lot (e.g., from `€0.20` to `€1.00`), a fixed amount may not always make sense. This feature is introduced in version 0.4.0, but defaults to `0%` (disabled). ### How It Works 1. **Absolute threshold** (`min_price_difference`) 1. Example: `€0.05` 1. Batcontrol checks if the future price is at least `€0.05` higher than the current price. 1. **Relative threshold** (`min_price_difference_rel`) 1. Example: `0.10` (i.e., 10% of the current price) 1. Batcontrol multiplies the current price by `0.10`. 1. If `current_price = €0.50`, the difference is `€0.05`. 1. If `current_price = €1.00`, the difference becomes `€0.10`. Batcontrol then takes **whichever value is larger**—the absolute amount or the relative amount. If your price is low, you might be governed mostly by the fixed value. If your price is high, the relative difference might exceed the absolute difference, so Batcontrol won’t trigger too soon. ### Example Scenarios 1. **Prices around `€0.30`:** 1. Absolute threshold: `€0.05` 1. Relative threshold (`10%`): `€0.30 * 0.10 = €0.03` 1. In this case, the absolute threshold (`€0.05`) is larger, so Batcontrol waits for at least a `€0.05` jump. 1. **Prices around `€1.00`:** 1. Absolute threshold: `€0.05` 1. Relative threshold (`10%`): `€1.00 * 0.10 = €0.10` 1. Now the relative threshold (`€0.10`) is bigger than `€0.05`, so Batcontrol won’t start charging unless the future price is at least `€0.10` higher. ### Why This Matters - **Fixed Absolute Value Only**: Easy to configure, but may be too small when prices get very high, or too large when prices are very low. - **Relative Plus Absolute**: Scales automatically with the current price, ensuring Batcontrol’s logic remains balanced across a wide range of price levels. This change helps prevent overreacting at high prices (where a small amount like `€0.03` is negligible) and avoids unnecessary waiting at low prices (where a large absolute amount might never occur). # Integrations # MQTT API Configuration Batcontrol provides an MQTT API that allows you to monitor and integrate your battery control system with other home automation platforms like Home Assistant. The MQTT interface publishes battery status, pricing information, and control states to configurable topics. ## Basic Configuration ``` mqtt: enabled: true logger: false broker: localhost port: 1883 topic: house/batcontrol username: user password: password ``` ### Basic Parameters | Parameter | Type | Default | Description | | ---------- | ------- | ------------------ | ----------------------------------------------------- | | `enabled` | boolean | `false` | Enable or disable the MQTT API | | `logger` | boolean | `false` | Enable MQTT logging for debugging | | `broker` | string | `localhost` | MQTT broker hostname or IP address | | `port` | integer | `1883` | MQTT broker port (1883 for unencrypted, 8883 for TLS) | | `topic` | string | `house/batcontrol` | Base topic for all batcontrol MQTT messages | | `username` | string | `user` | MQTT broker username (if authentication required) | | `password` | string | `password` | MQTT broker password (if authentication required) | ## Advanced Configuration ### Connection Reliability ``` mqtt: retry_attempts: 5 # Number of connection retry attempts retry_delay: 10 # Delay in seconds between retry attempts ``` | Parameter | Type | Default | Description | | ---------------- | ------- | ------- | ---------------------------------------------- | | `retry_attempts` | integer | `5` | Number of times to retry connection on failure | | `retry_delay` | integer | `10` | Seconds to wait between retry attempts | ### TLS/SSL Configuration > ⚠️ **Note**: TLS/SSL support is currently **untested and known to be non-functional**: the implementation expects the certificate options nested below `tls`, while the enable check expects a boolean — these requirements contradict each other. Track progress or report your use case in the project issues before relying on TLS. ## Home Assistant Auto-Discovery Batcontrol supports Home Assistant's MQTT auto-discovery feature, which automatically creates entities in Home Assistant without manual configuration. ``` mqtt: auto_discover_enable: true auto_discover_topic: homeassistant ``` | Parameter | Type | Default | Description | | ---------------------- | ------- | --------------- | -------------------------------------- | | `auto_discover_enable` | boolean | `true` | Enable Home Assistant auto-discovery | | `auto_discover_topic` | string | `homeassistant` | Base topic for auto-discovery messages | When enabled, batcontrol will publish device and entity configuration messages to topics like: - `homeassistant/sensor/batcontrol/battery_soc/config` - `homeassistant/sensor/batcontrol/current_price/config` - `homeassistant/binary_sensor/batcontrol/charging_active/config` ## Published Topics Batcontrol publishes data to the following topic structure (assuming base topic `house/batcontrol`): ### System Status - `house/batcontrol/status` - System status (`online`/`offline`) - `house/batcontrol/last_evaluation` - Timestamp of last evaluation (Unix timestamp) - `house/batcontrol/evaluation_intervall` - Evaluation interval in seconds ### Control & Mode - `house/batcontrol/mode` - Current operational mode: - `-1` = Charge from Grid - `0` = Avoid Discharge - `8` = Limit Battery Charge Rate ([peak shaving](https://mastr.github.io/batcontrol/features/peak-shaving/index.md)) - `10` = Discharge Allowed - `house/batcontrol/charge_rate` - Current charge rate in W - `house/batcontrol/limit_battery_charge_rate` - Dynamic battery charge rate limit in W - `house/batcontrol/discharge_blocked` - Whether discharge is blocked (`true`/`false`) - `house/batcontrol/api_override_active` - Whether a temporary external/API override is active (`true`/`false`) - `house/batcontrol/control_source` - Source that last selected the current control state (`api` or `optimizer`) ### Battery Information - `house/batcontrol/SOC` - State of Charge in % (two decimal places, e.g., `69.00`) - `house/batcontrol/max_energy_capacity` - Maximum battery capacity in Wh - `house/batcontrol/stored_energy_capacity` - Energy stored in battery in Wh - `house/batcontrol/stored_usable_energy_capacity` - Usable energy stored in battery in Wh (considering min SOC) - `house/batcontrol/reserved_energy_capacity` - Energy reserved for discharge in Wh ### Solar Surplus Information See [Forecast Metrics](https://mastr.github.io/batcontrol/integrations/forecast-metrics/index.md) for a detailed explanation of these values and how to use them for flexible load control. - `house/batcontrol/solar_surplus_wh` - Expected PV overflow in Wh that cannot be stored in the battery - `house/batcontrol/solar_active` - Whether solar is currently producing (`true`/`false`) - `house/batcontrol/pv_start_battery_wh` - Battery level in Wh at the next net-charging crossover - `house/batcontrol/forecast_min_battery_wh` - Minimum battery level in Wh over the entire forecast horizon ### Configuration Limits - `house/batcontrol/always_allow_discharge_limit` - Always discharge limit (0.0-1.0) - `house/batcontrol/always_allow_discharge_limit_percent` - Always discharge limit in % - `house/batcontrol/always_allow_discharge_limit_capacity` - Always discharge limit in Wh - `house/batcontrol/max_charging_from_grid_limit` - Max charging from grid limit (0.0-1.0) - `house/batcontrol/max_charging_from_grid_limit_percent` - Max charging from grid limit in % - `house/batcontrol/min_grid_charge_soc` - Optional minimum grid-charge target (0.0-1.0) - `house/batcontrol/min_grid_charge_soc_percent` - Optional minimum grid-charge target in % - `house/batcontrol/production_offset` - Production offset multiplier (`1.0` = 100%, `0.8` = 80%, etc.) ### Peak Shaving See [Peak Shaving](https://mastr.github.io/batcontrol/features/peak-shaving/index.md) for details: - `house/batcontrol/peak_shaving/enabled` - Whether peak shaving is enabled (`true`/`false`) - `house/batcontrol/peak_shaving/mode` - Active mode (`time`, `price`, or `combined`) - `house/batcontrol/peak_shaving/allow_full_battery_after` - Target hour (0-23) - `house/batcontrol/peak_shaving/charge_limit` - Current charge limit in W (`-1` = inactive / no limit) - `house/batcontrol/peak_shaving/price_limit` - Price threshold in EUR/kWh ### Price Information - `house/batcontrol/min_price_difference` - Minimum price difference in EUR (e.g., `0.050`) - `house/batcontrol/min_price_difference_rel` - Relative minimum price difference (e.g., `0.100`) - `house/batcontrol/min_dynamic_price_difference` - Dynamic price difference limit in EUR ### Forecasts (JSON Arrays) - `house/batcontrol/FCST/production` - Forecasted solar production in W - `house/batcontrol/FCST/consumption` - Forecasted consumption in W - `house/batcontrol/FCST/prices` - Forecasted electricity prices in EUR - `house/batcontrol/FCST/net_consumption` - Forecasted net consumption in W ### Inverter-Specific Topics (per inverter, e.g., inverter 0) - `house/batcontrol/inverters/0/SOC` - Inverter SOC in % - `house/batcontrol/inverters/0/stored_energy` - Stored energy in Wh - `house/batcontrol/inverters/0/free_capacity` - Free capacity in Wh - `house/batcontrol/inverters/0/max_capacity` - Maximum capacity in Wh - `house/batcontrol/inverters/0/usable_capacity` - Usable capacity in Wh - `house/batcontrol/inverters/0/max_grid_charge_rate` - Max grid charge rate in W - `house/batcontrol/inverters/0/max_pv_charge_rate` - Max PV charge rate in W - `house/batcontrol/inverters/0/min_soc` - Minimum SOC setting - `house/batcontrol/inverters/0/max_soc` - Maximum SOC setting - `house/batcontrol/inverters/0/capacity` - Total capacity in Wh - `house/batcontrol/inverters/0/em_mode` - Energy Manager mode (Fronius specific) - `house/batcontrol/inverters/0/em_power` - Energy Manager power setting in W (Fronius specific) ## Command Topics (Input API) Batcontrol listens to the following `/set` topics for remote control: ### Main Control - `house/batcontrol/mode/set` - Set operational mode (send `-1`, `0`, `8`, or `10`) - `house/batcontrol/charge_rate/set` - Set charge rate in W (automatically sets mode to `-1`) - `house/batcontrol/limit_battery_charge_rate/set` - Set dynamic battery charge rate limit in W ### Configuration - `house/batcontrol/always_allow_discharge_limit/set` - Set always discharge limit (0.0-1.0) - `house/batcontrol/max_charging_from_grid_limit/set` - Set max charging from grid limit (0.0-1.0) - `house/batcontrol/min_price_difference/set` - Set minimum price difference in EUR - `house/batcontrol/min_price_difference_rel/set` - Set relative minimum price difference (e.g. `0.10` for 10%) - `house/batcontrol/production_offset/set` - Set production offset multiplier (0.0-2.0) ### Peak Shaving - `house/batcontrol/peak_shaving/enabled/set` - Enable or disable peak shaving (`true`/`false`) - `house/batcontrol/peak_shaving/mode/set` - Set mode (`time`, `price`, or `combined`) - `house/batcontrol/peak_shaving/allow_full_battery_after/set` - Set target hour (0-23) - `house/batcontrol/peak_shaving/price_limit/set` - Set price threshold in EUR/kWh (`-1` disables the price component) All `/set` changes are temporary runtime overrides and are not written back to the configuration file. ### Inverter Control (per inverter, e.g., inverter 0) - `house/batcontrol/inverters/0/max_grid_charge_rate/set` - Set max grid charge rate in W - `house/batcontrol/inverters/0/max_pv_charge_rate/set` - Set max PV charge rate in W - `house/batcontrol/inverters/0/em_mode/set` - Set Energy Manager mode (Fronius: 0-2) - `house/batcontrol/inverters/0/em_power/set` - Set Energy Manager power in W (Fronius specific) ### Testdriver/Dummy Inverter (for testing) - `house/batcontrol/inverters/0/SOC/set` - Set SOC manually (0-100, testdriver only) ## Forecast Data Format The forecast topics (`/FCST/*`) publish JSON data with the following structure: ``` { "data": [ { "time_start": 1696435200, "value": 2500.5, "time_end": 1696438800 }, { "time_start": 1696438800, "value": 3200.0, "time_end": 1696442400 } ] } ``` Where: - `time_start` - Unix timestamp for start of hour - `time_end` - Unix timestamp for end of hour - `value` - Forecasted value (W for production/consumption, EUR for prices) ## Example Configurations ### Basic Setup (No Authentication) ``` mqtt: enabled: true broker: 192.168.1.100 port: 1883 topic: energy/batcontrol ``` ### With Authentication ``` mqtt: enabled: true broker: mqtt.example.com port: 1883 topic: home/energy/batcontrol username: batcontrol_user password: secure_password_here retry_attempts: 3 retry_delay: 5 ``` ### Home Assistant Integration ``` mqtt: enabled: true broker: homeassistant.local port: 1883 topic: batcontrol username: mqtt_user password: mqtt_password auto_discover_enable: true auto_discover_topic: homeassistant ``` ## Troubleshooting ### Common Issues 1. **Connection Failed** 1. Check broker hostname/IP and port 1. Verify network connectivity 1. Check username/password if authentication is enabled 1. **Messages Not Appearing** 1. Verify the topic configuration 1. Check broker logs for rejected messages 1. Ensure proper permissions for the MQTT user 1. **Home Assistant Auto-Discovery Not Working** 1. Verify `auto_discover_enable: true` 1. Check that Home Assistant MQTT integration is configured 1. Ensure the discovery topic matches Home Assistant configuration ### Debug Logging Enable MQTT logging for troubleshooting: ``` mqtt: enabled: true logger: true # Enable debug logging ``` This will provide detailed information about MQTT connections, published messages, and any errors in the batcontrol log files. ## Security Considerations - Always use authentication (`username`/`password`) in production - TLS encryption is currently not functional (see above) — keep MQTT traffic on a trusted local network - Limit MQTT user permissions to only necessary topics - Use strong, unique passwords for MQTT authentication # MQTT Inverter The MQTT Inverter driver enables batcontrol to integrate with any battery/inverter system via MQTT topics. It acts as a generic bridge, allowing external systems to provide battery state information and receive control commands over MQTT. ## Architecture Overview The MQTT inverter driver uses batcontrol's shared MQTT connection (configured in the main batcontrol MQTT API section). It does NOT create a separate MQTT client. This design ensures: - Single MQTT connection per batcontrol instance - Consistent MQTT broker configuration - Shared connection pool and resources - Unified logging and error handling ## Topic Structure All topics follow the pattern: `/inverters/$num/` Where: - `` is the MQTT base topic from your main MQTT configuration (the `topic` key in the `mqtt` section) - `$num` is the inverter number (e.g., 0, 1, 2), **not** the literal string "$num" - `` is the specific status or command topic **Example:** If your base topic is "batcontrol" and inverter number is 0, topics would be: - `batcontrol/inverters/0/status/soc` - `batcontrol/inverters/0/command/mode` ## Status Topics (Inverter → batcontrol) These topics **MUST be published as RETAINED** by your external inverter/bridge system: | Topic | Description | Type | Retention | | ------------------------------- | ------------------------------ | ----- | ----------------------- | | `/status/capacity` | Battery capacity in Wh | float | **RETAINED** (required) | | `/status/min_soc` | Minimum SoC limit in % (0-100) | float | **RETAINED** (optional) | | `/status/max_soc` | Maximum SoC limit in % (0-100) | float | **RETAINED** (optional) | | `/status/max_charge_rate` | Maximum charge rate in W | float | **RETAINED** (optional) | These topics should be **updated at least every 2 minutes** to ensure fresh data: | Topic | Description | Type | Retention | | ------------------- | ------------------------------------ | ----- | --------------------------------- | | `/status/soc` | Current State of Charge in % (0-100) | float | Non-retained (updated frequently) | ## Command Topics (batcontrol → Inverter) These topics are published by batcontrol and **MUST NOT be retained**: | Topic | Description | Values | | ---------------------------- | -------------------- | ---------------------------------------------------- | | `/command/mode` | Set operating mode | `force_charge`, `allow_discharge`, `avoid_discharge` | | `/command/charge_rate` | Set charge rate in W | float | ## Why Retention Matters ⚠️ **Critical for proper operation:** - **Status topics MUST be RETAINED** so batcontrol can read the current state immediately on startup - **Command topics MUST NOT be retained** to avoid re-executing stale commands on reconnect - If command topics are retained, the inverter may execute old commands after restart, causing unexpected behavior ## Configuration ### Main MQTT Connection Configure the MQTT connection in batcontrol's main MQTT API section (not in the inverter configuration): ``` mqtt: broker: 192.168.1.100 port: 1883 username: batcontrol password: secret topic: batcontrol # Base topic for all MQTT messages ``` ### Inverter Configuration ``` inverter: type: mqtt capacity: 10000 # Battery capacity in Wh (required) min_soc: 5 # Minimum SoC % (default: 5) max_soc: 100 # Maximum SoC % (default: 100) max_grid_charge_rate: 5000 # Maximum charge rate in W (required) cache_ttl: 120 # Cache TTL for SOC values in seconds (default: 120) base_topic: batcontrol/inverters/0 # Optional: override default topic structure ``` ### Configuration Parameters | Parameter | Required | Default | Description | | ---------------------- | -------- | ------------------------------ | -------------------------------------------- | | `type` | Yes | - | Must be `mqtt` | | `capacity` | Yes | - | Battery capacity in Wh | | `max_grid_charge_rate` | Yes | - | Maximum charge rate from grid in W | | `min_soc` | No | 5 | Minimum State of Charge in % | | `max_soc` | No | 100 | Maximum State of Charge in % | | `cache_ttl` | No | 120 | Cache TTL for SOC values in seconds | | `base_topic` | No | `/inverters/` | Custom base topic for inverter MQTT messages | ## External Bridge Requirements Your external system (inverter bridge script, inverter firmware, etc.) must: 1. **Publish battery status as RETAINED messages:** 1. Battery capacity in Wh (required) 1. Optional: min_soc, max_soc, max_charge_rate 1. **Publish current SOC regularly (at least every 2 minutes):** 1. Current State of Charge as a normal message (can be retained or non-retained) 1. **Subscribe to command topics (non-retained):** 1. Mode changes (`force_charge`, `allow_discharge`, `avoid_discharge`) 1. Charge rate adjustments 1. **Handle reconnection gracefully:** 1. Re-publish all status topics as RETAINED on reconnect 1. Don't retain command topics to avoid stale command execution ## Example Bridge Implementation Here's a simple Python example using paho-mqtt to bridge your inverter to batcontrol: ``` import paho.mqtt.client as mqtt import time # Configuration MQTT_BROKER = "192.168.1.100" MQTT_PORT = 1883 MQTT_USER = "batcontrol" MQTT_PASSWORD = "secret" BASE_TOPIC = "batcontrol/inverters/0" def on_connect(client, userdata, flags, rc): """Called when connected to MQTT broker""" print(f"Connected with result code {rc}") # Publish initial state (RETAINED) client.publish(f"{BASE_TOPIC}/status/capacity", "10000", retain=True) client.publish(f"{BASE_TOPIC}/status/min_soc", "5", retain=True) client.publish(f"{BASE_TOPIC}/status/max_soc", "100", retain=True) client.publish(f"{BASE_TOPIC}/status/max_charge_rate", "5000", retain=True) # Subscribe to commands client.subscribe(f"{BASE_TOPIC}/command/#") print(f"Subscribed to {BASE_TOPIC}/command/#") def on_message(client, userdata, message): """Handle incoming commands from batcontrol""" topic = message.topic value = message.payload.decode() print(f"Received: {topic} = {value}") if topic == f"{BASE_TOPIC}/command/mode": print(f"Setting mode to: {value}") # TODO: Implement your inverter control here # Examples: # - force_charge: Enable grid charging # - allow_discharge: Normal operation # - avoid_discharge: Prevent battery discharge elif topic == f"{BASE_TOPIC}/command/charge_rate": print(f"Setting charge rate to: {value}W") # TODO: Implement your charge rate control here def publish_soc(client, soc_value): """Publish current State of Charge""" client.publish(f"{BASE_TOPIC}/status/soc", str(soc_value)) # Create MQTT client client = mqtt.Client() client.username_pw_set(MQTT_USER, MQTT_PASSWORD) client.on_connect = on_connect client.on_message = on_message # Connect to broker client.connect(MQTT_BROKER, MQTT_PORT, 60) # Start network loop in background client.loop_start() # Main loop: Periodically publish SOC try: while True: # TODO: Read actual SOC from your inverter soc = 65.5 # Example value publish_soc(client, soc) time.sleep(60) # Update every 60 seconds except KeyboardInterrupt: print("Shutting down...") client.loop_stop() client.disconnect() ``` ## Operating Modes The MQTT inverter supports three operating modes: ### force_charge Forces the battery to charge from grid at the specified rate. Used during low-price periods. ``` Topic: /command/mode Payload: force_charge Topic: /command/charge_rate Payload: 5000 # Charge at 5000W ``` ### allow_discharge Normal operation mode. Battery can charge from PV and discharge to supply loads. ``` Topic: /command/mode Payload: allow_discharge ``` ### avoid_discharge Prevents battery discharge. Battery can still charge from PV but won't discharge to supply loads. Used to preserve battery for later use. ``` Topic: /command/mode Payload: avoid_discharge ``` ## Home Assistant Integration The MQTT inverter automatically publishes Home Assistant MQTT Discovery messages for all status and command topics. This allows you to monitor your inverter's status and commands in Home Assistant without manual configuration. Discovered entities include: - MQTT Inverter Status SOC - MQTT Inverter Status Capacity - MQTT Inverter Status Min SOC - MQTT Inverter Status Max SOC - MQTT Inverter Status Max Charge Rate - MQTT Inverter Command Mode - MQTT Inverter Command Charge Rate ## Limitations - **No bidirectional acknowledgment:** batcontrol assumes commands succeed immediately - **No auto-discovery:** All topics must follow the documented structure exactly - **Network dependency:** MQTT broker must be reliable and accessible - **Initial state required:** Status topics must be available at batcontrol startup - **Clock synchronization:** Ensure time is synchronized between batcontrol and your inverter system for accurate scheduling - **QoS 1 for commands:** Guarantees delivery but not exactly-once semantics (commands may be delivered multiple times) ## Advanced Configuration ### Custom Topic Structure By default, the MQTT inverter uses the topic structure `/inverters/`, where `` is the base topic from the main MQTT configuration. You can override this: ``` inverter: type: mqtt base_topic: custom/battery/system # Use custom topic structure capacity: 10000 max_grid_charge_rate: 5000 ``` This would result in topics like: - `custom/battery/system/status/soc` - `custom/battery/system/command/mode` Each inverter will have its own set of MQTT topics. ## See Also - [Inverter Configuration](https://mastr.github.io/batcontrol/configuration/inverter-configuration/index.md) - General inverter configuration options - [MQTT API](https://mastr.github.io/batcontrol/integrations/mqtt-api/index.md) - Main MQTT API configuration and topics - [How batcontrol works](https://mastr.github.io/batcontrol/getting-started/how-batcontrol-works/index.md) - Understanding batcontrol's operation # evcc Integration Batcontrol can integrate with [evcc (Electric Vehicle Charging Controller)](https://evcc.io/) to intelligently manage battery usage during electric vehicle charging. This integration helps prevent unnecessary battery discharge while your EV is charging, optimizing your overall energy management. ## How It Works When evcc is charging your electric vehicle, batcontrol can automatically: 1. **Block battery discharge** to prevent the home battery from being used while the EV charges 1. **Temporarily adjust discharge limits** based on evcc's buffer SOC settings 1. **Monitor multiple charging loadpoints** for comprehensive EV charging detection 1. **Restore original settings** when charging stops ## Basic Configuration ``` evcc: enabled: true broker: localhost port: 1883 status_topic: evcc/status loadpoint_topic: - evcc/loadpoints/1/charging - evcc/loadpoints/2/charging block_battery_while_charging: true ``` ### Required Parameters | Parameter | Type | Description | | ----------------- | ----------- | ------------------------------------------------------ | | `enabled` | boolean | Enable or disable evcc integration | | `broker` | string | MQTT broker hostname or IP address (same as evcc uses) | | `port` | integer | MQTT broker port (typically 1883 or 8883 for TLS) | | `status_topic` | string | MQTT topic for evcc online/offline status | | `loadpoint_topic` | list/string | MQTT topic(s) for loadpoint charging status | ### Basic Parameters Explained - **`status_topic`**: Usually `evcc/status` - monitors if evcc is online/offline - **`loadpoint_topic`**: Can be a single string or list of topics like: - `evcc/loadpoints/1/charging` (for loadpoint 1) - `evcc/loadpoints/2/charging` (for loadpoint 2) - Add more loadpoints as needed for your setup ## Advanced Configuration ### Authentication ``` evcc: username: mqtt_user password: mqtt_password ``` ### TLS/SSL Support > ⚠️ **Note**: TLS/SSL support is currently **untested and known to be non-functional** (same limitation as in the [MQTT API](https://mastr.github.io/batcontrol/integrations/mqtt-api/index.md)). Keep MQTT traffic on a trusted local network. ### Battery Management Options ``` evcc: block_battery_while_charging: true battery_halt_topic: evcc/site/bufferSoc ``` | Parameter | Type | Default | Description | | ------------------------------ | ------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `block_battery_while_charging` | boolean | `true` | If `true`: Block battery discharge while EV is charging. If `false`: Battery discharge follows normal batcontrol algorithm regardless of EV charging status | | `battery_halt_topic` | string | *unset (disabled)* | Topic for dynamic discharge limit control, e.g. `evcc/site/bufferSoc` | ## Battery Halt Topic (Advanced) The `battery_halt_topic` enables dynamic battery discharge limit management based on evcc's buffer SOC setting. ### How It Works 1. **Normal Operation**: Batcontrol uses your configured `always_allow_discharge_limit` 1. **EV Charging Starts**: 1. Batcontrol saves current discharge limit 1. Sets new limit based on evcc's `bufferSoc` value 1. **EV Charging Stops**: 1. Restores original discharge limit 1. Returns to normal battery management ### Example Scenario - Your normal `always_allow_discharge_limit`: `0.20` (20%) - evcc `bufferSoc` setting: `50` (50%) - **Result**: While EV charges, battery discharge is blocked above 50% SOC instead of 20% ## MQTT Topics Monitored Batcontrol subscribes to the following evcc MQTT topics: ### Status Monitoring - `evcc/status` - evcc online/offline status (`online`/`offline`) ### Charging Detection - `evcc/loadpoints/1/charging` - Loadpoint 1 charging status (`true`/`false`) - `evcc/loadpoints/2/charging` - Loadpoint 2 charging status (`true`/`false`) - Additional loadpoints as configured ### Loadpoint Mode and Connection State (derived automatically) For every configured `.../charging` topic, batcontrol additionally subscribes to the sibling topics: - `evcc/loadpoints/1/mode` - Loadpoint charging mode (`pv`, `now`, `minpv`, `off`) - `evcc/loadpoints/1/connected` - Whether an EV is connected (`true`/`false`) These are used by [peak shaving](https://mastr.github.io/batcontrol/features/peak-shaving/index.md): peak shaving is automatically disabled while evcc is actively charging or while an EV is connected in PV mode, and re-enabled when the EV disconnects or the mode changes. ### Buffer SOC (Optional) - `evcc/site/bufferSoc` - Dynamic discharge threshold (integer 0-100) ## Behavior During EV Charging ### When Charging Starts 1. **Battery Blocking**: If `block_battery_while_charging: true`, battery discharge is blocked. If `false`, battery discharge continues according to normal batcontrol algorithm 1. **Limit Adjustment**: If `battery_halt_topic` configured, discharge limit is temporarily set to buffer SOC 1. **Logging**: Batcontrol logs: `"evcc is charging, set block"` (only if blocking enabled) ### When Charging Stops 1. **Battery Unblocking**: Battery discharge blocking is removed 1. **Limit Restoration**: Original discharge limit is restored 1. **Logging**: Batcontrol logs: `"evcc is not charging, remove block"` ### When evcc Goes Offline 1. **Safety Mechanism**: If evcc goes offline while charging, blocks are automatically removed 1. **Limit Restoration**: Original settings are restored 1. **Logging**: Batcontrol logs: `"evcc went offline"` and `"evcc was charging, remove block"` ## Example Configurations ### Single Loadpoint Setup ``` evcc: enabled: true broker: 192.168.1.100 port: 1883 status_topic: evcc/status loadpoint_topic: evcc/loadpoints/1/charging block_battery_while_charging: true ``` ### Multiple Loadpoints with Authentication ``` evcc: enabled: true broker: evcc.local port: 1883 status_topic: evcc/status loadpoint_topic: - evcc/loadpoints/1/charging - evcc/loadpoints/2/charging block_battery_while_charging: true username: batcontrol password: secure_password ``` ### Advanced Setup with Buffer SOC ``` evcc: enabled: true broker: mqtt.home.local port: 1883 status_topic: evcc/status loadpoint_topic: - evcc/loadpoints/1/charging block_battery_while_charging: true battery_halt_topic: evcc/site/bufferSoc username: mqtt_user password: mqtt_pass ``` ### Monitoring Only (No Battery Blocking) ``` evcc: enabled: true broker: localhost port: 1883 status_topic: evcc/status loadpoint_topic: - evcc/loadpoints/1/charging block_battery_while_charging: false # Battery discharge follows normal batcontrol algorithm ``` **Use Case**: This configuration allows you to monitor EV charging status without affecting battery discharge behavior. The battery will charge/discharge according to batcontrol's normal price-based algorithm, regardless of whether the EV is charging. ## Troubleshooting ### Common Issues 1. **Connection Failed** 1. Verify evcc MQTT broker settings match batcontrol configuration 1. Check network connectivity between batcontrol and MQTT broker 1. Ensure MQTT credentials are correct 1. **Charging Not Detected** 1. Verify loadpoint topic names match your evcc configuration 1. Check evcc MQTT API is enabled and publishing messages 1. Use MQTT client to monitor topics: `mosquitto_sub -h localhost -t evcc/+/+` 1. **Buffer SOC Not Working** 1. Ensure `battery_halt_topic` matches evcc's bufferSoc topic 1. Verify evcc is publishing bufferSoc values 1. Check logs for: `"Enabling battery threshold management"` ### Debug Logging Enable detailed logging for troubleshooting: ``` evcc: enabled: true # ... other config ... logger: true # Enable MQTT debug logging ``` ### Log Messages to Watch For - `"evcc is online"` - evcc status detection working - `"Loadpoint evcc/loadpoints/1/charging is charging"` - charging detection - `"evcc is charging, set block"` - battery blocking activated - `"Enabling battery threshold management"` - buffer SOC feature active - `"New battery_halt value: 50"` - buffer SOC updated ## Integration with Home Assistant When using both batcontrol and evcc with Home Assistant: 1. Use the same MQTT broker for all three systems 1. Configure evcc auto-discovery: `homeassistant` topic 1. Configure batcontrol MQTT auto-discovery for the same topic 1. Both systems will create entities in Home Assistant automatically ## Security Considerations - Use authentication for production MQTT brokers - TLS encryption is currently not functional (see above) — keep MQTT traffic on a trusted local network - Ensure MQTT user has appropriate topic permissions - Keep MQTT credentials secure and unique # Forecast Metrics `ForecastMetrics` is a stateless module that derives three battery indicators from the production/consumption forecast arrays and the current battery state. These indicators are published via MQTT and are intended to drive downstream automation decisions such as "should I run the heat pump now or save the battery for tomorrow's solar charge?" All three values are updated once per evaluation cycle (every 3 minutes by default). They are based on the same forecast window that the main optimizer uses, so their horizon is `min(max available price hours, max available solar hours)`. ## The Three Metrics ### `solar_surplus_wh` — Current-Window Solar Overflow **MQTT topic:** `{base}/solar_surplus_wh` Expected energy in Wh that the solar production in the **current production window** will generate above what the battery can absorb. - **solar_active = true (slot 0 is producing):** surplus is the net solar production of the ongoing window minus the remaining free battery capacity. `surplus > 0` means that PV power will be exported to the grid even if the battery is managed optimally. - **solar_active = false (nighttime or break before next window):** surplus accounts for the overnight discharge first — the battery will self-discharge from consumption before solar starts, which creates room. `surplus > 0` means even after that extra room is created the next solar window will still overflow. A value of `0` means the battery can absorb everything the upcoming solar window produces. A value `> 0` means some PV will inevitably be exported; it is safe to run flexible loads (heat pump, EV charging) from the grid right now because the solar surplus will offset them. **Use case:** If `solar_surplus_wh >= estimated_load_wh`, running a flexible load (heat pump, EV charging via evcc) has no net grid cost over the forecast horizon. See [Use Cases](#use-cases) for concrete automation examples. ______________________________________________________________________ ### `pv_start_battery_wh` — Battery Level at Next Charging Point **MQTT topic:** `{base}/pv_start_battery_wh` Battery level in Wh (above MIN_SOC) at the moment when solar production first exceeds household consumption (`net_consumption < 0`). This is the point where the battery transitions from discharging to charging. - Simulated slot-by-slot from the current moment forward. - If the battery hits `0` (MIN_SOC) before solar starts, the value is `0`. - If the battery has no net-charging slot in the forecast at all, the value is `0`. - If slot 0 is already a net-charging slot (solar already exceeds consumption), the value equals the current stored usable energy. **Use case:** "How much charge will the battery have left when solar starts tomorrow?" A low value (e.g. < 500 Wh) means flexible loads tonight should be reduced; a high value means overnight loads can run freely. See [Use Cases](#use-cases) for heat pump and evcc automation examples. Note `pv_start_battery_wh` depends on `net_consumption < 0`, not just on `production > 0`. The battery does not switch to charging until solar output exceeds household consumption — on a partly-cloudy morning the cross-over can happen later than sunrise. ______________________________________________________________________ ### `forecast_min_battery_wh` — Forecast Minimum Battery Level **MQTT topic:** `{base}/forecast_min_battery_wh` The lowest battery level in Wh (above MIN_SOC) reached at any point during the entire forecast horizon, based on slot-by-slot simulation with proper floor/ceiling clamping. - A value of `0` means the battery is expected to hit MIN_SOC at some point in the forecast — a signal that the system will be energy-constrained. - The simulation respects both the floor (MIN_SOC = 0 usable Wh) and the ceiling (MAX_SOC = stored_usable + free_capacity), so multi-day charge/discharge cycles are tracked correctly. - The horizon covers the full forecast window (same as the optimizer), not just the next 24 hours. **Use case:** "Will the battery run out at any point in the planning horizon?" `forecast_min_battery_wh == 0` means batcontrol may need to grid-charge later — block or reduce flexible loads. A value comfortably above zero means the battery has buffer and loads can run. See [Use Cases](#use-cases). ______________________________________________________________________ ## Decision Matrix The three metrics form a natural 2-D decision space for flexible load control: | `solar_surplus_wh` | `forecast_min_battery_wh` | Recommended action | | ------------------ | ------------------------- | -------------------------------------------------------------------------- | | > 0 | > 0 | Run flexible loads freely — PV will cover them and battery stays healthy | | > 0 | = 0 | PV surplus exists but battery will be short later — run light loads only | | = 0 | > 0 | No surplus but battery OK — use `pv_start_battery_wh` to judge night loads | | = 0 | = 0 | Constrained — block flexible loads, preserve battery | `pv_start_battery_wh` refines the third row: if it is high, the overnight discharge is gentle and a moderate flexible load (e.g. heat pump one cycle) is fine. If it is near zero, defer to the next solar window. ______________________________________________________________________ ## MQTT Topics | Topic | Unit | Retained | Description | | -------------------------------- | ---- | -------- | -------------------------------------------------- | | `{base}/solar_surplus_wh` | Wh | No | PV overflow that cannot be stored in the battery | | `{base}/solar_active` | bool | No | `true` if solar is producing in slot 0 | | `{base}/pv_start_battery_wh` | Wh | No | Battery level at next net-charging crossover | | `{base}/forecast_min_battery_wh` | Wh | No | Minimum battery level over entire forecast horizon | All values are published after each evaluation cycle (together with the inverter control decision). See [MQTT API](https://mastr.github.io/batcontrol/integrations/mqtt-api/index.md) for the full topic reference and configuration options. ### Home Assistant Auto-Discovery The following HA entities are created automatically when `auto_discover_enable: true` is configured: - **Solar Surplus** — sensor (energy, Wh) - **Solar Active** — binary sensor (on/off diagnostic) - **PV Start Battery** — sensor (energy, Wh) - **Forecast Min Battery** — sensor (energy, Wh) ______________________________________________________________________ ## Use Cases ### Heat Pump Control A heat pump is a flexible load: it can pre-heat a buffer tank or run an extra heating cycle when energy is cheap or free — but running it at the wrong time can deplete the battery before the next solar window. The three metrics together answer the key questions for heat pump automation: **"Can I run a heating cycle right now without net grid cost?"** Check `solar_surplus_wh >= estimated_cycle_wh`. If yes, the PV will produce more than the battery can store anyway — running the heat pump consumes what would otherwise be exported. No additional grid draw over the forecast horizon. **"Is the battery safe enough for a cycle tonight?"** Check `pv_start_battery_wh`. A high value (e.g. > 2000 Wh) means the battery will still have a comfortable charge when solar starts tomorrow; the heat pump can run. A low value (< 500 Wh) means overnight consumption will nearly deplete the battery — defer to the next solar window. **"Should I block flexible loads entirely?"** Check `forecast_min_battery_wh == 0`. If the battery is expected to hit MIN_SOC at some point in the forecast, batcontrol may need to grid-charge later; flexible loads should wait. A simple Home Assistant automation combining all three: ``` # Allow heat pump if PV will overflow OR battery is healthy and not forecast-constrained condition: - condition: or conditions: - condition: numeric_state entity_id: sensor.batcontrol_solar_surplus_wh above: 1500 # surplus covers one heat pump cycle - condition: and conditions: - condition: numeric_state entity_id: sensor.batcontrol_pv_start_battery_wh above: 1000 # enough battery charge at dawn - condition: numeric_state entity_id: sensor.batcontrol_forecast_min_battery_wh above: 0 # battery not forecast to run empty ``` ______________________________________________________________________ ### EV Charging via evcc [evcc](https://evcc.io) supports a `min_soc`/`target_soc` model as well as charging from PV surplus. The batcontrol metrics integrate naturally with evcc's MQTT API to let you charge the car only when it does not compete with the battery. **Scenario: charge only from true PV overflow** `solar_surplus_wh` tells you exactly how much energy the PV will produce above what the battery can absorb. Pass this value to evcc's `pv_action` or use it in an automation to set the evcc charging mode: - `solar_surplus_wh > 0` → switch evcc to **PV** mode (charge from surplus) - `solar_surplus_wh == 0` and `forecast_min_battery_wh > 0` → switch to **Min+PV** (keep a minimum charge rate, fill up with PV where possible) - `forecast_min_battery_wh == 0` → switch to **Off** or **Min** only (battery needs the energy) **Scenario: opportunistic overnight charge** Use `pv_start_battery_wh` to decide whether to allow evcc to draw from the battery overnight. If the battery is forecast to still be above a threshold at dawn, a slow overnight charge (e.g. 6 A / 1.4 kW) will not noticeably affect the next day's solar cycle. ______________________________________________________________________ ### General Pattern for Any Flexible Load | Question | Metric to check | Threshold example | | ------------------------------------ | ------------------------- | --------------------- | | Is PV overflowing right now? | `solar_surplus_wh` | `> estimated_load_wh` | | Will battery survive the night? | `pv_start_battery_wh` | `> 500 Wh` | | Is the battery forecast-constrained? | `forecast_min_battery_wh` | `> 0` | All three are dimensioned in Wh, so you can directly compare them against the energy consumption of the load you want to schedule. ______________________________________________________________________ ## Implementation Notes - All values are computed in `src/batcontrol/forecast_metrics.py` by the `ForecastMetrics` class. - Slot 0 is time-adjusted: the elapsed fraction of the current interval is subtracted so that a slot already 80% elapsed only contributes 20% of its forecast energy. - `net_consumption = consumption - production`; negative = battery charging, positive = battery discharging / grid draw. - `stored_usable_energy` is the energy above MIN_SOC. `free_capacity` is the space between the current level and MAX_SOC. - The slot-by-slot simulation clamps at both ends: `battery = max(0, min(stored_usable + free_capacity, battery - net))`. A simple net-sum over slots would overestimate available energy because it ignores that the battery cannot go below 0 or above MAX_SOC. # Optional # 15-Minute Interval Transformation Analysis ## Executive Summary This document analyzes the required changes to transform the batcontrol system from hourly (60-minute) to 15-minute time intervals. The transition involves modifications across forecast providers, core logic, MQTT API, and data structures. **Key Finding**: Making the interval configurable (15 or 60 minutes) is **highly recommended** to maintain flexibility and backward compatibility during migration. ______________________________________________________________________ ## Current Architecture Overview ### Time Resolution Comparison ``` CURRENT (Hourly): Timeline: |----Hour 0----|----Hour 1----|----Hour 2----| Intervals: 0 1 2 3 Data points: 48 (for 48 hours) Array size: ~2 KB per forecast PROPOSED (15-minute): Timeline: |--15m--|--15m--|--15m--|--15m--| (= 1 hour) Intervals: 0 1 2 3 4 5 6 7 8 9 ... Data points: 192 (for 48 hours) Array size: ~8 KB per forecast Evaluation: Every 3 minutes (unchanged) ``` ### Data Flow Diagram ``` ┌─────────────────────────────────────────────────────────────┐ │ BATCONTROL CORE │ │ (Evaluation Every 3 min) │ └─────────────────────────────────────────────────────────────┘ │ ├──> Config: time_resolution_minutes │ ┌───────────────────┼───────────────────┐ │ │ │ ▼ ▼ ▼ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │Solar Forecast│ │Consumption │ │Dynamic Tariff│ │ │ │Forecast │ │ │ ├──────────────┤ ├──────────────┤ ├──────────────┤ │ FCSolar: │ │ CSV Profile: │ │ Awattar: │ │ Hourly │ │ Hourly │ │ Hourly │ │ ↓ Upsample │ │ ↓ Divide │ │ ↓ Repeat │ │ 15-min │ │ 15-min │ │ 15-min │ │ │ │ │ │ │ │ EvccSolar: │ │ Future: │ │ Evcc: │ │ Native │ │ Native │ │ Native │ │ 15-min │ │ 15-min │ │ 15-min │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ └───────────────────┼───────────────────┘ │ ▼ ┌──────────────┐ │Array Merge │ │[0..191] │ └──────────────┘ │ ┌───────────┼───────────┐ │ │ │ ▼ ▼ ▼ ┌──────────┐ ┌──────────┐ ┌──────────┐ │Production│ │Consumption│ │ Prices │ │ Array │ │ Array │ │ Array │ │ [0..191] │ │ [0..191] │ │ [0..191]│ └──────────┘ └──────────┘ └──────────┘ │ ▼ ┌──────────────┐ │ Logic │ │ Calculation │ │ (192 iters) │ └──────────────┘ │ ┌───────────┴───────────┐ │ │ ▼ ▼ ┌──────────────┐ ┌──────────────┐ │ Inverter │ │ MQTT Publish │ │ Control │ │ (Telegraf) │ │ (charge_rate)│ │ ↓ │ └──────────────┘ │ InfluxDB │ │ ↓ │ │ Grafana │ └──────────────┘ ``` ### Decision Tree: Which Interval to Choose? ``` Do you have dynamic electricity pricing? │ ├─ NO ───> Use 60 min (hourly sufficient) │ └─ YES │ Does your tariff change every 15 minutes? │ ├─ YES ───> Use 15 min (required) │ └─ NO (hourly) ───> Use 60 min OR 15 min └─> 15 min gives more responsive charging Hardware considerations: ├─ Raspberry Pi 3 ──> Start with 60 min ├─ Raspberry Pi 4+ ─> Use 15 min └─ Server/Desktop ──> Use 15 min Network considerations: ├─ Slow/metered ──> Use 60 min (less MQTT data) └─ Fast/unlimited ─> Use 15 min ``` ### Time Resolution - **Core Interval**: 60 minutes (hourly) - **Evaluation Frequency**: Every 3 minutes (configurable via `EVALUATIONS_EVERY_MINUTES`) - **Data Structure**: Arrays indexed by hour (0-48 for forecasts) - **API Updates**: Every 15 minutes (`TIME_BETWEEN_UTILITY_API_CALLS = 900`) ### Current Workflow 1. Forecasts are fetched hourly for production, consumption, and prices 1. Arrays are indexed by `hour` (relative hours from now) 1. Logic calculates based on hourly energy values (Wh) 1. Charge rates are calculated based on remaining time in current hour 1. MQTT publishes forecast data with hourly timestamps ______________________________________________________________________ ## Required Changes by Component ### 1. Core Module (`src/batcontrol/core.py`) #### Critical Design Decision: Full-Hour Alignment **Problem**: Index misalignment between provider data and current time causes inconsistencies. **Example of the issue**: ``` Time: 10:20 (20 minutes into hour) Provider returns (hour-aligned): [0] = 10:00-10:15 (250 Wh) - ALREADY PASSED [1] = 10:15-10:30 (300 Wh) - CURRENT interval (5 min elapsed) [2] = 10:30-10:45 (350 Wh) [3] = 10:45-11:00 (400 Wh) Core.py expects (time-aligned): [0] = current interval = 10:15-10:30 [1] = next interval = 10:30-10:45 etc. If core.py factorizes [0] thinking it's the current interval: 250 * (1 - 20/15) = NEGATIVE! ❌ ``` **Solution**: Providers always return **full-hour aligned data**, and Core.py handles the offset: #### Changes Required: ``` # Lines 347-351: Time correction for current interval # CURRENT (hourly): production[0] *= 1 - datetime.datetime.now().astimezone(self.timezone).minute/60 consumption[0] *= 1 - datetime.datetime.now().astimezone(self.timezone).minute/60 # PROPOSED (configurable with proper indexing): now = datetime.datetime.now().astimezone(self.timezone) current_minute = now.minute current_second = now.second # Find which interval we're in within the current hour current_interval_in_hour = current_minute // interval_minutes # 0, 1, 2, or 3 for 15-min # Calculate elapsed time in the CURRENT interval elapsed_in_current = (current_minute % interval_minutes + current_second / 60) / interval_minutes # Provider data is hour-aligned, so we need to adjust the index # The current interval is at index [current_interval_in_hour] if len(production) > current_interval_in_hour: production[current_interval_in_hour] *= (1 - elapsed_in_current) consumption[current_interval_in_hour] *= (1 - elapsed_in_current) # For MQTT publishing and logic calculation, we work from the START of current hour # but for actual control decisions, we use only future intervals ``` **Alternative Approach** (Cleaner): ``` # Option: Providers return data starting from CURRENT interval (not full hour) # This requires providers to calculate the starting interval themselves def get_forecast(self, intervals: int) -> Dict[int, float]: """ Returns forecast starting from CURRENT interval. Returns: Dict where [0] = current interval, [1] = next, etc. """ now = datetime.datetime.now().astimezone(self.timezone) current_interval_in_hour = now.minute // self.target_resolution # Fetch full hour data full_hour_data = self._fetch_forecast() # Shift to start from current interval shifted_data = {} for idx, value in full_hour_data.items(): if idx >= current_interval_in_hour: shifted_data[idx - current_interval_in_hour] = value return shifted_data # Then core.py simply factorizes [0]: elapsed_in_current = (now.minute % interval_minutes + now.second / 60) / interval_minutes production[0] *= (1 - elapsed_in_current) consumption[0] *= (1 - elapsed_in_current) ``` #### Design Choice: Data Alignment Strategy Two approaches to handle the timing/indexing issue: **Approach A: Full-Hour Alignment (Provider-centric)** - ✅ Providers return data aligned to hour boundaries (simpler provider implementation) - ✅ Good for MQTT publishing (always starts at hour boundary) - ❌ Core.py must track offset and adjust indexing - ❌ More complex factorization logic - **Use case**: When you want MQTT data to always show full hours **Approach B: Current-Interval Alignment (Core-centric)** ⭐ RECOMMENDED - ✅ Provider shifts data so [0] = current interval (matches core.py expectations) - ✅ Core.py logic is simpler (just factorize [0]) - ✅ No index offset tracking needed - ❌ Providers need to calculate current interval position - ❌ MQTT data starts from "now" not hour boundary - **Use case**: For control logic (most important use case) **Recommendation**: Use **Approach B** (Current-Interval Alignment) because: 1. **Control logic is primary concern** - we need accurate charge decisions NOW 1. **Simpler core.py** - less error-prone 1. **Baseclass handles complexity** - providers stay simple 1. **MQTT can round to hour** - if needed for display purposes **Implementation** (in baseclass): ``` class ForecastSolarBase(ABC): def get_forecast(self, intervals: int = None) -> Dict[int, float]: """ Get forecast starting from CURRENT interval. Key behavior: - Returns [0] = current interval (e.g., 10:15-10:30 if now is 10:20) - Returns [1] = next interval (e.g., 10:30-10:45) - Provider data is hour-aligned, baseclass shifts indices """ # Fetch data at native resolution (hour-aligned) native_data = self._fetch_forecast() # Convert resolution if needed if self.native_resolution != self.target_resolution: native_data = self._convert_resolution(native_data) # Shift indices to start from current interval now = datetime.datetime.now().astimezone(self.timezone) current_interval_in_hour = now.minute // self.target_resolution shifted_data = {} for idx, value in native_data.items(): if idx >= current_interval_in_hour: shifted_data[idx - current_interval_in_hour] = value return shifted_data # [0] = current interval, ready for core.py ``` #### Array Handling: - **Current**: Arrays sized for 48+ hours (indices 0-48) - **15-min**: Arrays sized for 192+ intervals (4 × 48 hours) - **Index [0]**: Always represents the CURRENT interval (not start of hour) - **Recommendation**: Use `interval_count` variable: `hours * (60 / interval_minutes)` #### Configuration: Add new parameter to `batcontrol_config_dummy.yaml`: ``` general: time_resolution_minutes: 15 # Options: 15, 60 ``` ______________________________________________________________________ ### 2. Solar Forecast Providers (`src/batcontrol/forecastsolar/`) #### Architecture Pattern: Baseclass with Automatic Upsampling **Design Philosophy**: Instead of each provider implementing upsampling logic, use a **baseclass pattern** where: 1. Each provider declares its **native resolution** via attribute 1. Baseclass handles **automatic upsampling/downsampling** 1. Provider focuses solely on **data fetching** This approach: - ✅ Eliminates code duplication - ✅ Centralizes upsampling logic (easier to maintain/test) - ✅ Supports dynamic resolution switching (e.g., Tibber API) - ✅ Consistent behavior across all providers #### Implementation Pattern: ``` # src/batcontrol/forecastsolar/baseclass.py from abc import ABC, abstractmethod from typing import Dict, Literal import datetime import logging logger = logging.getLogger(__name__) class ForecastSolarBase(ABC): """ Base class for solar forecast providers with automatic resolution handling. Key Design: Providers return FULL-HOUR aligned data, baseclass shifts to CURRENT interval. Subclasses must: 1. Set self.native_resolution (15 or 60) in __init__ 2. Implement _fetch_forecast() to return hour-aligned data Example at 10:20: Provider returns: {0: val_10:00, 1: val_10:15, 2: val_10:30, ...} (hour-aligned) get_forecast() returns: {0: val_10:15, 1: val_10:30, ...} (current-aligned) """ def __init__(self, config: dict): self.config = config self.target_resolution = config.get('general', {}).get('time_resolution_minutes', 60) self.native_resolution = 60 # Override in subclass self.timezone = config.get('general', {}).get('timezone', 'UTC') @abstractmethod def _fetch_forecast(self) -> Dict[int, float]: """ Fetch forecast data at native resolution, HOUR-ALIGNED. Returns: Dict mapping interval index to energy value (Wh) Index 0 = start of current hour (e.g., 10:00 if now is 10:20) Index 1 = next interval in hour etc. Note: Baseclass will shift indices so [0] = current interval """ pass def get_forecast(self, intervals: int = None) -> Dict[int, float]: """ Get forecast at target resolution, CURRENT-INTERVAL aligned. Args: intervals: Number of intervals to forecast (optional) Returns: Dict where [0] = current interval, [1] = next, etc. Ready for core.py to factorize [0] based on elapsed time """ # Fetch data at native resolution (hour-aligned) native_data = self._fetch_forecast() if not native_data: logger.warning(f"{self.__class__.__name__}: No data returned from API") return {} # Convert resolution if needed converted_data = native_data if self.native_resolution != self.target_resolution: if self.native_resolution == 60 and self.target_resolution == 15: logger.debug(f"{self.__class__.__name__}: Upsampling 60min -> 15min") from ..interval_utils import upsample_forecast converted_data = upsample_forecast(native_data, self.target_resolution, method='linear') elif self.native_resolution == 15 and self.target_resolution == 60: logger.debug(f"{self.__class__.__name__}: Downsampling 15min -> 60min") converted_data = self._downsample_to_hourly(native_data) else: logger.error(f"{self.__class__.__name__}: Cannot convert " f"{self.native_resolution}min -> {self.target_resolution}min") return native_data # Shift indices to start from CURRENT interval now = datetime.datetime.now(datetime.timezone.utc).astimezone( datetime.timezone(datetime.timedelta(hours=0))) # Use configured timezone current_interval_in_hour = now.minute // self.target_resolution logger.debug(f"{self.__class__.__name__}: Shifting from hour-aligned to current interval " f"(offset: {current_interval_in_hour} intervals)") shifted_data = {} for idx, value in converted_data.items(): if idx >= current_interval_in_hour: new_idx = idx - current_interval_in_hour shifted_data[new_idx] = value return shifted_data # [0] = current interval def _downsample_to_hourly(self, data_15min: Dict[int, float]) -> Dict[int, float]: """Convert 15-minute intervals to hourly by summing quarters.""" hourly = {} for interval_15, value in data_15min.items(): hour = interval_15 // 4 if hour not in hourly: hourly[hour] = 0 hourly[hour] += value return hourly ``` #### Complete Flow Example: **Scenario**: Current time is **10:20:30** (20 minutes, 30 seconds into hour) **Step 1: Provider fetches data (hour-aligned)** ``` # FCSolar._fetch_forecast() returns: { 0: 250, # 10:00-10:15 (already passed 5.5 min ago) 1: 300, # 10:15-10:30 (current interval, 5.5 min elapsed) 2: 350, # 10:30-10:45 (future) 3: 400, # 10:45-11:00 (future) 4: 450, # 11:00-11:15 (future) ... } ``` **Step 2: Baseclass upsamples (if needed)** - Already 15-min in this case, skip **Step 3: Baseclass shifts to current interval** ``` current_interval_in_hour = 20 // 15 = 1 # We're in the 2nd interval (index 1) # Shift: subtract 1 from all indices >= 1 { 0: 300, # Was [1]: 10:15-10:30 (CURRENT interval) 1: 350, # Was [2]: 10:30-10:45 2: 400, # Was [3]: 10:45-11:00 3: 450, # Was [4]: 11:00-11:15 ... } # Index 0 (10:00-10:15) was dropped because it's in the past ``` **Step 4: Core.py receives current-aligned data** ``` production = get_forecast() # Gets shifted data from Step 3 # Factorize [0] for elapsed time in CURRENT interval elapsed_in_current = (20 % 15 + 30/60) / 15 = (5 + 0.5) / 15 = 0.367 production[0] *= (1 - 0.367) # 300 * 0.633 = 190 Wh # This is correct: 10 minutes remain in interval, ~67% of 15 min = 190 Wh ``` **Step 5: MQTT publishing** ``` # For MQTT, we can optionally round timestamps to hour boundaries # or publish from actual current time mqtt_api._create_forecast(production, timestamp=time.time(), interval_minutes=15) # Output timestamps: # 10:20:30 -> round to 10:15:00 (interval start) # Data points at: 10:15, 10:30, 10:45, 11:00, ... ``` **Visual Timeline**: ``` Current time: 10:20:30 ⏰ ↓ Hour: 10:00 10:15 10:30 10:45 11:00 │──────────│──────────│──────────│──────────│ │ 250 Wh │ 300 Wh │ 350 Wh │ 400 Wh │ │ PASSED │ CURRENT │ FUTURE │ FUTURE │ │ │ ⏰ (5.5m) │ │ │ Provider returns (hour-aligned): [0]=250 [1]=300 [2]=350 [3]=400 Baseclass shifts (current-aligned): [0]=300 [1]=350 [2]=400 Core.py factorizes: [0]*0.633 [1] [2] = 190 Wh = 350 Wh = 400 Wh Result: Correct! No past data, current interval properly reduced. ``` **Key Insight**: - Provider thinks in "full hours" (simpler API logic) - Baseclass translates to "current interval" (simpler core.py logic) - Core.py gets data ready to use (no index math needed) - Everyone is happy! ✅ #### Provider Implementations: ##### A. FCSolar (Hourly Native) ``` # src/batcontrol/forecastsolar/fcsolar.py from .baseclass import ForecastSolarBase class FCSolar(ForecastSolarBase): """Forecast.solar provider - returns hourly data.""" def __init__(self, config: dict): super().__init__(config) self.native_resolution = 60 # Declares: "I provide hourly data" # ... rest of init ... def _fetch_forecast(self) -> Dict[int, float]: """Fetch hourly forecast from forecast.solar API.""" # Existing API call logic here # Returns: {0: 1000, 1: 1500, 2: 2000, ...} # Wh per hour response = self._call_api() return self._parse_response(response) ``` ##### B. EvccSolar (15-min Native) ``` # src/batcontrol/forecastsolar/evcc_solar.py from .baseclass import ForecastSolarBase class EvccSolar(ForecastSolarBase): """evcc solar provider - returns 15-minute data.""" def __init__(self, config: dict): super().__init__(config) self.native_resolution = 15 # Declares: "I provide 15-min data" # ... rest of init ... def _fetch_forecast(self) -> Dict[int, float]: """Fetch 15-minute forecast from evcc API.""" # Existing API call logic # Returns: {0: 250, 1: 300, 2: 350, ...} # Wh per 15-min response = self._call_evcc_api() return self._parse_15min_data(response) ``` ##### C. Tibber (Dynamic Resolution) ``` # src/batcontrol/dynamictariff/tibber.py from .baseclass import DynamicTariffBase class Tibber(DynamicTariffBase): """Tibber provider - supports both hourly and 15-min via API parameter.""" def __init__(self, config: dict): super().__init__(config) # Decide native resolution based on target # Tibber API supports both, so fetch at target resolution directly target = config.get('general', {}).get('time_resolution_minutes', 60) if target == 15: self.native_resolution = 15 # Fetch 15-min data from API self.api_resolution = "QUARTER_HOURLY" # Tibber API parameter else: self.native_resolution = 60 # Fetch hourly data from API self.api_resolution = "HOURLY" logger.info(f"Tibber: Configured to fetch {self.native_resolution}-min data") def _fetch_forecast(self) -> Dict[int, float]: """Fetch prices at configured resolution.""" query = self._build_graphql_query(resolution=self.api_resolution) response = self._call_api(query) return self._parse_response(response) ``` #### Benefits Summary: | Aspect | Old Approach | New Baseclass Approach | | -------------------- | ----------------------- | ---------------------- | | **Code Duplication** | Each provider upsamples | Once in baseclass | | **Maintainability** | Update N providers | Update 1 baseclass | | **Testing** | Test upsampling N times | Test once + mock | | **Provider Focus** | Data fetch + transform | Data fetch only | | **Dynamic APIs** | Hard to support | Easy (Tibber example) | | **Consistency** | Risk of differences | Guaranteed consistent | #### Migration Path: **Phase 1**: Create baseclass with upsampling - Implement `ForecastSolarBase` - Implement `DynamicTariffBase` - Implement `ForecastConsumptionBase` - Add `interval_utils.py` with shared upsampling functions **Phase 2**: Migrate providers one by one - Start with simplest (FCSolar, Awattar) - Then complex ones (EvccSolar, Tibber) - Each migration is independent (low risk) **Phase 3**: Remove old upsampling code - Clean up redundant logic - Consolidate tests #### Required Changes: ##### A. Linear Interpolation for Hourly Data **Note**: With the baseclass pattern, this upsampling logic is centralized in `interval_utils.py` and called automatically by the baseclass. Individual providers no longer need to implement this. **CRITICAL**: Energy (Wh) vs Power (W) distinction ``` # This function now lives in src/batcontrol/interval_utils.py # and is used by all baseclass implementations def upsample_hourly_to_15min(hourly_forecast: dict) -> dict: """ Convert hourly Wh forecast to 15-minute intervals with linear interpolation. Important: - Input is Wh per hour (energy values) - Output must be Wh per 15 minutes - Use linear power interpolation, then convert to energy Method: 1. Calculate average power per hour (Wh → W) 2. Interpolate power linearly between hours 3. Convert interpolated power back to energy (W → Wh for 15 min) Example: Hour 0: 1000 Wh → avg power = 1000 W Hour 1: 2000 Wh → avg power = 2000 W 15-min intervals (linear power ramp): [0]: Power = 1000 W → Energy = 1000 * 0.25 = 250 Wh [1]: Power = 1250 W → Energy = 1250 * 0.25 = 312.5 Wh [2]: Power = 1500 W → Energy = 1500 * 0.25 = 375 Wh [3]: Power = 1750 W → Energy = 1750 * 0.25 = 437.5 Wh [4]: Power = 2000 W → Energy = 2000 * 0.25 = 500 Wh (next hour begins) """ forecast_15min = {} max_hour = max(hourly_forecast.keys()) for hour in range(max_hour): current_wh = hourly_forecast.get(hour, 0) next_wh = hourly_forecast.get(hour + 1, 0) # Convert Wh to average W (power) current_power = current_wh # 1 Wh over 1 hour = 1 W average next_power = next_wh # Linear power interpolation across 4 quarters for quarter in range(4): interval_idx = hour * 4 + quarter fraction = quarter / 4 # Interpolate power linearly interpolated_power = current_power + (next_power - current_power) * fraction # Convert power to energy for 15 minutes: P[W] * 0.25[h] = E[Wh] forecast_15min[interval_idx] = interpolated_power * 0.25 return forecast_15min ``` ##### B. Provider Implementation (With Baseclass Pattern) **With the baseclass pattern, providers become much simpler:** **fcsolar.py**: ``` from .baseclass import ForecastSolarBase class FCSolar(ForecastSolarBase): def __init__(self, config: dict): super().__init__(config) self.native_resolution = 60 # Declares native resolution def _fetch_forecast(self) -> dict[int, float]: # Just fetch and return hourly data # Baseclass handles upsampling automatically response = self._call_api() return self._parse_response(response) # Returns {0: 1000, 1: 1500, ...} ``` **solarprognose.py**: ``` from .baseclass import ForecastSolarBase class SolarPrognose(ForecastSolarBase): def __init__(self, config: dict): super().__init__(config) self.native_resolution = 60 # Declares native resolution def _fetch_forecast(self) -> dict[int, float]: # Just fetch and return hourly data # Baseclass handles upsampling automatically response = self._call_api() return self._parse_response(response) # Returns {0: 2000, 1: 2500, ...} ``` **evcc_solar.py**: ``` from .baseclass import ForecastSolarBase class EvccSolar(ForecastSolarBase): def __init__(self, config: dict): super().__init__(config) self.native_resolution = 15 # evcc provides 15-min data def _fetch_forecast(self) -> dict[int, float]: # Return native 15-minute data # Baseclass handles downsampling if target is 60-min response = self._call_evcc_api() return self._parse_15min_data(response) # Returns {0: 250, 1: 300, ...} ``` **Result**: - Each provider is ~20-30 lines instead of ~50-100 lines - No upsampling logic duplicated - Clear declaration of native resolution - Automatic conversion by baseclass - **Automatic index shifting** - providers don't worry about current time ______________________________________________________________________ #### Summary: Solving the Timing Issue **The Problem You Identified** ✅: ``` At 10:20, provider returns hour-aligned data [0]=10:00, [1]=10:15, ... Core.py expects [0] to be current interval (10:15-10:30) Mismatch causes incorrect factorization ``` **The Solution**: 1. **Providers**: Return full-hour aligned data (indices 0-3 for current hour) 1. **Baseclass**: Automatically shifts indices so [0] = current interval 1. **Core.py**: Receives current-aligned data, simple factorization of [0] 1. **MQTT**: Can publish with proper timestamps (rounded to interval starts) **Benefits**: - ✅ No negative factorization values - ✅ Correct energy accounting (no double-counting or missing intervals) - ✅ Providers stay simple (don't track current time) - ✅ Core.py stays simple (assumes [0] = now) - ✅ Consistent across all forecast types (solar, consumption, tariff) ______________________________________________________________________ ### 3. Consumption Forecast (`src/batcontrol/forecastconsumption/`) #### Implementation: Using Baseclass Pattern Following the same pattern as solar forecasts, consumption providers inherit from a baseclass: ``` # src/batcontrol/forecastconsumption/baseclass.py class ForecastConsumptionBase(ABC): """Base class for consumption forecast providers.""" def __init__(self, config: dict): self.config = config self.target_resolution = config.get('general', {}).get('time_resolution_minutes', 60) self.native_resolution = 60 # Override in subclass @abstractmethod def _fetch_forecast(self) -> Dict[int, float]: """Fetch forecast at native resolution.""" pass def get_forecast(self, intervals: int = None) -> Dict[int, float]: """Get forecast with automatic resolution handling.""" native_data = self._fetch_forecast() if self.native_resolution == self.target_resolution: return native_data if self.native_resolution == 60 and self.target_resolution == 15: from ..interval_utils import upsample_forecast return upsample_forecast(native_data, self.target_resolution, method='constant') return native_data ``` #### Current Implementation: - CSV-based load profile with hourly granularity - Structure: `month, weekday, hour, energy` - Native resolution: 60 minutes (hourly) #### Challenge: **No sub-hourly data available** from CSV load profiles #### Solution Approach: **Option Comparison**: | Option | Accuracy | Implementation | Data Required | Timeline | Recommended Phase | | ---------------------- | ------------------ | ------------------ | ---------------------- | ---------- | ----------------- | | A: Simple Division | Low (flat profile) | Easy (1 day) | None | Immediate | Phase 1 ✅ | | B: Enhanced Profiles | High (realistic) | Medium (1-2 weeks) | 15-min historical data | 3-6 months | Phase 2 🎯 | | C: Smart Interpolation | Medium (heuristic) | Hard (2-3 weeks) | Time-of-day patterns | Optional | Phase 3 (future) | **Option A: Simple Division (Recommended for Phase 1)** ``` def get_forecast(self, intervals): """ Get forecast for specified number of intervals. For 15-min intervals, divides hourly values by 4. """ t0 = datetime.datetime.now().astimezone(self.timezone) prediction = {} if self.interval_minutes == 15: # Calculate how many hours we need hours_needed = math.ceil(intervals / 4) for h in range(hours_needed): delta_t = datetime.timedelta(hours=h) t1 = t0 + delta_t # Get hourly energy energy_hour = df.loc[df['hour'] == t1.hour].loc[ df['month'] == t1.month].loc[ df['weekday'] == t1.weekday()]['energy'].median() if math.isnan(energy_hour): energy_hour = df['energy'].median() # Distribute equally across 4 quarters energy_quarter = energy_hour * self.scaling_factor / 4 for quarter in range(4): interval_idx = h * 4 + quarter if interval_idx < intervals: prediction[interval_idx] = energy_quarter else: # Keep existing hourly logic ... return prediction ``` **Option B: Sub-hour Patterns (Future Enhancement)** - Create enhanced load profiles with 15-minute granularity - Requires historical smart meter data at 15-min resolution - Load profile format: `month, weekday, hour, quarter, energy` - More accurate but requires data collection/migration **Option C: Smart Interpolation** ``` def get_forecast_with_interpolation(self, intervals): """ Uses weighted interpolation based on typical consumption patterns: - Morning/Evening ramps: More variation between quarters - Night/Midday: More uniform distribution """ # Implementation would use time-of-day heuristics # to distribute hourly energy non-uniformly ``` **Recommendation**: Start with **Option A** (simple division), then migrate to **Option B** when better data becomes available. ______________________________________________________________________ ### 4. Dynamic Tariff Providers (`src/batcontrol/dynamictariff/`) #### Implementation: Using Baseclass Pattern Similar to solar and consumption forecasts, tariff providers use a baseclass: ``` # src/batcontrol/dynamictariff/baseclass.py class DynamicTariffBase(ABC): """Base class for dynamic tariff providers.""" def __init__(self, config: dict): self.config = config self.target_resolution = config.get('general', {}).get('time_resolution_minutes', 60) self.native_resolution = 60 # Override in subclass @abstractmethod def _fetch_prices(self) -> Dict[int, float]: """Fetch prices at native resolution.""" pass def get_prices(self, intervals: int = None) -> Dict[int, float]: """Get prices with automatic resolution handling.""" native_data = self._fetch_prices() if self.native_resolution == self.target_resolution: return native_data # For prices, replication makes more sense than interpolation if self.native_resolution == 60 and self.target_resolution == 15: return self._replicate_hourly_to_15min(native_data) if self.native_resolution == 15 and self.target_resolution == 60: return self._average_15min_to_hourly(native_data) return native_data def _replicate_hourly_to_15min(self, hourly: Dict[int, float]) -> Dict[int, float]: """Replicate each hourly price to 4 quarters.""" prices_15min = {} for hour, price in hourly.items(): for quarter in range(4): prices_15min[hour * 4 + quarter] = price return prices_15min def _average_15min_to_hourly(self, prices_15min: Dict[int, float]) -> Dict[int, float]: """Average 15-min prices to hourly.""" hourly = {} for interval, price in prices_15min.items(): hour = interval // 4 if hour not in hourly: hourly[hour] = [] hourly[hour].append(price) return {h: sum(prices) / len(prices) for h, prices in hourly.items()} ``` #### Provider-Specific Implementations: **Awattar** (`awattar.py`): ``` class Awattar(DynamicTariffBase): def __init__(self, config: dict): super().__init__(config) self.native_resolution = 60 # Awattar provides hourly prices def _fetch_prices(self) -> Dict[int, float]: """Fetch hourly prices from Awattar API.""" # Existing API logic return self._parse_api_response() ``` - Native resolution: 60 minutes - Baseclass automatically replicates to 15-min if needed - **Action Required**: Test to verify correct data delivery **Tibber** (`tibber.py`): ``` class Tibber(DynamicTariffBase): def __init__(self, config: dict): super().__init__(config) # Tibber supports both resolutions via API target = config.get('general', {}).get('time_resolution_minutes', 60) if target == 15: self.native_resolution = 15 self.api_resolution = "HOURLY_15" # Or whatever Tibber's API uses else: self.native_resolution = 60 self.api_resolution = "HOURLY" def _fetch_prices(self) -> Dict[int, float]: """Fetch prices at configured resolution.""" query = self._build_query(resolution=self.api_resolution) return self._parse_api_response(query) ``` - Dynamic resolution based on configuration - Fetches data at target resolution (no conversion needed) - **Action Required**: Verify API parameter for 15-min data **evcc** (`evcc.py`): ``` class EvccTariff(DynamicTariffBase): def __init__(self, config: dict): super().__init__(config) self.native_resolution = 15 # evcc provides 15-min data def _fetch_prices(self) -> Dict[int, float]: """Fetch 15-minute prices from evcc API.""" # Existing API logic # Old code averaged to hourly - now returns native 15-min return self._parse_15min_data() ``` - Native resolution: 15 minutes - Baseclass automatically averages to hourly if needed - **No manual averaging needed anymore** **Energyforecast** (`energyforecast.py`): ``` class Energyforecast(DynamicTariffBase): def __init__(self, config: dict): super().__init__(config) # Energyforecast supports both resolutions via API target = config.get('general', {}).get('time_resolution_minutes', 60) if target == 15: self.native_resolution = 15 self.api_resolution = "QUARTER_HOURLY" # 15-minute resolution else: self.native_resolution = 60 self.api_resolution = "HOURLY" # Default hourly resolution def _fetch_prices(self) -> Dict[int, float]: """Fetch prices at configured resolution from energyforecast.de API.""" params = { 'resolution': self.api_resolution, # 'HOURLY' or 'QUARTER_HOURLY' 'token': self.token, 'vat': 0, 'fixed_cost_cent': 0 } response = requests.get(self.url, params=params, timeout=30) response.raise_for_status() # Parse response and apply local fees/markup/vat return self._parse_api_response(response.json()) ``` - **Dynamic resolution**: Based on configuration - **Native API support**: Fetches data at target resolution (no conversion needed) - **API parameters**: - `resolution`: `"HOURLY"` (default) or `"QUARTER_HOURLY"` (15-minute) - Returns base prices, local calculation of fees/markup/vat - **No baseclass conversion needed** when using native resolution **With baseclass pattern, this complex logic is removed from individual providers.** **API Documentation**: - **Awattar API**: https://www.awattar.de/services/api - **Tibber API**: https://developer.tibber.com/docs/guides/pricing - **evcc API**: https://docs.evcc.io/docs/reference/configuration/messaging#grid-tariff - **Energyforecast API**: https://www.energyforecast.de/api/v1/predictions/next_48_hours **Price Handling Note**: - If tariff provider gives hourly prices, replicate to all 4 quarters: `prices[h*4+q] = hourly_price[h]` - Better granularity if provider supports it ______________________________________________________________________ ### 5. Logic Module (`src/batcontrol/logic/`) #### Charge Rate Calculation (`default.py` lines 145-153) **Current Implementation**: ``` remaining_time = (60 - calc_timestamp.minute) / 60 # Hours remaining in current hour charge_rate = required_recharge_energy / remaining_time # Wh / h = W ``` **15-Minute Sample Implementation**: Remember, that it is configurable for 15 or 60 minute intervals. ``` # Calculate remaining time in current interval current_minute = calc_timestamp.minute current_second = calc_timestamp.second # Find which quarter we're in and time remaining if interval_minutes == 15: current_interval_start = (current_minute // 15) * 15 remaining_minutes = 15 - (current_minute % 15) - current_second / 60 remaining_time = remaining_minutes / 60 # Convert to hours else: # interval_minutes == 60 remaining_time = (60 - current_minute) / 60 charge_rate = required_recharge_energy / remaining_time # Still W ``` #### Array Iteration (`default.py` lines 166-394) **Impact**: All loops currently iterate by hour: ``` for h in range(max_hour): future_price = prices[h] ... ``` **Change Required**: - Rename variable: `h` → `interval` or `i` - Update docstrings mentioning "hours" - Logic remains the same (just more iterations) **Example**: ``` # Lines 178-202: Reserved energy calculation max_interval = len(net_consumption) for i in range(1, max_interval): future_price = prices[i] if future_price <= current_price - min_dynamic_price_difference: max_interval = i logger.debug( "[Rule] Recharge possible in %d intervals (%.1f hours), limiting evaluation window.", i, i * interval_minutes / 60) break ``` **No algorithmic changes needed** - the logic works at any time resolution. ______________________________________________________________________ ### 6. MQTT API (`src/batcontrol/mqtt_api.py`) #### Current Implementation (lines 182-201): ``` def _create_forecast(self, forecast: np.ndarray, timestamp: float) -> dict: """Create forecast JSON with hourly timestamps""" now = timestamp - (timestamp % 3600) # Round to hour data_list = [] for h, value in enumerate(forecast): data_list.append({ 'time_start': now + h * 3600, # 3600s = 1 hour 'value': value, 'time_end': now + (h + 1) * 3600 }) return {'data': data_list} ``` #### Required Changes: ``` def _create_forecast(self, forecast: np.ndarray, timestamp: float, interval_minutes: int = 60) -> dict: """ Create forecast JSON with configurable interval timestamps. Args: forecast: Array of values per interval timestamp: Current timestamp interval_minutes: Time resolution (15, 30, or 60) """ interval_seconds = interval_minutes * 60 # Round to current interval now = timestamp - (timestamp % interval_seconds) data_list = [] for i, value in enumerate(forecast): data_list.append({ 'time_start': now + i * interval_seconds, 'value': value, 'time_end': now + (i + 1) * interval_seconds }) return {'data': data_list} ``` #### Publishing Changes: Update all publish methods to pass `interval_minutes`: ``` def publish_production(self, production: np.ndarray, timestamp: float) -> None: if self.client.is_connected(): self.client.publish( self.base_topic + '/FCST/production', json.dumps(self._create_forecast( production, timestamp, self.interval_minutes)) ) ``` #### Impact on Data Volume: - **Hourly**: 48 data points for 48h forecast - **15-min**: 192 data points for 48h forecast - **Size increase**: ~4× more data per message - **Network**: Minimal impact (JSON compression helps) - **Storage**: InfluxDB/Grafana handle this well ______________________________________________________________________ ### 7. Telegraf Configuration (`config/telegraf.sample.conf`) #### Current Implementation (lines 44-67): ``` [[inputs.mqtt_consumer]] servers = ["tcp://mqtt:1883"] topics = ["house/batcontrol/FCST/+"] data_format = "json_v2" [[inputs.mqtt_consumer.json_v2]] [[inputs.mqtt_consumer.json_v2.object]] path = "data" timestamp_key = "time_start" timestamp_format = "unix" ``` #### Issues Identified: **Problem 1: Time Series Storage** - With 15-min data, you'll have 4× more data points - InfluxDB retention policies may need adjustment - Grafana queries may need to aggregate differently **Problem 2: Visualization** - Hourly charts will now have more granular data - May need to add `GROUP BY time(15m)` in queries - Or keep `GROUP BY time(1h)` with `MEAN()` for backward compatibility #### Recommended Changes: ``` # Add comment explaining interval handling [[inputs.mqtt_consumer]] servers = ["tcp://mqtt:1883"] topics = ["house/batcontrol/FCST/+"] data_format = "json_v2" # Note: Batcontrol may send data at different intervals (15min or 60min) # InfluxDB will store at native resolution. Adjust Grafana queries as needed. [[inputs.mqtt_consumer.json_v2]] [[inputs.mqtt_consumer.json_v2.object]] path = "data" timestamp_key = "time_start" timestamp_format = "unix" # timestamp_precision = "1s" # Optional: Specify precision ``` **Grafana Dashboard Updates**: ``` -- Old query (hourly) SELECT mean("value") FROM "batcontrol-production" WHERE $timeFilter GROUP BY time(1h) -- New query (15-min aware, backward compatible) SELECT mean("value") FROM "batcontrol-production" WHERE $timeFilter GROUP BY time($__interval) -- Auto-adjusts based on time range -- Or explicit 15-min SELECT mean("value") FROM "batcontrol-production" WHERE $timeFilter GROUP BY time(15m) ``` #### Storage Considerations: - **Retention Policy**: Consider shorter retention for 15-min data ``` CREATE RETENTION POLICY "15min_detail" ON "db0" DURATION 7d REPLICATION 1 DEFAULT CREATE RETENTION POLICY "hourly_summary" ON "db0" DURATION 90d REPLICATION 1 ``` - **Continuous Queries**: Auto-downsample 15-min → hourly after 7 days ______________________________________________________________________ ## Configuration Design ### Recommended Configuration Structure Add to `batcontrol_config_dummy.yaml`: ``` general: timezone: Europe/Berlin loglevel: debug # Time resolution configuration # Options: 15, 60 (default: 60) # 15-min: Best accuracy, requires 4x data storage, recommended for dynamic tariffs # 60-min: Legacy mode, backward compatible time_resolution_minutes: 15 battery_control: min_price_difference: 0.05 # ... existing parameters ... # Optional: Per-provider interval overrides (advanced usage) # forecast_providers: # solar: # forced_interval_minutes: 60 # Force hourly even if system uses 15-min # consumption: # forced_interval_minutes: 60 # Keep hourly consumption forecasts # dynamictariff: # forced_interval_minutes: 15 # Force 15-min for prices (if available) ``` ### Configuration Validation Add to `core.py` `__init__`: ``` # Validate interval configuration interval_minutes = config.get('general', {}).get('time_resolution_minutes', 60) if interval_minutes not in [15, 60]: raise ValueError(f"time_resolution_minutes must be 15 or 60. Got: {interval_minutes}") self.interval_minutes = interval_minutes self.intervals_per_hour = 60 // interval_minutes logger.info(f"Using {interval_minutes}-minute time resolution " f"({self.intervals_per_hour} intervals per hour)") # Environment variable support (for Docker deployments) # Can override via: BATCONTROL_TIME_RESOLUTION_MINUTES=15 env_interval = os.environ.get('BATCONTROL_TIME_RESOLUTION_MINUTES') if env_interval: self.interval_minutes = int(env_interval) logger.info(f"Override from environment: {self.interval_minutes} minutes") ``` ______________________________________________________________________ ## Error Handling & Validation ### Interval Mismatch Detection Add validation to ensure all forecast providers return consistent intervals: ``` # src/batcontrol/core.py - Add after forecast collection def validate_forecast_intervals(self, forecasts: dict) -> bool: """ Validate that all forecasts have consistent interval counts. Args: forecasts: Dict of forecast arrays from different providers Returns: True if valid, raises ValueError if inconsistent """ expected_count = self.forecast_hours * self.intervals_per_hour for name, forecast in forecasts.items(): actual_count = len(forecast) if actual_count < expected_count * 0.9: # Allow 10% tolerance logger.error( f"Forecast '{name}' has {actual_count} intervals, " f"expected ~{expected_count}. Check provider implementation." ) raise ValueError(f"Invalid forecast interval count from {name}") if actual_count != expected_count: logger.warning( f"Forecast '{name}' has {actual_count} intervals, " f"expected {expected_count}. Will truncate/pad." ) # Pad with last value or truncate if actual_count < expected_count: pad_value = forecast[-1] if len(forecast) > 0 else 0 forecasts[name] = np.pad(forecast, (0, expected_count - actual_count), constant_values=pad_value) else: forecasts[name] = forecast[:expected_count] return True ``` ### Provider Error Handling ``` def get_forecast_safe(self, provider_name: str, provider_func: callable) -> dict: """ Safely get forecast with error handling and fallback. """ try: forecast = provider_func() # Validate interval count expected = self.forecast_hours * self.intervals_per_hour if len(forecast) < expected * 0.5: logger.warning(f"{provider_name} returned too few intervals, using fallback") return self.get_fallback_forecast(provider_name, expected) return forecast except Exception as e: logger.error(f"Error getting forecast from {provider_name}: {e}") return self.get_fallback_forecast(provider_name, self.forecast_hours * self.intervals_per_hour) ``` ______________________________________________________________________ ## Monitoring & Observability ### Performance Metrics Add metrics to track 15-min performance: ``` # src/batcontrol/core.py - Add metrics collection import time class PerformanceMetrics: def __init__(self): self.metrics = { 'forecast_fetch_time': [], 'logic_calculation_time': [], 'total_cycle_time': [], 'array_size': 0, 'interval_minutes': 0 } def record_cycle(self, forecast_time: float, logic_time: float, total_time: float, array_size: int): self.metrics['forecast_fetch_time'].append(forecast_time) self.metrics['logic_calculation_time'].append(logic_time) self.metrics['total_cycle_time'].append(total_time) self.metrics['array_size'] = array_size def report(self): if not self.metrics['total_cycle_time']: return logger.info( f"Performance (last 10 cycles, {self.metrics['interval_minutes']}min intervals): " f"Fetch: {np.mean(self.metrics['forecast_fetch_time'][-10:]):.2f}s, " f"Logic: {np.mean(self.metrics['logic_calculation_time'][-10:]):.2f}s, " f"Total: {np.mean(self.metrics['total_cycle_time'][-10:]):.2f}s, " f"Array size: {self.metrics['array_size']}" ) # Usage in main loop: metrics = PerformanceMetrics() metrics.metrics['interval_minutes'] = self.interval_minutes while True: cycle_start = time.time() fetch_start = time.time() # ... fetch forecasts ... fetch_time = time.time() - fetch_start logic_start = time.time() # ... run logic ... logic_time = time.time() - logic_start total_time = time.time() - cycle_start metrics.record_cycle(fetch_time, logic_time, total_time, len(production)) # Report every 10 cycles if len(metrics.metrics['total_cycle_time']) % 10 == 0: metrics.report() ``` ### MQTT Monitoring Topics Publish interval configuration and health metrics: ``` # Publish system configuration self.mqtt_api.publish( 'house/batcontrol/config/interval_minutes', json.dumps({'value': self.interval_minutes}) ) # Publish performance metrics self.mqtt_api.publish( 'house/batcontrol/metrics/performance', json.dumps({ 'timestamp': time.time(), 'interval_minutes': self.interval_minutes, 'forecast_fetch_time_s': fetch_time, 'logic_calculation_time_s': logic_time, 'array_size': len(production) }) ) ``` ______________________________________________________________________ ## Rollback Procedure ### Quick Rollback Steps If 15-min mode causes issues in production: **Method 1: Configuration Change (No Restart)** ``` # Edit config file nano /path/to/batcontrol_config.yaml # Change: # time_resolution_minutes: 15 # To: time_resolution_minutes: 60 # Config is reloaded automatically on next cycle (3 minutes) ``` **Method 2: Environment Variable (Docker)** ``` # Edit docker-compose.yml or restart with env var docker stop batcontrol docker run -e BATCONTROL_TIME_RESOLUTION_MINUTES=60 ... ``` **Method 3: Git Rollback** ``` # Checkout previous stable version git checkout v1.9.0 # Last hourly-only version docker-compose build docker-compose up -d ``` ### Validation After Rollback ``` # Check logs for interval setting docker logs batcontrol | grep "time resolution" # Verify MQTT output shows hourly timestamps mosquitto_sub -h mqtt -t "house/batcontrol/FCST/+" -C 1 | jq . # Check array sizes (should be ~48, not ~192) docker logs batcontrol | grep "Array size" ``` ______________________________________________________________________ ## Migration Strategy ### Phase 1: Foundation (2-3 weeks) 1. **Add configuration parameter** for `time_resolution_minutes` 1. **Update core.py** to use configurable intervals 1. **Create utility functions** for time conversions 1. **Unit tests** for interval handling ### Phase 2: Forecast Providers (2-3 weeks) 1. **Implement solar upsampling** (linear interpolation) 1. **Update consumption forecast** (simple division) 1. **Modify evcc providers** to support native 15-min 1. **Add provider error handling and fallbacks** 1. **Integration tests** with all providers ### Phase 3: Logic & MQTT (2-3 weeks) 1. **Update charge rate calculation** 1. **Refactor loop variables** (h → interval) 1. **Modify MQTT timestamp generation** 1. **Add performance monitoring** 1. **Update Telegraf config documentation** 1. **End-to-end tests** ### Phase 4: Testing & Documentation (2-3 weeks) 1. **Extended testing** at 15-min intervals 1. **Performance testing** (4× data volume) 1. **Beta testing** with community volunteers 1. **Update README and HOWITWORKS.md** 1. **Migration guide** for existing users 1. **Release as beta** (v2.0-beta) ### Phase 5: Production Release (1-2 weeks) 1. **Address beta feedback** 1. **Final bug fixes** 1. **Production monitoring setup** 1. **Release v2.0 stable** ### Phase 6: Optimization (Ongoing) 1. **Enhanced consumption profiles** (15-min granularity) 1. **Better interpolation algorithms** for solar 1. **Adaptive interval selection** based on data availability 1. **Performance optimizations** **Total Realistic Timeline: 10-14 weeks** (vs original estimate of 4-5 weeks) ______________________________________________________________________ ## Testing Requirements ### Unit Tests Needed ``` # tests/batcontrol/test_interval_handling.py import pytest from datetime import datetime, timezone from batcontrol.interval_utils import ( get_elapsed_fraction, get_remaining_time_hours, upsample_forecast ) def test_time_correction_15min(): """Test that partial interval is correctly calculated at 15-min resolution""" # Test at 7 minutes into interval test_time = datetime(2025, 10, 14, 10, 7, 30, tzinfo=timezone.utc) elapsed = get_elapsed_fraction(test_time, interval_minutes=15) # Expected: (7 + 30/60) / 15 = 7.5 / 15 = 0.5 assert abs(elapsed - 0.5) < 0.01, f"Expected ~0.5, got {elapsed}" # Test at interval boundary test_time = datetime(2025, 10, 14, 10, 0, 0, tzinfo=timezone.utc) elapsed = get_elapsed_fraction(test_time, interval_minutes=15) assert elapsed == 0.0 def test_time_correction_60min(): """Test backward compatibility with hourly intervals""" # Test at 30 minutes into hour test_time = datetime(2025, 10, 14, 10, 30, 0, tzinfo=timezone.utc) elapsed = get_elapsed_fraction(test_time, interval_minutes=60) assert abs(elapsed - 0.5) < 0.01 def test_remaining_time_15min(): """Test remaining time calculation for 15-min intervals""" # At 7.5 minutes into 15-min interval test_time = datetime(2025, 10, 14, 10, 7, 30, tzinfo=timezone.utc) remaining = get_remaining_time_hours(test_time, interval_minutes=15) # Expected: 7.5 minutes remaining = 0.125 hours expected = 7.5 / 60 assert abs(remaining - expected) < 0.001, f"Expected {expected}, got {remaining}" def test_charge_rate_calculation_15min(): """Test charge rate with 15 minutes remaining""" # Need to charge 500 Wh in 7.5 minutes (0.125 hours) required_energy = 500 # Wh remaining_time = 7.5 / 60 # hours charge_rate = required_energy / remaining_time expected_rate = 500 / 0.125 # = 4000 W assert abs(charge_rate - expected_rate) < 1, f"Expected {expected_rate}W, got {charge_rate}W" def test_solar_upsampling_constant(): """Test constant upsampling (simple division)""" hourly = {0: 1000, 1: 1000, 2: 1000} result = upsample_forecast(hourly, interval_minutes=15, method='constant') # Each 15-min interval should be 250 Wh for i in range(12): # 3 hours * 4 intervals assert result[i] == 250, f"Interval {i}: expected 250, got {result[i]}" def test_solar_upsampling_linear(): """Test linear interpolation of hourly to 15-min solar forecast""" hourly = {0: 1000, 1: 2000, 2: 1000} result = upsample_forecast(hourly, interval_minutes=15, method='linear') # Hour 0→1: ramping up from 1000W to 2000W # Interval 0: 1000W * 0.25h = 250 Wh # Interval 1: 1250W * 0.25h = 312.5 Wh # Interval 2: 1500W * 0.25h = 375 Wh # Interval 3: 1750W * 0.25h = 437.5 Wh assert abs(result[0] - 250.0) < 0.1 assert abs(result[1] - 312.5) < 0.1 assert abs(result[2] - 375.0) < 0.1 assert abs(result[3] - 437.5) < 0.1 # Hour 1→2: ramping down from 2000W to 1000W assert abs(result[4] - 500.0) < 0.1 assert abs(result[7] - 312.5) < 0.1 def test_solar_upsampling_edge_cases(): """Test edge cases in solar upsampling""" # Empty dict result = upsample_forecast({}, interval_minutes=15, method='linear') assert len(result) == 0 # Single hour result = upsample_forecast({0: 1000}, interval_minutes=15, method='linear') assert len(result) == 0 # Can't interpolate with only one point # With zeros hourly = {0: 0, 1: 1000, 2: 0} result = upsample_forecast(hourly, interval_minutes=15, method='linear') assert result[0] == 0 assert result[4] > 200 # Should be ramping up assert result[8] == 0 def test_solar_upsampling_cubic(): """Test cubic spline interpolation for smooth solar forecast""" try: import scipy except ImportError: pytest.skip("scipy not available, skipping cubic interpolation test") # Test typical solar production curve: sunrise -> peak -> sunset hourly = { 0: 0, # Night 1: 100, # Early morning 2: 500, # Morning 3: 1500, # Mid-morning 4: 2500, # Near noon 5: 3000, # Noon peak 6: 2500, # Afternoon 7: 1500, # Late afternoon 8: 500, # Evening 9: 100, # Dusk 10: 0 # Night } result = upsample_forecast(hourly, interval_minutes=15, method='cubic') # Should have 4 intervals per hour assert len(result) >= 40 # 10 hours * 4 intervals # Verify smooth transitions (cubic should be smoother than linear) # Check that values are non-negative (clamped) for idx, value in result.items(): assert value >= 0, f"Interval {idx} has negative value: {value}" # Verify general shape: values should increase to peak then decrease # Peak should be around hour 5 (intervals 20-23) peak_intervals = [result.get(i, 0) for i in range(20, 24)] early_intervals = [result.get(i, 0) for i in range(0, 4)] late_intervals = [result.get(i, 0) for i in range(36, 40)] assert max(peak_intervals) > max(early_intervals), "Peak should be higher than morning" assert max(peak_intervals) > max(late_intervals), "Peak should be higher than evening" # Cubic should produce smoother transitions (check rate of change) # Compare with linear interpolation result_linear = upsample_forecast(hourly, interval_minutes=15, method='linear') # Calculate variance in second derivative (measure of smoothness) def second_derivative_variance(data): """Calculate variance of second derivative as smoothness metric""" values = [data.get(i, 0) for i in range(len(data))] if len(values) < 3: return 0 second_diffs = [] for i in range(len(values) - 2): second_diff = values[i+2] - 2*values[i+1] + values[i] second_diffs.append(second_diff) return sum(d**2 for d in second_diffs) / len(second_diffs) if second_diffs else 0 smoothness_cubic = second_derivative_variance(result) smoothness_linear = second_derivative_variance(result_linear) # Cubic should be smoother (lower second derivative variance) # Note: This might not always hold due to clamping, so we just check it doesn't fail assert smoothness_cubic >= 0 # Just verify calculation works def test_cubic_interpolation_requires_scipy(): """Test that cubic method raises ImportError without scipy""" # Temporarily hide scipy if it exists import sys scipy_backup = sys.modules.get('scipy') try: # Remove scipy from modules to simulate it not being installed if 'scipy' in sys.modules: del sys.modules['scipy'] if 'scipy.interpolate' in sys.modules: del sys.modules['scipy.interpolate'] hourly = {0: 1000, 1: 2000, 2: 1500} # Should raise ImportError with pytest.raises(ImportError, match="scipy"): result = upsample_forecast(hourly, interval_minutes=15, method='cubic') finally: # Restore scipy if it was available if scipy_backup is not None: sys.modules['scipy'] = scipy_backup def test_cubic_vs_linear_comparison(): """Compare cubic and linear interpolation behavior""" try: import scipy except ImportError: pytest.skip("scipy not available") # Test case: Sharp peak (where cubic should smooth better) hourly = {0: 100, 1: 500, 2: 2000, 3: 500, 4: 100} result_linear = upsample_forecast(hourly, interval_minutes=15, method='linear') result_cubic = upsample_forecast(hourly, interval_minutes=15, method='cubic') # Both should have same number of intervals assert len(result_linear) == len(result_cubic) # Peak should be around hour 2 (intervals 8-11) linear_peak = max(result_linear.get(i, 0) for i in range(8, 12)) cubic_peak = max(result_cubic.get(i, 0) for i in range(8, 12)) # Both should capture the peak assert linear_peak > 400 assert cubic_peak > 400 # Cubic might overshoot slightly (natural spline behavior) # but should be clamped to non-negative for idx, value in result_cubic.items(): assert value >= 0 def test_consumption_division(): """Test hourly consumption divided into 15-min intervals""" from batcontrol.forecastconsumption.forecast_csv import ConsumptionForecast # Mock config with 15-min intervals config = {'interval_minutes': 15, 'scaling_factor': 1.0} forecast = ConsumptionForecast(config) # Get forecast for 8 intervals (2 hours at 15-min) result = forecast.get_forecast(intervals=8) # Should return 8 intervals assert len(result) == 8 # Each interval should be roughly 1/4 of hourly consumption # (Actual values depend on load profile CSV) def test_mqtt_timestamps_15min(): """Test MQTT forecast timestamps at 15-min intervals""" from batcontrol.mqtt_api import MQTTApi import numpy as np mqtt = MQTTApi(config={'interval_minutes': 15}) # Test forecast with 8 intervals forecast_data = np.array([100, 150, 200, 250, 300, 350, 400, 450]) timestamp = 1697270400 # 2023-10-14 10:00:00 UTC result = mqtt._create_forecast(forecast_data, timestamp, interval_minutes=15) # Should have 8 data points assert len(result['data']) == 8 # Check timestamps are 15 minutes apart (900 seconds) for i in range(len(result['data']) - 1): time_diff = result['data'][i+1]['time_start'] - result['data'][i]['time_start'] assert time_diff == 900, f"Expected 900s, got {time_diff}s" # Check values match for i, item in enumerate(result['data']): assert item['value'] == forecast_data[i] def test_edge_cases_dst_transition(): """Test handling of daylight saving time transitions""" # Test spring forward (skip hour) # Test fall back (repeat hour) # Ensure interval counts remain consistent pass def test_edge_cases_midnight_rollover(): """Test interval calculation across midnight""" test_time = datetime(2025, 10, 14, 23, 52, 30, tzinfo=timezone.utc) elapsed = get_elapsed_fraction(test_time, interval_minutes=15) # At XX:52:30, in 4th quarter (45-60 min range) # Minutes in interval: 52 % 15 = 7 minutes # Elapsed: (7 + 30/60) / 15 = 0.5 assert abs(elapsed - 0.5) < 0.01 def test_performance_array_operations(): """Test that 15-min arrays don't cause performance regression""" import time import numpy as np # Test with hourly data (48 elements) hourly_data = np.random.rand(48) start = time.time() for _ in range(1000): result = np.sum(hourly_data * 2.0) hourly_time = time.time() - start # Test with 15-min data (192 elements) min15_data = np.random.rand(192) start = time.time() for _ in range(1000): result = np.sum(min15_data * 2.0) min15_time = time.time() - start # Should be less than 5x slower (4x data = ~4x time) assert min15_time < hourly_time * 5, \ f"15-min operations too slow: {min15_time}s vs {hourly_time}s" ``` ### Integration Tests ``` # tests/batcontrol/test_core_15min_integration.py import pytest from unittest.mock import Mock, patch import numpy as np from batcontrol.core import BatControl def test_full_cycle_15min(monkeypatch): """Test complete run cycle with 15-min intervals""" # Mock all forecast providers to return 15-min data mock_solar = {i: 100 + i * 10 for i in range(192)} # 48 hours at 15-min mock_consumption = {i: 200 + i * 5 for i in range(192)} mock_prices = {i: 0.20 + (i % 96) * 0.001 for i in range(192)} with patch('batcontrol.forecastsolar.Solar.get_forecast', return_value=mock_solar), \ patch('batcontrol.forecastconsumption.Consumption.get_forecast', return_value=mock_consumption), \ patch('batcontrol.dynamictariff.DynamicTariff.get_prices', return_value=mock_prices): config = { 'general': {'time_resolution_minutes': 15}, # ... other config ... } bc = BatControl(config) bc.run_once() # Verify array sizes assert len(bc.production) == 192 assert len(bc.consumption) == 192 assert len(bc.prices) == 192 # Verify logic produced valid charge rate assert bc.charge_rate >= bc.config['min_charge_rate'] assert bc.charge_rate <= bc.config['max_charge_rate'] def test_backward_compatibility_60min(): """Ensure 60-min mode still works exactly as before""" config = { 'general': {'time_resolution_minutes': 60}, # ... other config ... } bc = BatControl(config) bc.run_once() # Should have 48 hourly intervals assert len(bc.production) == 48 assert len(bc.consumption) == 48 assert len(bc.prices) == 48 def test_provider_mismatch_handling(): """Test system handles mismatched provider intervals gracefully""" # Solar returns 192 intervals (15-min) mock_solar = {i: 100 for i in range(192)} # But consumption only returns 48 (hourly) mock_consumption = {i: 200 for i in range(48)} with patch('batcontrol.forecastsolar.Solar.get_forecast', return_value=mock_solar), \ patch('batcontrol.forecastconsumption.Consumption.get_forecast', return_value=mock_consumption): config = {'general': {'time_resolution_minutes': 15}} bc = BatControl(config) # Should either pad consumption or raise clear error with pytest.raises(ValueError, match="Invalid forecast interval count"): bc.run_once() def test_mqtt_output_format(): """Test MQTT messages have correct format at 15-min intervals""" config = {'general': {'time_resolution_minutes': 15}} bc = BatControl(config) # Capture MQTT publish calls published_messages = [] bc.mqtt_api.publish = lambda topic, message: published_messages.append((topic, message)) bc.run_once() # Find production forecast message prod_msg = next(msg for topic, msg in published_messages if 'production' in topic) import json data = json.loads(prod_msg) # Should have many intervals assert len(data['data']) > 48 # Timestamps should be 15 min apart time_diff = data['data'][1]['time_start'] - data['data'][0]['time_start'] assert time_diff == 900 # 15 minutes in seconds ``` ### Performance Benchmarks ``` # tests/batcontrol/test_performance.py import time import pytest from batcontrol.core import BatControl @pytest.mark.slow def test_performance_15min_vs_60min(): """Compare performance: 15-min should be < 2x slower than 60-min""" # Benchmark 60-min mode config_60 = {'general': {'time_resolution_minutes': 60}} bc_60 = BatControl(config_60) start = time.time() for _ in range(10): bc_60.run_once() time_60 = (time.time() - start) / 10 # Benchmark 15-min mode config_15 = {'general': {'time_resolution_minutes': 15}} bc_15 = BatControl(config_15) start = time.time() for _ in range(10): bc_15.run_once() time_15 = (time.time() - start) / 10 print(f"60-min: {time_60:.3f}s, 15-min: {time_15:.3f}s, ratio: {time_15/time_60:.2f}x") # 15-min should not be more than 2x slower assert time_15 < time_60 * 2.0, \ f"15-min mode too slow: {time_15:.3f}s vs {time_60:.3f}s" # Absolute performance target: should complete in < 5 seconds assert time_15 < 5.0, f"15-min cycle too slow: {time_15:.3f}s" @pytest.mark.slow def test_memory_usage(): """Test memory usage doesn't grow excessively with 15-min intervals""" import psutil import os process = psutil.Process(os.getpid()) config = {'general': {'time_resolution_minutes': 15}} bc = BatControl(config) mem_before = process.memory_info().rss / 1024 / 1024 # MB # Run 100 cycles for _ in range(100): bc.run_once() mem_after = process.memory_info().rss / 1024 / 1024 # MB mem_increase = mem_after - mem_before # Memory increase should be < 50 MB assert mem_increase < 50, f"Memory leak detected: +{mem_increase:.1f} MB" def test_backward_compatibility_60min(): """Ensure 60-min mode still works exactly as before""" pass ``` ______________________________________________________________________ ## Risks and Mitigation ### Risk 1: Data Quality **Problem**: Simple division of consumption creates unrealistic flat profiles\ **Mitigation**: - Phase 1: Accept limitation, still better than hourly - Phase 2: Collect real 15-min data, create enhanced profiles - Phase 3: Machine learning for pattern prediction ### Risk 2: Increased Complexity **Problem**: More intervals = more computation, more data\ **Mitigation**: - Performance profiling before/after - Consider limiting forecast horizon (e.g., 24h instead of 48h at 15-min) - Optimize array operations with NumPy ### Risk 3: Breaking Changes **Problem**: Existing users on hourly system\ **Mitigation**: - Default to 60 minutes (backward compatible) - Clear migration documentation - Feature flag for beta testing - Version bump: 1.x → 2.0 ### Risk 4: MQTT Data Volume **Problem**: 4× more data points per message\ **Mitigation**: - MQTT handles this well (tested up to 1000s of points) - Consider optional data thinning for slow networks - Document InfluxDB retention strategies ### Risk 5: Visualization Overload **Problem**: Grafana charts may look cluttered\ **Mitigation**: - Provide dashboard examples for 15-min data - Document query aggregation patterns - Auto-detect interval in dashboard variables ______________________________________________________________________ ## Evaluation: Configurable Interval (15-60 minutes) ### Arguments FOR Configurability **1. Backward Compatibility** - Existing users can stay on 60-min without changes - Gradual migration path **2. Flexibility** - Some users may have hourly-only data sources - Different markets have different price granularity **3. Risk Mitigation** - Easy rollback if issues found - A/B testing possible - Beta testing without breaking production **4. Future-Proofing** - Easy to add 5-minute intervals later - Framework for dynamic interval selection **5. Resource Optimization** - Lower-end hardware can use 60-min - High-performance systems use 15-min ### Arguments AGAINST Configurability **1. Complexity** - More code paths to test - More documentation needed - More user confusion **2. Maintenance Burden** - Need to maintain multiple modes - Bug fixes across all intervals - Version compatibility matrix **3. Delayed Adoption** - Users may stick with "good enough" 60-min - 15-min benefits not realized **4. Half-Baked Features** - Simple division of hourly consumption still inaccurate at 15-min - Better to wait for proper 15-min data ### Recommendation: **MAKE IT CONFIGURABLE** **Rationale**: 1. **Safety First**: Allows thorough testing without breaking existing systems 1. **User Choice**: Different users have different needs and capabilities 1. **Iterative Improvement**: Can enhance 15-min implementation over time 1. **Market Evolution**: Not all markets offer 15-min prices yet 1. **Low Cost**: Code structure supports this naturally with minimal overhead **Implementation**: - Default: 60 minutes (no breaking changes) - Beta flag: `time_resolution_minutes: 15` (opt-in for testing) - v2.0 release: Default to 15 minutes (with loud warning in changelog) ______________________________________________________________________ ## Performance Considerations ### Computational Impact **Array Operations**: - **Before**: 48-element arrays - **After**: 192-element arrays - **Impact**: Negligible (NumPy optimized) **Loop Iterations**: - **Before**: ~48 iterations per calculation - **After**: ~192 iterations per calculation - **Impact**: < 1ms additional overhead **Memory**: - **Before**: ~10 KB per forecast - **After**: ~40 KB per forecast - **Impact**: Trivial on modern systems ### Network Impact **MQTT Message Sizes**: - **Hourly**: ~2-3 KB JSON - **15-min**: ~8-12 KB JSON - **Impact**: Still well under MQTT limits (256 MB default) **Database Storage**: - **Hourly**: ~1,000 points/day/metric - **15-min**: ~4,000 points/day/metric - **Impact**: InfluxDB handles millions easily ______________________________________________________________________ ## Conclusion & Recommendations ### Summary of Required Changes | Component | Complexity | Risk | Priority | | --------------------------- | ---------- | ------ | -------- | | Core (interval handling) | Medium | Low | HIGH | | Solar forecast (upsampling) | High | Medium | HIGH | | Consumption (division) | Low | Low | HIGH | | Dynamic tariff (evcc) | Low | Low | MEDIUM | | Logic (charge rate) | Medium | Medium | HIGH | | MQTT API | Low | Low | MEDIUM | | Telegraf config | Low | Low | LOW | ### Final Recommendations **1. Implement Configurable Intervals** ✅ - Default to 60 minutes for backward compatibility - Allow opt-in to 15 minutes via configuration - Plan for 30-minute option as middle ground **2. Solar Forecast Upsampling** ✅ - Use linear interpolation (simple, effective) - Correctly handle Wh → W conversion - Document limitations (especially for cloudy days) **3. Consumption Forecast** ⚠️ - Start with simple division (Wh/hour ÷ 4) - Mark as "TODO: Enhance with real 15-min profiles" - Create data collection guide for users **4. Price Forecasts** ✅ - evcc: Use native 15-min data when available - Awattar/Tibber: Replicate hourly prices to quarters - Future: Monitor for API updates offering 15-min granularity **5. Testing Strategy** 🧪 - Comprehensive unit tests for each component - Integration tests for full cycle - Beta testing with community (opt-in flag) - Performance benchmarks (ensure < 10% overhead) **6. Migration Path** 📋 - Release v1.9: Add configuration, default 60-min - Release v2.0-beta: Switch default to 15-min, call for testing - Release v2.0: Make 15-min official default - Timeline: 2-3 months for stable release **7. Documentation Updates** 📚 - Update README with interval configuration - Add migration guide for existing users - Document limitations (consumption profiles) - Provide Grafana dashboard examples **8. Future Enhancements** 🚀 - Collect user smart meter data (15-min resolution) - Build community database of 15-min load profiles - Implement adaptive interval selection - Add 5-minute interval support for ultra-dynamic tariffs ______________________________________________________________________ ## Appendix: Code Organization Recommendations ### New Files to Create ``` src/batcontrol/ interval_utils.py # Time interval utilities tests/batcontrol/ test_interval_handling.py # Unit tests for intervals test_core_15min_integration.py # Integration tests docs/ migration-15min.md # Migration guide interval-configuration.md # Configuration reference ``` ### Key Utility Functions ``` # src/batcontrol/interval_utils.py def get_interval_count(hours: int, interval_minutes: int) -> int: """Calculate number of intervals for given hours and resolution.""" return hours * (60 // interval_minutes) def get_elapsed_fraction(timestamp: datetime, interval_minutes: int) -> float: """Calculate fraction of current interval that has elapsed.""" minute_in_interval = timestamp.minute % interval_minutes second_fraction = timestamp.second / 60 return (minute_in_interval + second_fraction) / interval_minutes def get_remaining_time_hours(timestamp: datetime, interval_minutes: int) -> float: """Calculate remaining time in current interval (in hours).""" elapsed = get_elapsed_fraction(timestamp, interval_minutes) remaining_minutes = interval_minutes * (1 - elapsed) return remaining_minutes / 60 def upsample_forecast(hourly_data: dict, interval_minutes: int, method: str = 'linear') -> dict: """ Upsample hourly forecast to smaller intervals. Args: hourly_data: Dict of {hour: value_wh} interval_minutes: Target resolution (15, 30) method: 'linear', 'constant', or 'cubic' Returns: Dict of {interval: value_wh} at target resolution """ if interval_minutes == 60: return hourly_data intervals_per_hour = 60 // interval_minutes upsampled = {} if method == 'constant': # Simple division for hour, value in hourly_data.items(): for i in range(intervals_per_hour): upsampled[hour * intervals_per_hour + i] = value / intervals_per_hour elif method == 'linear': # Linear interpolation max_hour = max(hourly_data.keys()) for hour in range(max_hour): current_val = hourly_data.get(hour, 0) next_val = hourly_data.get(hour + 1, 0) for i in range(intervals_per_hour): idx = hour * intervals_per_hour + i fraction = i / intervals_per_hour interpolated = current_val + (next_val - current_val) * fraction upsampled[idx] = interpolated / intervals_per_hour return upsampled ``` ______________________________________________________________________ ## Questions for Stakeholders **Please answer these questions to guide implementation priorities:** ### 1. Data Availability - **Q**: Do you have access to 15-minute consumption data (smart meter exports)? - **Why**: Needed for Option B (enhanced load profiles) in Phase 2 - **Format**: CSV with timestamp and Wh/kWh per 15-min interval - **Timeframe**: At least 1 year of data preferred ### 2. Tariff Provider - **Q**: Which electricity tariff provider(s) do you currently use? - [ ] Awattar (Germany/Austria) - [ ] Tibber (Nordic countries, Germany, Netherlands) - [ ] evcc integration - [ ] Other: ***\_******\_***\_ - **Q**: What is your tariff's price update frequency? - [ ] Hourly (most common) - [ ] 15-minute intervals - [ ] Other: ***\_******\_***\_ ### 3. Hardware & Environment - **Q**: What hardware are you running batcontrol on? - [ ] Raspberry Pi 3 or older - [ ] Raspberry Pi 4/5 - [ ] Server (x86) - [ ] Docker container on: ***\_******\_***\_ - **Q**: What are your hardware specs? - CPU: ***\_******\_***\_ - RAM: ***\_******\_***\_ - Storage: ***\_******\_***\_ ### 4. Testing Capability - **Q**: Do you have a test environment separate from production? - [ ] Yes, separate test system - [ ] Yes, can run in parallel with production - [ ] No, must test in production carefully - **Q**: Are you willing to participate in beta testing? - [ ] Yes, can test immediately - [ ] Yes, but need 2-4 weeks notice - [ ] No, prefer to wait for stable release ### 5. Current Performance - **Q**: Do you have existing performance metrics? - [ ] Yes, have Grafana dashboards - [ ] Yes, have log analysis - [ ] No, but can collect - [ ] No baseline metrics - **Q**: What is your current evaluation cycle time? - Average: \_\_\_\_\_ seconds per cycle - Acceptable: \_\_\_\_\_ seconds per cycle ### 6. Project Motivation - **Q**: What's driving this 15-minute interval change? - [ ] New tariff structure requires it - [ ] Want more responsive battery control - [ ] Proactive optimization - [ ] Regulatory requirement - [ ] Other: ***\_******\_***\_ - **Q**: Is this critical or nice-to-have? - [ ] Critical - needed within: \_\_\_\_\_ weeks - [ ] Important - target date: ***\_******\_***\_ - [ ] Nice-to-have - no rush ### 7. Backward Compatibility - **Q**: Must the new version work with existing configurations? - [ ] Yes, must be fully backward compatible - [ ] Yes, but migration is acceptable - [ ] No, breaking changes OK ### 8. Documentation & Support - **Q**: What documentation would be most helpful? - [ ] Migration guide (existing -> 15-min) - [ ] Performance tuning guide - [ ] Troubleshooting guide - [ ] API reference updates - [ ] Grafana dashboard examples - [ ] Video tutorials ### 9. Community - **Q**: Are there other users you know who need 15-min intervals? - Number: \_\_\_\_\_ - Countries: ***\_******\_***\_ - Use cases: ***\_******\_***\_ ### 10. Future Features - **Q**: After 15-min support, what would be most valuable? - [ ] 5-minute intervals (ultra-responsive) - [ ] Adaptive interval selection (auto-switch based on conditions) - [ ] Machine learning for consumption forecasting - [ ] Better solar interpolation (weather-aware) - [ ] Other: ***\_******\_***\_ **Please provide answers in the GitHub issue or discussion thread.** ______________________________________________________________________ ## Appendix A: Visual Examples ### Example: Grafana Dashboard Comparison **Hourly Visualization (Current)**: ``` Solar Production Forecast (Hourly) 10:00 11:00 12:00 13:00 14:00 15:00 500 1200 2000 2500 2200 1800 Wh ─┘ ─┐ ─┐ ─┐ ─┐ ─┘ └─────┴─────┴─────┴─────┴───── Smooth line, but misses within-hour variations ``` **15-Minute Visualization (Proposed)**: ``` Solar Production Forecast (15-min) 10:00 :15 :30 :45 11:00 :15 :30 :45 12:00 125 150 175 200 250 300 350 400 Wh ─┐ ─┐ ─┐ ─┐ ─┐ ─┐ ─┐ ─┐ └───┴───┴───┴───┴───┴───┴───┴─── More granular, captures ramp-up patterns ``` ### Example: Charge Rate Decision **Scenario**: Current time 10:07, need to charge 500 Wh **Hourly Mode (Current)**: - Remaining time in hour: 53 minutes = 0.883 hours - Charge rate: 500 / 0.883 = 566 W - **Issue**: Rate calculated based on full hour, might over/under charge **15-Minute Mode (Proposed)**: - Current interval: 10:00-10:15 - Remaining time in interval: 8 minutes = 0.133 hours - Charge rate: 500 / 0.133 = 3,759 W - **Better**: More responsive, adjusts every 15 min instead of every hour ### Example: Price Optimization **Tariff**: - 10:00-11:00: 0.25 €/kWh - 11:00-12:00: 0.15 €/kWh (cheaper) - 12:00-13:00: 0.30 €/kWh **Hourly Logic**: - Sees 3 price levels, decides at hour boundaries - May miss opportunity at 11:00 exactly **15-Min Logic**: - Sees 12 price levels (4 per hour, possibly different if provider supports) - Can start charging at 11:00:00 precisely - **Savings**: Up to 15 minutes of cheaper charging = 0.25 kWh × 0.10 €/kWh = 0.025 € - Over a year: ~9 € in savings (assuming 1 cycle/day) ______________________________________________________________________ ## Appendix B: Code Organization Recommendations ### New Files to Create ``` src/batcontrol/ interval_utils.py # Time interval utilities (NEW) validators.py # Forecast validation (NEW) tests/batcontrol/ test_interval_handling.py # Unit tests for intervals (NEW) test_core_15min_integration.py # Integration tests (NEW) test_performance.py # Performance benchmarks (NEW) docs/ migration-15min.md # Migration guide (NEW) interval-configuration.md # Configuration reference (NEW) troubleshooting-15min.md # Troubleshooting guide (NEW) performance-tuning.md # Performance optimization (NEW) ``` ### Key Utility Functions ``` # src/batcontrol/interval_utils.py import datetime import math from typing import Dict, Literal def get_interval_count(hours: int, interval_minutes: int) -> int: """Calculate number of intervals for given hours and resolution.""" return hours * (60 // interval_minutes) def get_elapsed_fraction(timestamp: datetime.datetime, interval_minutes: int) -> float: """Calculate fraction of current interval that has elapsed.""" minute_in_interval = timestamp.minute % interval_minutes second_fraction = timestamp.second / 60 return (minute_in_interval + second_fraction) / interval_minutes def get_remaining_time_hours(timestamp: datetime.datetime, interval_minutes: int) -> float: """Calculate remaining time in current interval (in hours).""" elapsed = get_elapsed_fraction(timestamp, interval_minutes) remaining_minutes = interval_minutes * (1 - elapsed) return remaining_minutes / 60 def round_to_interval(timestamp: datetime.datetime, interval_minutes: int) -> datetime.datetime: """Round timestamp down to the start of its interval.""" interval_seconds = interval_minutes * 60 unix_time = timestamp.timestamp() rounded_unix = unix_time - (unix_time % interval_seconds) return datetime.datetime.fromtimestamp(rounded_unix, tz=timestamp.tzinfo) def upsample_forecast( hourly_data: Dict[int, float], interval_minutes: int, method: Literal['linear', 'constant', 'cubic'] = 'linear' ) -> Dict[int, float]: """ Upsample hourly forecast to smaller intervals. Args: hourly_data: Dict of {hour: value_wh} interval_minutes: Target resolution (15 or 60) method: Interpolation method - 'constant': Simple division (value/4 for each quarter) Best for: Uniform loads, quick calculations - 'linear': Linear power interpolation between hours Best for: Solar forecasts, general purpose - 'cubic': Cubic spline interpolation (requires scipy) Best for: Smooth transitions, realistic power curves Note: May overshoot/undershoot, clamped to non-negative Returns: Dict of {interval: value_wh} at target resolution Raises: ImportError: If cubic method is used without scipy installed """ if interval_minutes == 60: return hourly_data if len(hourly_data) == 0: return {} intervals_per_hour = 60 // interval_minutes upsampled = {} if method == 'constant': # Simple division for hour, value in hourly_data.items(): for i in range(intervals_per_hour): upsampled[hour * intervals_per_hour + i] = value / intervals_per_hour elif method == 'linear': # Linear power interpolation max_hour = max(hourly_data.keys()) if max_hour == 0: # Only one data point, use constant return upsample_forecast(hourly_data, interval_minutes, method='constant') for hour in range(max_hour): current_wh = hourly_data.get(hour, 0) next_wh = hourly_data.get(hour + 1, 0) # Convert Wh to average power current_power = current_wh # 1 Wh / 1 h = 1 W next_power = next_wh for i in range(intervals_per_hour): idx = hour * intervals_per_hour + i fraction = i / intervals_per_hour # Interpolate power linearly interpolated_power = current_power + (next_power - current_power) * fraction # Convert power to energy for interval interval_hours = interval_minutes / 60 upsampled[idx] = interpolated_power * interval_hours # Handle last hour (can't interpolate beyond) if max_hour in hourly_data: for i in range(intervals_per_hour): idx = max_hour * intervals_per_hour + i upsampled[idx] = hourly_data[max_hour] / intervals_per_hour elif method == 'cubic': # Cubic spline interpolation for smoother transitions # Note: Requires scipy library try: from scipy.interpolate import CubicSpline except ImportError: raise ImportError( "Cubic interpolation requires scipy. " "Install with: pip install scipy" ) max_hour = max(hourly_data.keys()) if max_hour == 0: # Only one data point, use constant return upsample_forecast(hourly_data, interval_minutes, method='constant') # Prepare data for cubic spline hours = sorted(hourly_data.keys()) powers = [hourly_data[h] for h in hours] # Wh values (avg power over 1h) # Create cubic spline (using power values) cs = CubicSpline(hours, powers, bc_type='natural') # Sample at interval points for hour in range(max_hour): for i in range(intervals_per_hour): idx = hour * intervals_per_hour + i # Position in hours (e.g., 0.00, 0.25, 0.50, 0.75 for 15-min) time_position = hour + (i / intervals_per_hour) # Get interpolated power at this position interpolated_power = cs(time_position) # Ensure non-negative (cubic spline can overshoot) interpolated_power = max(0, interpolated_power) # Convert power to energy for interval interval_hours = interval_minutes / 60 upsampled[idx] = interpolated_power * interval_hours # Handle last hour if max_hour in hourly_data: for i in range(intervals_per_hour): idx = max_hour * intervals_per_hour + i upsampled[idx] = hourly_data[max_hour] / intervals_per_hour return upsampled def validate_interval_config(interval_minutes: int) -> None: """Validate interval configuration.""" if interval_minutes not in [15, 60]: raise ValueError( f"time_resolution_minutes must be 15 or 60. Got: {interval_minutes}" ) def get_interval_from_timestamp( timestamp: datetime.datetime, reference: datetime.datetime, interval_minutes: int ) -> int: """Get interval index for a timestamp relative to reference.""" diff_seconds = (timestamp - reference).total_seconds() interval_seconds = interval_minutes * 60 return math.floor(diff_seconds / interval_seconds) ``` ______________________________________________________________________ ## Questions for Stakeholders Before implementation, clarify: 1. **Timeline**: What's the target release date? 1. **Priority**: Is 15-min support critical or nice-to-have? 1. **Resources**: Available developer time for implementation? 1. **Testing**: Access to hardware for end-to-end testing? 1. **Users**: Any beta testers willing to try 15-min mode? 1. **Data**: Anyone with 15-min consumption data to share? 1. **Market**: Which electricity markets support 15-min pricing? 1. **Backward Compatibility**: Must v2.0 work with v1.x configs? ______________________________________________________________________ **Document Version**: 2.0\ **Date**: 2025-10-14\ **Author**: Analysis by GitHub Copilot (Enhanced with comprehensive review)\ **Reviewed by**: GitHub Copilot (Document Reviewer)\ **Status**: Design Proposal - Ready for Implementation **Change Log**: - v1.0 (2025-10-14): Initial analysis - v2.0 (2025-10-14): Comprehensive review with improvements: - Fixed solar interpolation algorithm and examples - Added comparison table for consumption forecast options - Fixed typos and language inconsistencies - Clarified dynamic tariff provider requirements with API links - Expanded configuration design with environment variables - Added error handling and validation section - Added monitoring and observability section - Added rollback procedure - Adjusted timeline to realistic 10-14 weeks - Added comprehensive test specifications with actual code - Added visual diagrams (data flow, decision tree, timeline) - Added stakeholder questionnaire - Added code organization recommendations - Added complete interval_utils.py implementation - Added Grafana dashboard examples - Added performance targets and benchmarks **Next Steps**: 1. Stakeholders answer questions (Appendix: Questions for Stakeholders) 1. Create GitHub issue for implementation tracking 1. Set up project board with phases 1. Begin Phase 1: Foundation (2-3 weeks) **Related Documents**: - `README.MD` - Project overview - `HOWITWORKS.md` - System architecture - `config/batcontrol_config_dummy.yaml` - Configuration reference - Future: `docs/migration-15min.md` - Migration guide (to be created) - Future: `docs/troubleshooting-15min.md` - Troubleshooting (to be created) **Contact**: - GitHub Issues: Report bugs or request clarifications - GitHub Discussions: Ask questions or share experiences - Pull Requests: Contribute implementations ______________________________________________________________________ ## Architecture Decision: Baseclass Pattern ### Key Design Choice Instead of each provider implementing upsampling/downsampling logic, we use a **baseclass pattern**: ``` ┌─────────────────────────────────────────────┐ │ interval_utils.py │ │ (Shared upsampling/downsampling functions) │ └─────────────────────────────────────────────┘ ▲ │ uses ┌───────────────┼───────────────┐ │ │ │ ┌───▼────┐ ┌───▼────┐ ┌───▼────┐ │ Solar │ │ Tariff │ │Consump.│ │ Base │ │ Base │ │ Base │ └────────┘ └────────┘ └────────┘ ▲ ▲ ▲ │ │ │ ├─────┬─────┬───┼───┬───────┬───┼────┬────┐ │ │ │ │ │ │ │ │ │ FCSolar │ evcc│ Awattar Tibber evcc│ CSV │ HA Prognose Solar Tariff Profile Forecast ``` ### Benefits: | Aspect | Old Approach | Baseclass Approach | | -------------------- | ------------------ | ----------------------- | | **Lines of code** | ~200 per provider | ~30 per provider | | **Upsampling logic** | Duplicated N times | Once in baseclass | | **Testing** | Test each provider | Test baseclass once | | **Maintenance** | Fix in N places | Fix in 1 place | | **Consistency** | Risk variations | Guaranteed same | | **New providers** | Reimplement logic | Just declare resolution | | **Dynamic APIs** | Complex switches | Simple attribute | ### Example: Adding New Provider **Old way** (50+ lines): ``` class NewProvider: def get_forecast(self): data = fetch_from_api() if interval_minutes == 15: # Complex upsampling logic (30 lines) ... return data ``` **New way** (15 lines): ``` class NewProvider(ForecastSolarBase): def __init__(self, config): super().__init__(config) self.native_resolution = 60 # That's it! def _fetch_forecast(self): return fetch_from_api() # Just return data, baseclass handles rest ``` ### Implementation Files: ``` src/batcontrol/ ├── interval_utils.py # NEW: Shared upsampling functions │ ├── forecastsolar/ │ ├── baseclass.py # NEW: Base with auto-upsampling │ ├── fcsolar.py # MODIFIED: Inherits from base │ ├── solarprognose.py # MODIFIED: Inherits from base │ └── evcc_solar.py # MODIFIED: Inherits from base │ ├── dynamictariff/ │ ├── baseclass.py # NEW: Base with auto-upsampling │ ├── awattar.py # MODIFIED: Inherits from base │ ├── tibber.py # MODIFIED: Inherits from base │ └── evcc.py # MODIFIED: Inherits from base │ └── forecastconsumption/ ├── baseclass.py # NEW: Base with auto-upsampling ├── forecast_csv.py # MODIFIED: Inherits from base └── forecast_homeassistant.py # MODIFIED: Inherits from base ``` ______________________________________________________________________ ## Summary This document provides a **complete blueprint** for transforming batcontrol from hourly to configurable 15/60-minute intervals. Key takeaways: ✅ **Feasible**: No blocking technical issues identified\ ✅ **Configurable**: Support for 15 and 60-minute intervals\ ✅ **Backward Compatible**: Default to 60 minutes, opt-in to faster intervals\ ✅ **Well-Architected**: Baseclass pattern eliminates code duplication\ ✅ **Well-Tested**: Comprehensive test strategy defined\ ✅ **Monitored**: Performance and health metrics included\ ✅ **Documented**: Clear migration and troubleshooting paths **Estimated Effort**: 10-14 weeks for complete implementation\ **Risk Level**: Low to Medium (with mitigation strategies)\ **Recommended Approach**: Phased rollout with beta testing The document is now **ready to guide implementation**. Please answer the stakeholder questions and proceed with Phase 1.