I'm working with a cleaned hourly table of electricity generation by source (MW), indexed by UTC timestamp.
I want to predict total thermal generation from the renewable sources. That means:
- the target is the sum of several thermal series, but it should be missing if any one of them is missing (a missing reading isn't zero)
- the inputs are the renewable series plus hour, day of week and month
- only hours where the target and every input are present should be kept
- the last full year should be held out as a test set, and the two years before it used as validation years in expanding-window folds
I can't split randomly, because neighbouring hours are almost identical and that would leak information. How can I set this up in pandas?