Visiting Researcher at Meta

Since about a week I’m on a sabbatical for 6 months in the Infrastructure Data Science and Engineering team at Meta Platforms Inc. in Menlo Park, California. I’m fortunate that Zeynep Erkin Baz, Dario Benavides, and Ashish Kelkar have given me this opportunity to work with them in their great team. I’m looking forward to tackling interesting forecasting problems and learning a lot more about forecasting and other things along the way.

I’ve also just updated my web presences since some time and let me take this opportunity to point you to some of the more interesting of my most recent work in Global Forecasting Models (GFM), as I often don’t post about them in a very coordinated way, so you may have missed some of it:

• GFMs for Causal Inference (with Priscila Grecov and others). We are able to show that GFMs can be trained across both treated and control time series, which has some advantages over other methods such as Google’s CausalImpact. Our first paper in the space is here, in our second paper we extend the framework to probabilistic forecasting, if interventions affect, e.g., mostly the extremes of a time series, and in another paper we take a more practical approach and show the framework in action to analyse the effect of Covid-19 on electricity consumption patterns in Europe and Australia. We hope to soon implement the model into GluonTS.

• Explainability for GFMs (with Dilini Rajapaksha and Rob Hyndman). In the LOMEF paper we show how local models can be used to interpret GFMs. So, let’s say you want to use a GFM but your stakeholders are most comfortable with, e.g., ETS, you can run ETS on the in-sample fit of the GFM to that series and present that as an explanation of what the GFM does. LIMREF is another work where we train a surrogate explainable model for explanations, in particular we build a surrogate dataset with more understandable features and extract impact rules from this dataset as explanations. We used this model to win the FUZZ-IEEE Competition on Explainable Energy Prediction in 2020.

• Datasets and Benchmarks for GFMs (with Hansika Hewamalage, Rakshitha Godahewa, and others). We put together a data repository of datasets that some of you will already know, here. We also wrote a tutorial paper about forecast evaluation for data scientists here. We have recently updated the paper for a journal revision and looked at evaluation setups of very recent papers in top conferences and journals. I hope we can update our arxiv paper soon with these new findings, but as a spoiler: Informer looses against DHR-ARIMA in the setup of the original Informer paper, and predicting daily exchange rates nearly 2 years out with no other information but past exchange rates is as adventurous as it sounds: Autoformer and others loose against naive, which they didn’t benchmark against in their respective original papers. There is another interesting paper out here that argues that these papers only win against relatively simple benchmarks because they have very long forecasting horizons and do direct forecasting, whereas their benchmarks need to iterate out and accumulate the error.

• Predict+Optimize (with Frits de Nijs, Peter Stuckey, and others). In a great collaboration with Monash’s Optimization group we think about optimization methods that have forecasts as inputs, and how these forecasts ideally should look like to make the optimization work well. Last year, I chaired the technical committee of the IEEE-CIS Technical Challenge on Energy Prediction From Smart Meter Data where we used solar and building demand data from the Monash Microgrid to forecast power consumption and solar power production, and then optimize battery and lecture schedules based on this. The competition web page has many interesting resources such as reports, slides and code from the 7 shortlisted solutions. The winning team forecast different scenarios with a LightGBM-based method, and then used the ``Sample Average Approximation Method (SAAM) in which the optimisation model minimises the average cost of a solution over multiple scenarios’’ (citing from their report). Again, we are currently finishing a paper that I can hopefully soon upload to arxiv.

• Time Series Features (with Alexey Chernikov and others). The paper is here. It talks about the very interesting idea of static and dynamic time series features, and argues that for forecasting we are often interested in static features that describe a time series as a whole, in distinction to other series, and that don’t change over time. In contrast, dynamic features describe the state of a time series at a current point in time. We then argue that autoencoders extract dynamic features, so that feature extraction with autoencoders for forecasting has only limited use and use cases. We also present a method to extract static features and show that within FFORMA it beats the original FFORMA on M4.

• SETAR-Tree GFMs (with Rakshitha Godahewa, Geoff Webb, and Daniel Schmidt). The paper is still not on arxiv, but again it should be out soon, and I gave a talk about it at ISF 2022 and a recording is here. The main idea is that linear models and off-the-shelf non-time-series-specific Gradient-Boosted-Trees performed very well in the M5. We construct a global piecewise hierarchical linear model, i.e., you can see it as a hierarchical SETAR or a decision tree with linear models in the leaves, with stopping criteria taken from the SETAR literature, and some cool tricks to make the computation fast, and show that it achieves good accuracy on many datasets.

Finally, I’m also very humbled and fortunate that I have been offered both a Ramón y Cajal Scholarship and a María Zambrano (Senior) Scholarship from Spain, in their 2022 rounds. That will have some impact on my plans for next year :)