Continuous Optimization DAG Learning in the Presence of Location Scale Noise: A Systematic Evaluation of Different Frameworks

Abstract

Causal discovery concerns the problem of learning the causal structures between variables of a system from observational data. To tackle the computational complexity when mul- tiple variables are involved, continuous optimization is a growing area. However, most approaches rely on additive noise, and no systematic evaluation of non-additive location- scale noise (LSN) has been performed. Modeling LSN or heteroscedasticity is important as it is common in many real-world systems. If heteroscedasticity is not modeled, this can lead to predicting the wrong causal structure. We build upon recent advances in contin- uous optimization for structure learning and provide extensions to a current method to work in the presence of location-scale noise. Further, we consider an existing method that models LSN and has not been evaluated on synthetic data. We provide extensive syn- thetic experiments demonstrating the superiority of LSN methods over current continuous methods that do not model LSN. The existing method modeling LSN achieves the best performance on all experiment types except one, where our proposed methods perform better. On the other hand, we do not observe performance improvements by modeling LSN on real-world data sets. Finally, we discuss the limitations and insights of learning the causal structure through continuous optimization-based approaches and propose multiple ideas to improve our methods.