Summary of recent work on causal representation learning, causal discovery, and machine learning

(see the publication list for the papers)

• Methodological developments of causal representation learning in the i.i.d. case

  • estimating latent variables and their relations in the linear-Gaussian case: a versatile rank deficiency-based approach, known as Rank-based Latent Causal Discovery (RLCD), applications to psychometrical studies (Dong et al., ICLR'24) and parameter identifiability (Dong et al., NeurIPS'24) and and its earlier version that does not allow causal links between measured variables (Huang & Low et al., NeurIP'22)--they can deal with linear-Gaussian hierarchical structural as a special case; score-based causal discovery in the presence of causally-related latent variables (Ng et al., ICML'24)
  • estimating latent variables and their relations in the linear, non-Gaussian case: sufficient and necessary theoretical identifiability results (Adams et al., NeurIPS'21); the Generalized Independent Noise (GIN) condition-based practical methods (Jin et al., ICLR'24; Xie et al., JMLR'24; Xie et al., NeurIPS'20; Li et al, ICLR'24; Cai et al., NeurIPS'19); dealing with linear, non-Gaussian latent hierarchical structure as a special case (Xie et al., ICML'22)
  • establishment of the identifiability of nonlinear ICA and its variants (to allow dependence, for instance) based on proper sparsity constraints (Zheng et al., NeurIPS'22; Zheng & Zhang, NeurIPS'23; Ng et al., NeurIPS'23); using sparsity to improve the understanding of extrapolation (Kong et al., NeurIPS'24)
  • learning discrete concepts from images (Kong et al., NeurIPS'24; Chen et al., NeurIPS'24).

• Methodological developments of causal representation learning in the non-i.i.d. case (with temporal constraints and/or multiple distributions)

  • general setting of causal representation learning from multiple distributions (Zhang et al., ICML'24); learning hidden changing component in nonparametric cases with partial disentanglement with component-wise identifiability (Kong et al., ICML'22; Xie et al., ICLR'23) or subspace identifiability (Li et al., NeurIPS'23).
  • learning latent temporal causal processes from time series assuming invertibility of the mixing procedure(Yao et al., ICLR'22; Yao et al., NeurIPS'22) or without this assumption (Chen et al., ICML'24); applications to reasoning-based video question answer (Chen et al., ICLR'24); learning latent processes with nonstationary sparse transition (Song et al., NeurIPS'24; NeurIPS'23)
  • as an extension, learning interpretable world model for reinforcement learning (Liu et al., NeurIPS'23) or action-sufficient state representations in reinforcement learning (Huang et al., ICML'22);
  • learning general linear structure with latent variables in the linear, non-Gaussian or heterogeneous case: theoretical identifiability results (Adams et al., NeurIPS'21).

• Principles for causal discovery:

  • causal discovery in the presence of deterministic causal relations, in light of the “monotonicity” principle that having access to more variable will not hurt causal discovery results (Li et al., NeurIPS'24);
  • feasibility of causal discovery from temporally aggregated data (Fan et al., ICML'24); causal discovery from discretized continuous variables with corrected tests (Sun et al., arxiv'24)
  • independent noise in (post-)nonlinear causal model (Zhang and Hyvärinan, UAI’09 & ECML’09; Zhang and Chan, ICONIP’06);
  • independent transformation in deterministic systems (Janzing et al., AI12 & Daniusis et al., UAI’10);
  • exogeneity, as a way to characterize the `modularity' property of causal systems (Zhang et al., TARK’15);
  • independent changes (generalized notation of invariance) in nonstationary/heterogeneous data (Huang & Zhang et al., JMLR'20; Zhang et al., IJCAI’17; Huang et al., ICDM’17; Zhang et al., arxiv’15...); constraints on and estimation of functional causal models (Zhang et al., TIST'16);
  • Generalized independent noise conditions (including Triad conditions) for estimating linear, non-Gaussian hidden causal representations (see above).

• Review papers on causal discovery and causality-related learning

  • causal discovery in biology (Glymour, Zhang, and Spirtes, 2019) and public health (Guo et al., NC'24);
  • causal discovery in earth system sciences (Runge et al., NC'19);
  • cyclic causal model discovery in neuroscience (Sanchez-Romero et al, 2019);
  • evaluation of causal discovery methods (Ramsey, Zhang, and Glymour, 2019);
  • general reviews of causal discovery methods (Zhang et al., NSR18; Spirtes & Zhang, Applied_Informatics2016 & BookChapter2018).

• Causal discovery from various types of nonstationary and heterogeneous data

  • general framework for causal discovery from independent but non-identically distributed time series or multiple-domain data and beyond (Zhang et al., ICML'24; Kong et al., ICML'22; Huang & Zhang et al., JMLR’20;Zhang et al., IJCAI’17; Huang et al., ICDM’17; Zhang et al., arxiv’15);
  • learning hidden causal variables with changing distributions (Zhang et al., ICML'24; Kong et al., ICML'22);
  • causal discovery and forecasting in nonstationary environments with state-space modeling (Huang et al., ICML’19);
  • causal discovery with fixed general nonlinear causal mechanisms and nonstationary noise (Monti et al., UAI’19);
  • modeling and estimation of time-varying causal relations with Gaussian processes (Huang et al., IJCAI’15);
  • multi-domain causal structure learning in linear systems with regression invariance (Ghassami et al., NIPS'17) or with independent changes (Ghassami et al., NeurIPS’18).

• Functional causal model-based causal discovery

  • theory and methods for causal discovery based on the post-nonlinear (PNL) causal model (Zhang and Hyvärinan, UAI’09 & JMLR WCP’10 (NIPS'08 Workshop); Zhang and Chan, ICONIP’06);
  • nonlinear causal models with additive noise and application to time series (Zhang and Hyvärinan, ECML’09);
  • generalized score-based search of general nonlinear causal relations (Huang et al., KDD’18);
  • cascade nonlinear additive noise model (Cai et al., IJCAI’19);
  • causal discovery with fixed general nonlinear causal mechanisms and nonstationary noise (Monti et al., UAI’19);
  • local causal discovery with linear non-gaussian cyclic models (Dai et al., AISTATS'24).

• Detection of or handling selection bias

  • handling dropouts in gene regulatory network (Dai et al., ICLR'24);
  • detecting selection pattern in sequential data such as music (Zheng et al., ICML'24);
  • learning subtasks from demonstration trajectories for causal understanding and generating novel solutions by framing subtasks as selection problems (Qiu et al., NeurIPS'24);
  • causal discovery under selection bias (Zhang et al., UAI’16).

• Other practical issues in causal discovery

  • Causal discovery from low-resolution or partially observable time series

    • causality discovery in time series as constraint-based or functional causal model-based causal discovery with temporal constraints & time-delayed and instantaneous relations (Zhang et al., ECML'09; Hyvarinen et al., JMLR'10);
    • issues with causal discovery from temporally aggregated (nonlinear) data (Fan et al., ICML'24); causal discovery from subsampling / temporally aggregation (Gong et al., ICML'15 & UAI'17);
    • causal discovery from partially observable time series (Geiger et al., ICML'15; Salehkaleyber et al., AAA’18).

  • Causal discovery in the presence of measurement error or confounders

    • causal discovery with linear, non-Gaussian models under measurement error (Zhang et al., UAI’18);
    • both linear, Gaussian and linear, non-Gaussian cases (Zhang et al., UAI WS’17);
    • independence testing-based approach to causal discovery under measurement error and linear non-Gaussian models (Tang et al., NeurIPS'22);
    • learning Linear Non-Gaussian Causal Models in the Presence of Latent Variables (Salehkaleybar et al., JMLR’20).

  • Causal discovery under missing values: Constraint-based causal discovery in the presence of missing values (Tu et al., AIStats’19).

  • Causal discovery in discrete or mixed continuous and discrete cases

    • causal search based on generalized score functions that apply to general nonlinear relations and mixed cases (Huang et al., KDD’18);
    • causal discovery from discrete variables with hidden compact representations (Cai et al., NeurIPS’18).

  • Conditional independence test

    • kernel-based conditional independence test (KCI-test) with application to causal discovery (Zhang et al., UAI’11);
    • permutation-based kernel conditional independence test (Doran et al., UAI’14);
    • approximate kernel-based conditional independence tests for causal discovery (Strobl et al., 2019).

• Domain adaptation / transfer learning, reinforcement learning, as well as other learning problems from a causal perspective

  • domain adaptation as a problem of Bayesian inference on the learned graphical presentation: a principled, end-to-end framework of domain adaptation (Zhang & Gong et al., NeurIPS’20);
  • partial disentanglement: learning changing hidden sources for domain adaptation (Kong et al., ICML'22);
  • subspace identifiability for domain adaptation (Li et al., NeurIPS'23);
  • causal and anti-causal learning (Schölkopf et al., ICML’12); domain adaptation under target and conditional shift (Zhang et al., ICML’13);
  • a general causal view of domain adaptation (Zhang et al., AAAI’15);
  • domain adaptation with conditionally transferrable components or invariant mechanisms (Gong et al., ICML’16);
  • domain adaptation with invariant representation learning: what transformations to learn? (Stojanov et al., NeurIPS'21);
  • data-driven approach to multiple-source domain adaptation (Stojanov et al., AIStats'19a);
  • adaptive reinforcement learning (Huang et al., ICLR'22; Feng et al., NeurIPS'22);
  • unaligned image-to-image translation by Learning to reweight with changing distributions for the content (Xie et al., ICCV'21);
  • unsupervised image-to-image translation with density changing regularization (Xie et al., ICLR'23; Xie et al., NeurIPS'22);
  • low-dimensional density ratio estimation for covariate shift correction (Stojanov et al., AIStats'19b);
  • properties of invariant component-based domain adaptation (Zhao et al., ICML’19);
  • geometry-consistent GANs for one-sided unsupervised domain mapping (Fu et al., CVPR’19);
  • deep domain generalization via conditional invariant adversarial networks (Li et al., ECCV’18);
  • domain generalization via multi-domain discriminant analysis (Hu et al., UAI’19);
  • multi-label learning by exploiting label dependency (Zhang & Zhang, KDD’10);
  • learning disentangled semantic representation for domain adaptation (Cai et al., IJCAI’19);
  • causal discovery and forecasting in nonstationary environments with state-space models (Huang et al., ICML’19);
  • xausal treatment of recommender systems (Wang et al., AAAI'18; Wang et al., NeurIPS'18);
  • counterfactual generation of text and images (Yan & Kong et al., NeurIPS'23; Sun et al., AAAI'24); natural counterfactual reasoning (Hao et al., NeurIPS'24);
  • advancing the understanding and implementation of fairness in machine learning (Tang et al., ICLR'24; Li et al., NeurIPS'24); attainability of optimality of certain fairness constraints (Tang & Zhang, CLeaR'22); counterfactual fairness with partially known causal graph (Zuo et al., NeurIPS'22).

Academic service

• Associate editor for
  • Journal of the American Statistical Association (JASA)
  • ACM Computing Surveys
  • Pattern Recognition

• Organizational activities
• Program co-chair of the 2024 IEEE International Conference on Data Mining (ICDM)
• Co-organizer of NeurIPS'24 Workshop on Causal Representation Learning
• General co-chair of the 39th Conference on Uncertainty in Artificial Intelligence (UAI 2023)
• Program co-chair of the 38th Conference on Uncertainty in Artificial Intelligence (UAI 2022)
• General & program co-chair of the 1st Conference on Causal Learning and Reasoning (CLeaR 2022)
• Co-organizer of the 9th Causal Inference Workshop at UAI, 2021
• Co-organizer of NeurIPS 2020 Worksshop on Causal Discovery and Causality-Inspired Machine Learning (with Biwei Huang, Sara Magliacane, Danielle Belgrave, Elias Bareinboim, Danial Malinsky, Thomas Richardson, Christopher Meek, Peter Spirtes, and Bernhard Schölkopf), 2020
• Co-organizer of the Weakly-supervised and Unsupervised Learning Workshop at SIAM International Conference on Data Mining 2020 (SDM20) (with Mingming Gong, Chunyuan Li, Tongliang Liu, Bo Han, Quanming Yao, Gang Niu, and Masashi Sugiyama), Ohio, USA, May 9, 2020
• Co-organizer of the 2019 ACM SIGKDD Workshop on Causal Discovery (with Thuc Le, Jiuyong Li, Emre Kiciman, Peng Cui, and Aapo Hyvärinen), Alaska, August, 2019
• Co-organizer of the 2019 international Workshop on Causal Modeling and Machine Learning (with Ruichu Cai Zhifeng Hao), Guangzhou, China, November, 2019
• Guest editor of the ACM Transactions on Intelligent Systems and Technologies (ACM TIST) Special Issue on Causal Discovery and Inference (with Jiuyong Li, Emre Kiciman, and Peng Cu), 2018
• Co-organizer of the 2018 international Workshop on Causal Modeling and Machine Learning (with Ruichu Cai Zhifeng Hao), Guangzhou, China, June, 2018
• Co-organizer of the 2018 ACM SIGKDD Workshop on Causal Discovery (with Thuc Le, Emre Kiciman, Aapo Hyvärinen, and Lin Liu), London, England, August, 2018
• Co-organizer of the 2017 ACM SIGKDD Workshop on Causal Discovery (with Lin Liu, Jiuyong Li, Emre Kiciman, and Negar Kiyavash)
• Co-organizer of Workshop “Causality: Dialogues between Machine Learning and Psychology” at Data Learning and Inference (DALI) 2017 (with David Danks and Felix Wichmann), April 18, 2017
• Co-organizer of The UAI 2017 Workshop on Causality: Learning, Inference, and Decision-Making (with Elias Bareinboim, Caroline Uhler, Jiji Zhang, and Dominik Janzing), Sydney, Australia, August 15, 2017
• Co-organizer of AMIA 2017 Pre-symposium Workshops on Data Mining for Medical Informatics (DMMI) – Causal Inference for Health Data Analytics (with Kenney Ng, Bisakha Ray, SiSi Ma, and Fei Wang)
• Co-organizer of the Munich Workshop on Causal Inference and Information Theory (with Negar Kiyavash and Gerhard Kramer), May 23-24, 2016
• Co-organizer of the 2016 ACM SIGKDD Workshop of Causal Discovery (With Jiuyong Li, Elias Bareinboim, and Lin Liu)
• Guest editor of the Journal of Data Science and Analytics Special Issue on Causal Discovery (with Jiuyong Li, Elias Bareinboim, and Lin Liu), 2016
• Organizer of workshop “Causal modeling & machine learning” at ICML 2014 (with Bernhard Schölkopf , Eias Bareinboim, and Jiji Zhang), Beijing, China, June, 2014
• Guest editor of the ACM Transactions on Intelligent Systems and Technologies (ACM TIST) Special Issue on Causal Discovery and Inference (with Jiuyong Li, Elias Bareinboim, Bernhard Schölkopf, and Judea Pearl), 2013 - 2014
• Organizer of workshop “Causality: Perspectives from different disciplines” (with Jiji Zhang and Bernhard Schölkopf), Vals, Switzerland, August, 2013
• Co-organizer and program chair of the First IEEE / ICDM Workshop on Causal Discovery (CD 2013, with Jiuyong Li, Lin Liu, and Jian Pei)
• Co-organizer of IJCNN’13 cause-effect pairs challenge (causality challenge #3)
• Co-organizer of workshop “Networks -- Processes and causality” (with Manuel G. Rodriguez, and Bernhard Schölkopf), Menorca, Spain, September, 2012
• Publicity chair of AISTATS 2012 (15th International Conference on Artificial Intelligence and Statistics)
• Organizer and chair of special session on ICA at ICONIP 2006

• (Senior) program committee member/area chair for international conferences
• 2025: CLeaR (area chair), ICML (area chair), AISTATS (senior area chair), ICLR (senior area chair)...
• 2024: CLeaR (area chair), UAI (area chair), ICML (area chair), NeurIPS (senior area chair), AISTATS (senior area chair), ICLR (senior area chair), ICDM (program co-chair)
• 2023: CLeaR (area chair), UAI (area chair), ICML (area chair), NeurIPS (area chair), AISTATS (area chair), ICLR (area chair)
• 2022: CLeaR (program co-chair), UAI (program co-chair), ICLR (area chair), AISTATS (area chair)
• 2021: IJCAI (senior area chair), ICLR (area chair), AISTATS (area chair), ICML (area chair), UAI (area chair), NeurIPS (area chair);
• 2020: AAAI (area chair), UAI (senior PC), ICML (area chair), AISTATS (senior PC), NeurIPS (area chair), IJCAI (senior PC), ICONIP (senior PC);
• 2019: ICLR, AAAI (area chair), ICML, KDD, UAI (senior PC), NeurIPS (area chair), IJCAI (senior PC), IScIDE (program co-chair), ACML (senior PC);
• 2018: ICLR, ICML (area chair), IJCAI (senior PC), UAI (senior PC), KDD, NeurIPS (area chair), ACML (senior PC), ICDM (area chair);
• 2017: AISTATS (senior PC), IJCAI (senior PC), AAAI, ICML, UAI, NIPS (area chair), KDD, ACML (senior PC), AMBN;
• 2016: AISTATS (senior PC), IJCAI (senior PC), ICML, KDD (research track), NIPS (area chair), UAI (senior PC), AAAI;
• 2015: AISTATS, KDD, UAI, IJCAI, ECML-PKDD, NIPS;
• 2014: AISTATS (senior PC), WSDM, KDD (research & industry tracks), iKDD CoDS, UAI, NIPS, ACML;
• 2013: UAI, NIPS, AISTATS, SDM, KDD, IJCAI, IJCNN, Big Data;
• 2012: UAI, AISTATS, MLSP, WSDM, SDM;
• 2011: NIPS, UAI, KDD, IJCNN, ICONIP;
• 2010: NIPS, UAI, ICA/LVA, SDM, ACML, ICPR, ICNC-FSKD;
• 2009: NIPS, ACML, ICONIP;
• 2007: MLSP, IDEAL, ISNN; 2006: ICONIP, DSN;
• 2005: PhysCon;