publications

2023

1. NeurIPS

   ADGym: Design Choices for Deep Anomaly Detection

   Jiang, Minqi, Hou, Chaochuan, Zheng, Ao, Han, Songqiao, Huang, Hailiang, Wen, Qingsong, Hu, Xiyang, and Zhao, Yue

   Advances in Neural Information Processing Systems 2023

   Abs

   Deep learning (DL) techniques have recently been applied to anomaly detection (AD), with numerous successful applications in finance, medical service, cloud computing, etc. However, current research often directly evaluates a deep AD algorithm as a whole, which cannot disentangle the contribution of each design choice (e.g., loss functions and network architectures). Meanwhile, we may neglect the contribution of other meaningful prerequisite steps like preprocessing by giving all credits to newly designed loss functions and/or architectures. In this paper, we address the above gaps by answering: (i) which components (i.e., design choices) of deep AD methods play crucial roles in detecting anomalies? (ii) how can we build more effective AD algorithms given different datasets by automatically choosing the optimal design choices other than using existing off-the-shelf ones? To this end, we propose ADGym, the first and fully open-source design platform for large evaluation and automatic selection of AD design choices. Extensive experiments show that directly using existing leading methods is not optimal, and the AD models composed by ADGym significantly outperform state-of-the-art methods.

2. ACL

   Language Agnostic Multilingual Information Retrieval with Contrastive Learning

   Hu, Xiyang, Chen, Xinchi, Qi, Peng, Kong, Deguang, Liu, Kunlun, Wang, William Yang, and Huang, Zhiheng

   Annual Meeting of the Association for Computational Linguistics - Findings of ACL 2023

   Abs HTML

   Multilingual information retrieval (IR) is challenging since annotated training data is costly to obtain in many languages. We present an effective method to train multilingual IR systems when only English IR training data is available, by leveraging parallel and non-parallel corpora to improve the pretrained multilingual language models’ cross-lingual transfer ability. We design a semantic contrastive loss to align representations of parallel sentences that share the same semantics in different languages, and a new language contrastive loss to leverage parallel sentence pairs to remove language-specific information in sentence representations from non-parallel corpora. When trained on English IR data with these losses and evaluated zero-shot on non-English data, our model demonstrates significant improvement to prior work on retrieval performance, while it requires much less computational effort. Our model can work well even with a small number of parallel sentences, and it can be used as an add-on module to any backbone and other tasks.

3. preprint

   Inclusive Decision Making via Contrastive Learning and Domain Adaptation

   Hu, Xiyang, Huang, Yan, Li, Beibei, and Lu, Tian

   MIS Quarterly (Under Major Revision) 2023

   HTML
4. preprint

   Weakly Supervised Anomaly Detection: A Survey

   Jiang, Minqi, Hou, Chaochuan, Zheng, Ao, Hu, Xiyang, Han, Songqiao, Huang, Hailiang, He, Xiangnan, Yu, Philip S, and Zhao, Yue

   arXiv preprint arXiv:2302.04549 2023

   HTML

2022

1. preprint

   Human-Algorithmic Bias: Source, Evolution, and Impact

   Hu, Xiyang, Huang, Yan, Li, Beibei, and Lu, Tian

   Management Science (Under Major Revision) 2022

   Abs HTML

   Prior work on human-algorithmic bias has seen difficulty in empirically identifying the underlying mechanisms of bias, because in a typical "one-time" decision-making scenario, different mechanisms tend to generate the same patterns of observable decisions. In this study, leveraging a unique repeat decision-making setting in a high-stakes micro-lending context, we aim to uncover the underlying source, evolution dynamics, and associated impacts of bias. We first develop and estimate a structural econometric model of the decision dynamics to understand the source and evolution of potential bias in human evaluators in microloan granting. We find that both preference-based bias and belief-based bias are present in human evaluators’ decisions and are in favor of female applicants. Through counterfactual simulations, we quantify the effects of the two types of bias on both fairness and profits. The results show that the elimination of either of the two biases improves the fairness in financial resource allocation, as well as the platform profits. The profit improvement mainly stems from the increase in the approval probability for male borrowers, especially those who would eventually pay back loans. Furthermore, to examine how human biases evolve when being inherited by machine learning (ML) algorithms, we then train a set of state-of-the-art ML algorithms for default risk prediction on both real-world datasets with human biases encoded within and counterfactual datasets with human biases partially or fully removed. By comparing the decision outcomes in different counterfactual settings, we find that even fairness-unaware ML algorithms can reduce bias present in human loan-granting decisions. Interestingly, while removing both types of human biases from the training data can further improve ML fairness, the fairness-enhancing effects vary significantly between new and repeat applicants. Based on our findings, we discuss how to reduce decision bias most effectively in a human-machine learning pipeline.

2. NeurIPS

   ADBench: Anomaly Detection Benchmark

   Hu, Xiyang, Han, Songqiao, Huang, Hailiang, Jiang, Mingqi, and Zhao, Yue

   Advances in Neural Information Processing Systems 2022

   Abs HTML Code

   Given a long list of anomaly detection algorithms developed in the last few decades, how do they perform with regard to (i) varying levels of supervision, (ii) different types of anomalies, and (iii) noisy and corrupted data? In this work, we answer these key questions by conducting (to our best knowledge) the most comprehensive anomaly detection benchmark with 30 algorithms on 55 benchmark datasets, named ADBench. Our extensive experiments (93,654 in total) identify meaningful insights into the role of supervision and anomaly types, and unlock future directions for researchers in algorithm selection and design. With ADBench, researchers can easily conduct comprehensive and fair evaluations for newly proposed methods on the datasets (including our contributed ones from natural language and computer vision domains) against the existing baselines. To foster accessibility and reproducibility, we fully open-source ADBench and the corresponding results.

3. NeurIPS

   Benchmarking Node Outlier Detection on Graphs

   Liu, Kay, Dou, Yingtong, Zhao, Yue, Ding, Xueying, Hu, Xiyang, Zhang, Ruitong, Ding, Kaize, Chen, Canyu, Peng, Hao, Shu, Kai, and others,

   Advances in Neural Information Processing Systems 2022

   Abs HTML Code

   Graph outlier detection is an emerging but crucial machine learning task with numerous applications. Despite the proliferation of algorithms developed in recent years, the lack of a standard and unified setting for performance evaluation limits their advancement and usage in real-world applications. To tap the gap, we present, (to our best knowledge) the first comprehensive unsupervised node outlier detection benchmark for graphs called UNOD, with the following highlights: (1) evaluating fourteen methods with backbone spanning from classical matrix factorization to the latest graph neural networks; (2) benchmarking the method performance with different types of injected outliers and organic outliers on real-world datasets; (3) comparing the efficiency and scalability of the algorithms by runtime and GPU memory usage on synthetic graphs at different scales. Based on the analyses of extensive experimental results, we discuss the pros and cons of current UNOD methods and point out multiple crucial and promising future research directions.

4. ICIS

   Credit Risk Modeling without Sensitive Features: An Adversarial Deep Learning Model for Fairness and Profit

   Hu, Xiyang, Huang, Yan, Li, Beibei, and Lu, Tian

   International Conference on Information Systems 2022

   Abs HTML

   We propose an adversarial deep learning model for credit risk modeling. We make use of sophisticated machine learning model’s ability to triangulate (i.e., infer the sensitive group affiliation by using only permissible features), which is often deemed “troublesome” in fair machine learning research, in a positive way to increase both borrower welfare and lender profits while improving fairness. We train and test our model on a dataset from a real-world microloan company. Our model significantly outperforms regular deep neural networks without adversaries and the most popular credit risk model XGBoost, in terms of both improving borrowers’ welfare and lenders’ profits. Our empirical findings also suggest that the traditional AUC metric cannot reflect a model’s performance on the borrowers’ welfare and lenders’ profits. Our framework is ready to be customized for other microloan firms, and can be easily adapted to many other decision-making scenarios.

5. JMLR

   PyGOD: A Python Library for Graph Outlier Detection

   Liu, Kay, Dou, Yingtong, Ding, Xueying, Hu, Xiyang, Zhang, Ruitong, and others,

   Journal of Machine Learning Research (Under Minor Revision) 2023

   Abs HTML Code

   PyGOD is an open-source Python library for detecting outliers on graph data. As the first comprehensive library of its kind, PyGOD supports a wide array of leading graph-based methods for node-, edge-, subgraph-, and graph-level outlier detection, under a unified, well-documented API designed for use by both researchers and practitioners. To overcome the scalability issue in large graphs, we provide advanced functionalities for selected models, including mini-batch and sampling. PyGOD is equipped with best practices to foster code reliability and maintainability, including unit testing, continuous integration, and code coverage. To foster accessibility, PyGOD is released under a permissive BSD-license at this https URL and the Python Package Index (PyPI).

6. TKDE

   ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions

   Li, Zheng, Zhao, Yue, Hu, Xiyang, Botta, Nicola, Ionescu, Cezar, and Chen, George H

   IEEE Transactions on Knowledge and Data Engineering 2022

   Abs HTML Code

   Outlier detection refers to the identification of data points that deviate from a general data distribution. Existing unsupervised approaches often suffer from high computational cost, complex hyperparameter tuning, and limited interpretability, especially when working with large, high-dimensional datasets. To address these issues, we present a simple yet effective algorithm called ECOD (Empirical-Cumulative-distribution-based Outlier Detection), which is inspired by the fact that outliers are often the "rare events" that appear in the tails of a distribution. In a nutshell, ECOD first estimates the underlying distribution of the input data in a nonparametric fashion by computing the empirical cumulative distribution per dimension of the data. ECOD then uses these empirical distributions to estimate tail probabilities per dimension for each data point. Finally, ECOD computes an outlier score of each data point by aggregating estimated tail probabilities across dimensions. Our contributions are as follows: (1) we propose a novel outlier detection method called ECOD, which is both parameter-free and easy to interpret; (2) we perform extensive experiments on 30 benchmark datasets, where we find that ECOD outperforms 11 state-of-the-art baselines in terms of accuracy, efficiency, and scalability; and (3) we release an easy-to-use and scalable (with distributed support) Python implementation for accessibility and reproducibility.

2021

1. ICIS

   Uncovering the Source of Evaluation Bias in Micro-Lending

   Hu, Xiyang, Huang, Yan, Li, Beibei, and Lu, Tian

   International Conference on Information Systems 2021

   Abs HTML PDF

   We develop a structural econometric model to capture the decision dynamics of human evaluators on an online micro-lending platform, and estimate the model parameters using real-world data. We find two types of biases in gender, i.e. preference-based bias and belief-based bias, are present in human evaluators’ decisions. Both types of biases are in favor of female applicants. Through counterfactual simulations, we quantify the effect of gender bias on loan granting outcomes and the welfare of the company and the borrowers. Our results imply that both the existence of the preference-based bias and that of the belief-based bias reduce the company’s profits. When the preference-based bias is removed, the company earns more profits. When the belief-based bias is removed, the company’s profits also increase. Both increases result from lowering the approval probability for borrowers, especially female borrowers, who eventually default on loans. For borrowers, the elimination of either bias decreases the gender gap in the credit risk evaluation.

2. MLSys

   SUOD: Accelerating Large-scale Unsupervised Heterogeneous Outlier Detection

   Hu, Xiyang, Zhao, Yue, Cheng, Cheng, Wang, Cong, Wan, Changlin, Wang, Wen, Yang, Jianing, Bai, Haoping, LI, Zheng, Xiao, Cao, Wang, Yunlong, Qiao, Zhi, Sun, Jimeng, and Akoglu, Leman

   Conference on Machine Learning and Systems 2021

   Abs HTML

   Outlier detection (OD) is a key data mining task for identifying abnormal objects from general samples with numerous high-stake applications including fraud detection and intrusion detection. Due to the lack of ground truth labels, practitioners often have to build a large number of unsupervised models that are heterogeneous (i.e., different algorithms and hyperparameters) for further combination and analysis with ensemble learning, rather than relying on a single model. However, this yields severe scalability issues on high-dimensional, large datasets. How to accelerate the training and predicting with a large number of heterogeneous unsupervised OD models? How to ensure the acceleration does not deteriorate detection models’ accuracy? How to accommodate the acceleration need for both a single worker setting and a distributed system with multiple workers? In this study, we propose a three-module acceleration system called SUOD (scalable unsupervised outlier detection) to address these questions. It focuses on three complementary aspects to accelerate (dimensionality reduction for high-dimensional data, model approximation for complex models, and execution efficiency improvement for taskload imbalance within distributed systems), while controlling detection performance degradation. Extensive experiments on more than 20 benchmark datasets demonstrate SUOD’s effectiveness in heterogeneous OD acceleration. By the submission time, the released open-source system has been widely used with more than 700,000 times downloads. A real-world deployment case on fraudulent claim analysis at IQVIA, a leading healthcare firm, is also provided.

3. KDD

   Uncovering the Source of Machine Bias

   Hu, Xiyang, Huang, Yan, Li, Beibei, and Lu, Tian

   27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Machine Learning for Consumers and Markets Workshop 2021

   Abs HTML

   We develop a structural econometric model to capture the decision dynamics of human evaluators on an online micro-lending platform, and estimate the model parameters using a real-world dataset. We find two types of biases in gender, preference-based bias and belief-based bias, are present in human evaluators’ decisions. Both types of biases are in favor of female applicants. Through counterfactual simulations, we quantify the effect of gender bias on loan granting outcomes and the welfare of the company and the borrowers. Our results imply that both the existence of the preference-based bias and that of the belief-based bias reduce the company’s profits. When the preference-based bias is removed, the company earns more profits. When the belief-based bias is removed, the company’s profits also increase. Both increases result from raising the approval probability for borrowers, especially male borrowers, who eventually pay back loans. For borrowers, the elimination of either bias decreases the gender gap of the true positive rates in the credit risk evaluation. We also train machine learning algorithms on both the real-world data and the data from the counterfactual simulations. We compare the decisions made by those algorithms to see how evaluators’ biases are inherited by the algorithms and reflected in machine-based decisions. We find that machine learning algorithms can mitigate both the preference-based bias and the belief-based bias.

4. JAH

   开化寺大雄宝殿彩画测绘复原

   Hu, Xiyang, and others,

   建筑史学刊(Journal of Architecture History) 2021

   HTML PDF

2020

1. ICDM

   COPOD: Copula-Based Outlier Detection

   LI, Zheng, Zhao, Yue, Botta, Nicola, Ionescu, Cezar, and Hu, Xiyang

   IEEE International Conference on Data Mining 2020

   Abs HTML PDF

   Outlier detection refers to the identification of rare items that are deviant from the general data distribution. Existing approaches suffer from high computational complexity, low predictive capability, and limited interpretability. As a remedy, we present a novel outlier detection algorithm called COPOD, which is inspired by copulas for modeling multivariate data distribution. COPOD first constructs an empirical copula, and then uses it to predict tail probabilities of each given data point to determine its level of "extremeness". Intuitively, we think of this as calculating an anomalous p-value. This makes COPOD both parameter-free, highly interpretable, and computationally efficient. In this work, we make three key contributions, 1) propose a novel, parameter-free outlier detection algorithm with both great performance and interpretability, 2) perform extensive experiments on 30 benchmark datasets to show that COPOD outperforms in most cases and is also one of the fastest algorithms, and 3) release an easy-to-use Python implementation for reproducibility.

2019

1. NeurIPS

   Optimal Sparse Decision Trees [NeurIPS 2019 Spotlight (Top 3%)]

   Hu, Xiyang, Rudin, Cynthia, and Seltzer, Margo

   Advances in Neural Information Processing Systems 2019

   Abs HTML PDF Code

   Decision tree algorithms have been among the most popular algorithms for interpretable (transparent) machine learning since the early 1980’s. The problem that has plagued decision tree algorithms since their inception is their lack of optimality, or lack of guarantees of closeness to optimality: decision tree algorithms are often greedy or myopic, and sometimes produce unquestionably suboptimal models. Hardness of decision tree optimization is both a theoretical and practical obstacle, and even careful mathematical programming approaches have not been able to solve these problems efficiently. This work introduces the first practical algorithm for optimal decision trees for binary variables. The algorithm is a co-design of analytical bounds that reduce the search space and modern systems techniques, including data structures and a custom bit-vector library. Our experiments highlight advantages in scalability, speed, and proof of optimality.

2015

1. AC

   重访光辉城市

   Zhan, Xuqiang, Zhang, Jiaqi, and Hu, Xiyang

   建筑创作(Archicreation) 2015

   HTML PDF