Xiangming Meng 孟 祥明
I am an Assistant Professor at The Zhejiang UniversityUniversity of Illinois UrbanaChampaign Institute (ZJUI), Zhejiang University. Before that, I was a Project Assistant Professor at the Kabashima Lab in the Institute for Physics of Intelligence (i π), The University of Tokyo (UTokyo) from April 2022 to March 2023. I completed my Ph.D. in 2016 from Tsinghua University, supervised by Jianhua Lu. I received a B.E. from Xidian University in 2011.
Previously I was a postdoctoral researcher in i π , UTokyo under supervision of Yoshiyuki Kabashima from April 2020 to March 2022, and
a postdoctoral researcher in the Approximate Bayesian Inference Team, RIKEN center for Advanced Intelligence Project ( RIKENAIP) under supervision of Emtiyaz Khan from July 2019 to March 2020. I worked as a senior research engineer at Huawei Technologies Co., Ltd. from July 2016 to June 2019.
I am broadly interested in the intersection of machine learning, information theory, and statistical mechanics, with a special focus on graphical models, Bayesian inference, and learning algorithms.
Email /
Google Scholar /
Github /
Researchmap /
ZJUI page


Open Positions
I am always looking for highly motivated postdoctoral researchers and research assistants with a great passion for doing research in machine learning, signal processing, wireless communication, and other related fields. Please send your detailed CV (including education background, publication list, and research interests) to the email address above if you are interested.

Research and Selected Publications
My research interests lie at the intersection of machine learning, information theory and statistical mechanics, with an exploration of common principles within different fields. Specific focuses are graphical models, Bayesian inference, and learning algorithms.
For an uptodate publication list, please see the Google Scholar page. (*correspondence)


QCMSGM+: Improved Quantized Compressed Sensing With ScoreBased Generative Models
Xiangming Meng^{*} and Yoshiyuki Kabashima
[arXiv]
[code]
In practical compressed sensing (CS), the obtained measurements typically necessitate quantization to a limited number of bits prior to transmission or storage. This nonlinear quantization process poses significant recovery challenges, particularly with extreme coarse quantization such as 1bit. Recently, an efficient algorithm called QCSSGM was proposed for quantized CS (QCS) which utilizes scorebased generative models (SGM) as an implicit prior. Due to the adeptness of SGM in capturing the intricate structures of natural signals, QCSSGM substantially outperforms previous QCS methods. However, QCSSGM is constrained to (approximately) roworthogonal sensing matrices as the computation of the likelihood score becomes intractable otherwise. To address this limitation, we introduce an advanced variant of QCSSGM, termed QCSSGM+, capable of handling general matrices effectively. The key idea is a Bayesian inference perspective on the likelihood score computation, wherein an expectation propagation algorithm is employed for its approximate computation. We conduct extensive experiments on various settings, demonstrating the substantial superiority of QCSSGM+ over QCSSGM for general sensing matrices beyond mere roworthogonality.


Diffusion Model Based Posterior Samplng for Noisy Linear Inverse Problems
Xiangming Meng^{*} and Yoshiyuki Kabashima
[arXiv]
[code]
We consider the ubiquitous linear inverse problems with additive Gaussian noise and propose an unsupervised generalpurpose sampling approach called diffusion model based posterior sampling (DMPS) to reconstruct the unknown signal from noisy linear measurements. Specifically, the prior of the unknown signal is implicitly modeled by one pretrained diffusion model (DM). In posterior sampling, to address the intractability of exact noiseperturbed likelihood score, a simple yet effective noiseperturbed pseudolikelihood score is introduced under the uninformative prior assumption. While DMPS applies to any kind of DM with proper modifications, we focus on the ablated diffusion model (ADM) as one specific example and evaluate its efficacy on a variety of linear inverse problems such as image superresolution, denoising, deblurring, colorization. Experimental results demonstrate that, for both indistribution and outofdistribution samples, DMPS achieves highly competitive or even better performances on various tasks while being 3 times faster than the leading competitor.


Quantized Compressed Sensing with ScoreBased Generative Models
Xiangming Meng^{*} and Yoshiyuki Kabashima
[ICLR2023]
[arXiv]
[code]
We consider the general problem of recovering a highdimensional signal from noisy quantized measurements. Quantization, especially coarse quantization such as 1bit sign measurements, leads to severe information loss and thus a good prior knowledge of the unknown signal is helpful for accurate recovery. Motivated by the power of scorebased generative models (SGM, also known as diffusion models) in capturing the rich structure of natural signals beyond simple sparsity, we propose an unsupervised datadriven approach called quantized compressed sensing with SGM (QCSSGM), where the prior distribution is modeled by a pretrained SGM. To perform posterior sampling, an annealed pseudolikelihood score called noise perturbed pseudolikelihood score is introduced and combined with the prior score of SGM. The proposed QCSSGM applies to an arbitrary number of quantization bits. Experiments on a variety of baseline datasets demonstrate that the proposed QCSSGM significantly outperforms existing stateoftheart algorithms by a large margin for both indistribution and outofdistribution samples. Moreover, as a posterior sampling method, QCSSGM can be easily used to obtain confidence intervals or uncertainty estimates of the reconstructed results.


On Model Selection Consistency of Lasso for HighDimensional Ising Models
Xiangming Meng^{*}, Tomoyuki Obuchi, Yoshiyuki Kabashima
The 26th International Conference on Artificial Intelligence and Statistics (AISTATS) , 2023.
[arXiv]
[AISTATS]
We theoretically analyze the model selection consistency of least absolute shrinkage and selection operator (Lasso) for highdimensional Ising models. For general treelike graphs, it is rigorously proved that Lasso without postthresholding is model selection consistent in the whole paramagnetic phase with the same order of sample complexity as that of L1regularized logistic regression (L1LogR). This result is consistent with the conjecture in Meng, Obuchi, and Kabashima 2021 using the nonrigorous replica method from statistical physics and thus complements it with a rigorous proof. Moreover, we provide a rigorous proof of the model selection consistency of Lasso with postthresholding for general treelike graphs in the paramagnetic phase without further assumptions on the dependency and incoherence conditions.


Ising Model Selection Using L1Regularized Linear Regression: A Statistical Mechanics Analysis
Xiangming Meng^{*}, Tomoyuki Obuchi, Yoshiyuki Kabashima
Advances in Neural Information Processing Systems (NeurIPS), 2021.
[arXiv]
[NeurIPS]
[video]
[slides]
We theoretically investigate the typical learning performance of L1regularized linear regression (L1LinR, i.e., Lasso) for Ising model selection using the replica method from statistical mechanics. We obtain an accurate estimate of the typical sample complexity of L1LinR, which demonstrates that L1LinR is model selection consistent with M=0(log N) samples, where N is the number of variables of the Ising model. Moreover, we provide a computationally efficient method to accurately predict the nonasymptotic behavior of L1LinR for moderate M and N, such as the precision and recall rates.


Training Binary Neural Networks using the Bayesian Learning Rule
Xiangming Meng^{}, Roman Bachmann, Mohammad Emtiyaz Khan*
The Thirtyseventh International Conference on Machine Learning (ICML), 2020.
[arXiv]
[ICML]
[video]
[slides]
[code]
Neural networks with binary weights are computationefficient and hardwarefriendly, but their training is challenging because it involves a discrete optimization problem. Surprisingly, ignoring the discrete nature of the problem and using gradientbased methods, such as StraightThrough Estimator, still works well in practice. This raises the question: are there principled approaches which justify such methods? In this paper, we propose such an approach using the Bayesian learning rule. The rule, when applied to estimate a Bernoulli distribution over the binary weights, results in an algorithm which justifies some of the algorithmic choices made by the previous approaches. The algorithm not only obtains stateoftheart performance, but also enables uncertainty estimation and continual learning to avoid catastrophic forgetting. Our work provides a principled approach for training binary neural networks which also justifies and extends existing approaches.


Advanced NOMA Receivers From a Unified Variational Inference Perspective
Xiangming Meng^{}, Lei Zhang, Chao Wang, Lei Wang, Yiqun Wu, Yan Chen*, Wenjin Wang
IEEE Journal on Selected Areas in Communications (JSAC), 2021.
[IEEE]
Nonorthogonal multiple access (NOMA) on shared resources has been identified as a promising technology in 5G to improve resource efficiency and support massive access in all kinds of transmission modes. Power domain and code domain NOMA have been extensively studied and evaluated in both literatures and 3GPP standardization, especially for the uplink where large number of users would like to send their messages to the base station. Though different in the transmitter side design, power domain NOMA and code domain NOMA share the same need of the advanced multiuser detection (MUD) design at the receiver side. Various multiuser detection algorithms have been proposed, balancing performance and complexity in different ways, which is important for the implementation of NOMA in practical networks. In this paper, we introduce a unified variational inference (VI) perspective on various universal NOMA MUD algorithms such as belief propagation (BP), expectation propagation (EP), vector EP (VEP), approximate message passing (AMP) and vector AMP (VAMP), demonstrating how they could be derived from and adapted to each other within the VI framework. Moreover, we unveil and prove that conventional elementary signal estimator (ESE) and linear minimum mean square error (LMMSE) receivers are special cases of EP and VEP, respectively, thus bridging the gap between classic linear receivers and message passing based nonlinear receivers. Such a unified perspective would not only help the design and adaptation of NOMA receivers, but also open a door for the systematic design of joint active user detection and multiuser decoding for sporadic grantfree transmission.


A unified Bayesian Inference Framework for Generalized Linear Models
Xiangming Meng*, Sheng Wu, Jiang Zhu
IEEE Signal Processing Letters (SPL), March 2018.
[arXiv]
[IEEE]
[code]
Based on expectation propagation (EP), we present a unified Bayesian inference framework for generalized linear models (GLM) which iteratively reduces the GLM problem to a sequence of standard linear model (SLM) problems.
This framework provides new perspectives on some existing GLM algorithms and also suggests novel extensions for some other SLM algorithms. Specific instances elucidated under such framework are the GLM versions of approximate message passing (AMP), vector AMP (VAMP), and sparse Bayesian learning (SBL). In particular, we provide an EP perspective on the famous generalized approximate message passing (GAMP) algorithm, which leads to a concise derivation of GAMP via EP.


An Expectation Propagation Perspective on Approximate Message Passing
Xiangming Meng*, Sheng Wu, Linling Kuang, Jianhua Lu
IEEE Signal Processing Letters (SPL), August 2015.
[IEEE]
An alternative derivation for the wellknown approximate message passing (AMP) algorithm proposed by Donoho is presented in this letter. Compared with the original derivation, which exploits central limit theorem and Taylor expansion to simplify belief propagation (BP), our derivation resorts to expectation propagation (EP) and the neglect of highorder terms in large system limit. This alternative derivation leads to a different yet provably equivalent form of message passing, which explicitly establishes the intrinsic connection between AMP and EP, thereby offering some new insights in the understanding and improvement of AMP.

Training Binary Neural Networks Using the Bayesian Learning Rule
Zhejiang University, Online, July 29, 2022
A Statistical Mechanics Analysis of Ising Model Selection
Chinese Academy of Science, Online, Jan 03, 2022
A High Bias Low Variance Introduction to Approximate Inference
Tokyo Institute of Technology,Tokyo, Japan, Oct 11, 2019
[slides]
[code]
Approximate Bayesian Inference for Generalized Linear Models
RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan, Feb 27, 2019
[slides]
A Unified Approximate Bayesian Inference Framework for Generalized Linear Models
"Physics, Inference, and Learning", Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing, China, Oct 31, 2018
[slides]

Conference and Journal Reviewing

Conference: NeurIPS (2019), ICML (2021), AAAI (2020), ICLR(2020), AISTATS(2022), ALT(2020), NeurIPS workshop on Machine Learning and the Physical Sciences (2021).
Journal: Statistics and Computing, IEEE Journal on Selected Areas in Communications (JSAC), IEEE Signal Processing Letters (SPL), IEEE Communication Letters (CL)

Misc.
I love reading, music, and table tennis.

