Bobby He

Hi! I’m Bobby :blush: I’m a final year PhD student at the Department of Statistics in Oxford, supervised by Yee Whye Teh, Arnaud Doucet and George Deligiannidis.

I am broadly interested in the basics of deep learning. Most of my research attempts to improve our theoretical understanding of how Neural Networks (NNs) work, and use these insights to devise principled methods that improve NNs in practice.

Before starting my PhD I completed an integrated master’s in Mathematics at the University of Cambridge, where I received the Sprague (jointly) and Wishart prizes in Part III.

I enjoyed interning at Samsung Research UK hosted by Mete Ozay from July 2021 to April 2022, and with James Martens at DeepMind in the summer of 2022.

In my spare time I like to play and perform violin (example), and play and follow various sports such as football (Liverpool in particular) or tennis.

Publications

Deep Transformers without Shortcuts: Modifying Self-Attention for Faithful Signal Propagation
Bobby He, James Martens, Guodong Zhang, Alex Botev, Andy Brock, Sam Smith, Yee Whye Teh
ICLR 2023 (to appear)
Paper

UncertaINR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomography  
Francisca Vasconcelos*, Bobby He*, Nalini Singh, Yee Whye Teh
*Equal contribution.
TMLR 2023 (to appear)
Paper

Exploring the Gap between Collapsed & Whitened Features in Self-Supervised Learning
Bobby He, Mete Ozay
ICML 2022
Paper

Feature Kernel Distillation
Bobby He, Mete Ozay
ICLR 2022
Paper   Blog

Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning
Soufiane Hayou, Bobby He, Gintare Karolina Dziugaite
Preprint 2021
Paper

Stable ResNet
Soufiane Hayou*, Eugenio Clerico*, Bobby He*, George Deligiannidis, Arnaud Doucet, Judith Rousseau
*Equal contribution.
AISTATS 2021 (Oral)
Paper

Effectiveness and resource requirements of test, trace and isolate strategies for COVID in the UK
Bobby He*, Sheheryar Zaidi*, Bryn Elesedy*, Michael Hutchinson*, Andrei Paleyes*, Guy Harling, Anne M Johnson, Yee Whye Teh
*Equal contribution.
Royal Society Open Science 2021
DOI

Efficient Bayesian Inference of Instantaneous Reproduction Numbers at Fine Spatial Scales, with an Application to Mapping and Nowcasting the Covid-19 Epidemic in British Local Authorities
Yee Whye Teh, Avishkar Bhoopchand, Peter Diggle, Bryn Elesedy, Bobby He, Michael Hutchinson, Ulrich Paquet, Jonathan Read, Nenad Tomasev, Sheheryar Zaidi
Royal Statistical Society’s Covid-19 Task Force: Special Topic Meeting on R/local R/transmission 2021
Paper   Website

Bayesian Deep Ensembles via the Neural Tangent Kernel
Bobby He, Balaji Lakshminarayanan, Yee Whye Teh
NeurIPS 2020
Paper