I curate a weekly email of 10 or so highly interesting Machine Learning papers, and I send it to my lab. I thought it'd be interesting to put the repository here.


You can sign up for the email here.



#33: What happened in 2017 edition

Happy New Year! Humankind has survived (barely?) another year without coming crashing down due to internet hyper-dependence, the robot uprising, nuclear war, or AGI-based extermination. There, start your 2018 off on a cheery note!

Interesting News and Links from the past 7 days (papers below)

The top 10 papers from the past week

n Title Author(s)
1 SLAC: A Sparsely Labeled Dataset for Action Classification and Localization Zhao et al.
2 The ParallelEye Dataset: Constructing Large-Scale Artificial Scenes for Traffic Vision Research Li et al.
3 An Approximate Bayesian Long Short-Term Memory Algorithm for Outlier Detection Chen et al.
4 Dropout Feature Ranking for Deep Learning Models Chang et al.
5 Tensor Regression Networks with various Low-Rank Tensor Approximations Cao et al.
6 The Robust Manifold Defense: Adversarial Training using Generative Models Ilyas et al.
7 Visualizing the Loss Landscape of Neural Nets Li et al.
8 Adversarial Patch Brown et al.
9 Deep learning for universal linear embeddings of nonlinear dynamics Lusch et al.
10 Inverse Classification for Comparison-based Interpretability in Machine Learning Laugel et al.
11 Predicting Rich Drug-Drug Interactions via Biomedical Knowledge Graphs and Text Jointly Embedding Wang et al.
12 A Real-Time Autonomous Highway Accident Detection Model Based on Big Data Processing and Computational Intelligence Ozbayoglu et al.

#32: Speaking like humans, Evolving like humans, and Programming humans Edition

Hello WTMLP list! Greetings to the Oxford-based researchers, and the few joining us from afar. Given the international makeup of the labs here, I suspect most of everyone is back home right now scattered across Europe, increasingly so across Asia, and then the rest of the world.

Interesting News and Links from the past 7 days (papers below)

The top 10 papers from the past week

Particularly good:

n Title Author(s)
1 Model selection for Gaussian processes utilizing sensitivity of posterior predictive distribution Paananen et al.
2 Incremental Adversarial Domain Adaptation Wulfmeier et al.
3 A Flexible Approach to Automated RNN Architecture Generation Schrimpf et al.
4 Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents Conti et al.
5 Multiagent-based Participatory Urban Simulation through Inverse Reinforcement Learning Suzuki
6 Variational Attention for Sequence-to-Sequence Models Bahuleyan et al.
7 Generating and designing DNA with deep generative models Killoran et al.
8 Inverse Classification for Comparison-based Interpretability in Machine Learning Laugel et al.
9 Transformation Models in High-Dimensions Klaassen et al.
10 Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology Holzinger et al.
11 A brain signature highly predictive of future progression to Alzheimer’s dementia Dansereau et al.

#31: All I Want for Christmas is a GPU edition

Hi Oxford AI/ML Community! With Christmas around the corner, surely you don't have anything better to do than spend your break reading more papers?

Interesting News and Links from the past 7 days (papers below)

The top 10 papers from the past week

Papers of note: [1] provides a short tutorial on some of the basic mathematics of why deep learning works as well as it does. While we're still short for some of the more important answers, it would be useful to at least understand these results as a baseline.

While [5] was released in June, it develops a new technique that outperforms sequence transduction models – using no recurrence and convolution mechanisms, a simpler architecture, and significantly less training time. Highly recommended.

[6] shows you can predict a deep learning architecture's performance before training it, which could significantly change training practices and thus experimentation time.

[9] is by Oxford researchers within MLRG.

n Title Author(s)
1 AI2-THOR: An Interactive 3D Environment for Visual AI Kolve et al.
2 Mathematics of Deep Learning Vidal et al.
3 Deep Learning Scaling is Predictable, Empirically Hestness et al.
4 Nonparametric Neural Networks Philipp & Carbonell
5 Attention Is All You Need Vaswani et al.
6 Peephole: Predicting Network Performance Before Training Deng et al.
7 Privacy-Preserving Adversarial Networks Tripathy et al.
8 Sensitivity Analysis for Predictive Uncertainty in Bayesian Neural Networks Depeweg et al.
9 Safe Policy Search with Gaussian Process Models Polymenakos et al.
10 Practical Bayesian optimization in the presence of outliers Martinez-Cantin et al.

#30: Post-NIPS AI Winter 2017 Edition

Hi all, If you went, I hope you had a great time at NIPS in LA. I hear we have some Oxford lab members who have decided to take a productive research exchange to a Mexican beach on the way out. Looking forward to hearing what you research?

Interesting News and Links from the past 7 days (papers below)

The top 10 papers from the past week

Papers of note: [1], co-authored by Jeff Dean, supposedly has big implications for data structures. [2] is part of the push towards automatic Neural Network design. [3] suggested that human+computer hybrids aren't top dog any more. [11] is 5-6 times larger than standard self-driving car datasets. [9] and [10] are by Oxford researchers, and [4] by former Oxford researchers.

n Paper Title Author(s)
1 The Case for Learned Index Structures Kraska et al.
2 Progressive Neural Architecture Search Liu et al.
3 Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm Silver et al.
4 Learning a Generative Model for Validity in Complex Discrete Structures Janz et al.
5 Generative Adversarial Perturbations Poursaeed et al.
6 Sensitivity Analysis for Predictive Uncertainty in Bayesian Neural Networks Depeweg et al.
7 Discriminative k-shot learning using probabilistic models Bauer et al.
8 The reparameterization trick for acquisition functions Wilson et al.
9 Inferring agent objectives at different scales of a complex adaptive system Hendricks et al.
10 Cost-sensitive detection with variational autoencoders for environmental acoustic sensing Li et al.
11 TorontoCity: Seeing the World with a Million Eyes Wang et al.

#29: You Won't Read This Because You're At NIPS Edition

NIPS is this week! As such, I imagine your labs are looking almost as decimated as they might look if Uber/Facebook/Google/etc. had bought everyone out. To everyone at NIPS: stay safe, have fun, and come back with great ideas.

Interesting News and Links from the past 7 days

The Papers – 10 Interesting Papers from this Week

Some papers of note: [5] made a splash on Twitter, effectively claiming that GAN improvements have been due to hyperparameter hacking and not robust, generalizable improvements; [7] is DeepMind's 1000x speedup of WaveNet; I haven't read [11] in full but it looks hilarious (and according to the paper, it is – it was also accepted at a NIPS workshop).

n Title Author(s)
1 Stacked Kernel Network Zhang et al.
2 Snorkel: Rapid Training Data Creation with Weak Supervision Ratner et al.
3 Relation Networks for Object Detection Hu et al.
4 Population Based Training of Neural Networks Jaderberg et al.
5 Are GANs Created Equal? A Large-Scale Study Lucic et al.
6 Distilling a Neural Network Into a Soft Decision Tree Frosst & Hinton
7 Parallel WaveNet: Fast High-Fidelity Speech Synthesis Oord et al.
8 Demystifying AlphaGo Zero as AlphaGo GAN Dong et al.
9 Deep Reinforcement Learning for De-Novo Drug Design Popova et al.
10 Divide-and-Conquer Reinforcement Learning Ghosh et al.
11 Improvised Comedy as a Turing Test Mathewson & Mirowski


#28: Now You Can Publish in Nature So Mom Can Be Proud edition

Hi Oxford AI/ML people,

First – from now on I'm putting editions of the WTMLP on my website here, after requests to be able to share it easily. Others can sign up on a link on that page. Hello to those who are joining us from outside of PARG! Second – send me any papers you release and I'll highlight Oxford-specific ones.

Interesting News and Links


n Title Author(s)
1 Diversity-Promoting Bayesian Learning of Latent Variable Models Xie et al.
2 Improved Bayesian Compression Federici et al.
3 Hindsight policy gradients Rauber et al.
4 Action Branching Architectures for Deep Reinforcement Learning Tavakoli et al.
5 Non-local Neural Networks Wang et al.
6 Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions Wu et al.
7 BlockDrop: Dynamic Inference Paths in Residual Networks Wu et al.
8 Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge Bezenac et al.
9 Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN Gao et al.
10 Building Machines that Learn and Think for Themselves: Commentary on Lake et al., Behavioral and Brain Sciences, 2017 Botvinick et al.
11 Beyond Sparsity: Tree Regularization of Deep Models for Interpretability Wu et al.
12 Understanding Deep Learning Generalization by Maximum Entropy Zheng et al.
13 Nonparametric independence testing via mutual information Berrett & Samworth
14 Dropping Activation Outputs with Localized First-layer Deep Network for Enhancing User Privacy and Data Security Dong et al.
15 Learning to select computations Lieder et al.



#27: AI is Going to Kill Everyone Edition

Apologies for the grave title. Three very intriguing things have happened in the past few weeks around AI safety, and I recommend you give them a look and think for yourself where you stand. In other news: another TensorFlow release, a new Distill publication, a cookbook with potential, and the Chinese government's AI strategy.


Interesting News and Links


n Title Author(s)
1 GPflowOpt: A Bayesian Optimization Library using TensorFlow Knudde et al.
2 A Convex Parametrization of a New Class of Universal Kernel Functions for use in Kernel Learning Colbert & Peet
3 Kernel Conditional Exponential Family Arbel & Gretton
4 Prediction Under Uncertainty with Error-Encoding Networks Henaff et al.
5 Learning Explanatory Rules from Noisy Data Evans & Grefenstette
6 Data Augmentation Generative Adversarial Networks Antoniou et al.
7 Squeeze-and-Excitation Networks Hu et al.
8 Bayesian Paragraph Vectors Ji et al.
9 ACtuAL: Actor-Critic Under Adversarial Learning Goyal et al.
10 Automatic Conflict Detection in Police Body-Worn Video Letcher et al.