Joined Tom Hosking at the ICLR ‘24 poster session for our Human Feedback is Not Gold Standard work. Thanks for all the interest!

Presented Human Feedback is Not Gold Standard at the Amazon Themis Science Meeting.

Presented Human Feedback is Not Gold Standard at the nPlan Machine Learning Paper Club.

Gave an invited talk on the application of LLMs for Enterprise at the Oracle AI@Molitor event.

Gave a talk on NLP Applications and Large Language Models to the Capital Enterprise startup network.

Honoured to have been nominated by my students for the UCL Inspiring Teaching Delivery award 🙏

Gave an invited talk on Dynamic Advsersarial Data Collection for Large Language Models at the UCL AI Centre seminar on The Present and Future of Large Language Models in Theory and Practice.

That’s a wrap! Another year of the MSIN0221 Natural Language Processing lectures comes to an end. Exciting to see the growing interest in NLP and its application!

Presented recent work on DADC and GAAs at the King’s College London Distributed Artificial Intelligence group. Thanks for the insightful discussions!

Super excited to announce that I have joined Cohere and will be working on making large language models more useful and robust.

I’m in Seattle for NAACL 2022! I’ll be presenting Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants on Wednesday, 13th July at 10:45 PST. And don’t forget to join us at the DADC workshop on Thursday, 14th July for same amazing keynote talks, a diverse panel, presentations from our Shared Task participants and best paper winners, posters, prizes & much more!

Our work Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity has been selected as an outstanding paper at ACL 2022!

Our work Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants has been accepted as an oral presentation at NAACL 2022!

Excited to announce that I have joined DeepMind as a Research Scientist Intern.

Gave an invited talk on Dynamic Adversarial Data Collection for Question Answering at the Oracle Labs ML Seminar Series. This was a particularly fun and interactive one, thanks for the invite!

The call for participation for the Shared Task at the DADC Workshop co-located with NAACL ‘22 in Seattle is now live! We have three fantastic tracks for you to participate in. Sign up here!

Presented our work on Dynamic Adversarial Data Collection for QA at the University of Oxford.

Additional resources from our work on Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation at EMNLP 2021 are now available! We are releasing a collection of synthetically-generated adversarial QA pairs and related resources as well as the models used to generate the questions.

Just gave the last lecture of the MSIN0221 Natural Language Processing module for this year. Fantastic cohort as always and it was great to be back to in-person teaching!

AdversarialQA is currently the 3rd most downloaded QA dataset on Huggingface 🤗 Datasets right after the benchmark SQuADv1.1 and SQuADv2!

Our proposal for the First Workshop on Dynamic Adversarial Data Collection has been accepted! See you at NAACL ‘22 in Seattle!

Presented our work on Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation at EMNLP 2021. The recording is available here.

Dynabench is 1 year old! To celebrate, we’ve released Dynatask to help researchers host their own tasks.

Presented a live demonstration of Dynamic Benchmarking at the UCL AI Centre 2nd Anniversary Showcase.

Our work Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation has been accepted to the EMNLP 2021 Main Conference!

Our work Contrasting Human-and Machine-Generated Word-Level Adversarial Examples for Text Classification has been accepted to the EMNLP 2021 Main Conference!

ldbd.ly helps you make sense of ever-changing dynamic leaderboards.

Our work Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation is now available on arXiv!

Our work Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity is now available on arXiv!

The Dynabench paper introducing our unified research platform for dynamic benchmarking has been accepted to NAACL 2021!

Excited to announce that I have joined Facebook AI Research as an external research collaborator working on generation-assisted human adversarial annotation.

The AdversarialQA dataset is now available in Huggingface 🤗 Datasets! Usage is as simple as from datasets import load_dataset; adversarial_qa = load_dataset('adversarial_qa', 'adversarialQA')

The HAMLETS NeurIPS 2020 workshop kicks off today. Join us to learn more about Human And Model in the Loop Evaluation and Training Strategies.

Presented Humans-and-Machines in the Loopfor Dynamic Benchmarking and Evaluation at the Annual MURI Review Meeting.

Presented Adversarial Human Annotation for Dynamic Benchmarking and Evaluation at the UCL AI Centre session on AI in science, industry and society at TheAlgo2020.

Our work Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension will be presented at EMNLP 2020.

Dynabench, in collaboration with Stanford University, the University of North Carolina at Chapel Hill, and Facebook AI, is now live! Can you fool the QA model?

Call for papers for our NeurIPS2020 workshop HAMLETS: Human And Model in the Loop Evaluation & Training Strategies is now live!

Our work Undersensitivity in Neural Reading Comprehension has been accepted in Findings of EMNLP 2020!

Excited to announce that I have joined Facebook AI Research as a research intern working on adversarial benchmarking and robustness.

Our work Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension has been accepted to Transactions of the Association for Computational Linguistics (TACL)!

Delivered the MSIN0221: Natural Language Processing module to this year’s UCL MSc Business Analytics cohort.

Presented Adversarial Human Annotation for Reading Comprehension at the University of Cambridge NLIP Seminar Series.

Accepted onto Cohort III of the Conception X programme!

Presented Asking Harder Questions at the UCL NLP Inaugural Event followed by a poster session on ShARC.

Delivered a two-part workshop titled Overview of NLP to this year’s UCL MSc Business Analytics cohort.

Co-presented Interpretation of Natural Language Rules in Conversational Machine Reading at the South England Natural Language Processing (SENLP) meetup.

Led a workshop titled Introduction to Python and Machine Learning at the Peking University HSBC Business School (PHBS) in Oxford.

I have started a PhD at UCL under the guidance of Pontus Stenetorp and Sebastian Riedel.

Presented Interpretation of Natural Language Rules in Conversational Machine Reading at EMNLP together with Patrick Lewis and other co-authors.

The ShARC dataset from our EMNLP ‘18 paper is now live!

Bloomsbury AI has joined Facebook to strengthen its efforts in natural language processing research.

Cape (open source) is the new state-of-the-art for open-domain question answering on TriviaQA.

Our large-scale question answering system, Cape, is now available open source!

Interpretation of Natural Language Rules in Conversational Machine Reading has been accepted at EMNLP.

We’ve been accepted into the Allen & Overy Fuse accelerator programme.

Invited presentation of the work we’re doing at Bloomsbury AI at the A Common Language for Intelligence meet-up hosted by Grakn AI.

I have joined NLP-focused startup Bloomsbury AI, working on open-domain question answering.