NLP Boosts Geopolitical Forecasting
How the KaDSci Team leveraged NLP to jump into second place in IARPA competition
Basis Technology congratulates the authors of the peer-reviewed article “What do forecasting rationales reveal about thinking patterns of top geopolitical forecasters?”, which delves into the technical details of how the KaDSci team leapt from 27th place to 2nd place in three months during IARPA’s 11-month-long Geopolitical Forecasting Challenge 2. Team members Christopher W. Karvetski, Carolyn Meinel, Daniel T. Maxwell, YunziLu, Barbara A. Mellers, and Philip E. Tetlock used natural language processing (NLP) technology in our Rosette® text analytics program to become superforecasters.
The team created this huge improvement in large part by adding sophisticated NLP tools to triage data. By using semantic search and entity extraction to surface useful, accurate data from massive, noisy datasets — which sometimes contained misleading information — these human forecasters predicted answers to 305 geopolitical questions covering a staggering range of topics.
Some of the examples of how they used NLP discuss how cross-lingual semantic search delivered the relevant paragraphs of documents, from which they could forecast answers. These were data that could not normally be discovered from keyword searches or with less than a roomful of subject matter experts.
You can watch the talk that team member Meinel gave about this astonishing feat at the Human Language Technology Conference in 2020. [Click here to watch the recorded presentation | Download the slides]