Hi, I am Sebastian, PhD student and research & teaching assistant at TU Wien supervised by Prof. Allan
I work in the field of Information Retrieval on efficient & interpretable neural re-ranking and effective
dense retrieval models.
My goal is to make neural techniques in IR accessible to a large audience. Therefore, I study and try to optimize the cost-effectiveness tradeoff from multiple angles, so that everyone can deploy those techniques. In addition, I created an award-winning master-level course to teach neural advances in IR – all of which is open-source 🎉
Find me on Twitter, GitHub, HuggingFace,
or email me: firstname.lastname@example.org
In my PhD, I work on the topic of "Optimizing for the Cost-Effectiveness Tradeoff in Neural Retrieval and Re-Ranking" from multiple angles, split into two main parts:
Dense retrieval (using a nearest neighbor vector search) gained quick popularity as a promising future of search – I am focused on utilizing knowledge distillation from stronger, but slower teacher models to improve the dense retrieval quality. Neural re-ranking models always add time to the query latency, therefore I focus on improving the efficiency for short and long text neural re-ranking models.
Neural Retrieval & Knowledge Distillation
(Full) Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling
(Full) Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation
(Short) Enriching Word Embeddings for Patent Retrieval with Global Context
⭐ Won best systems short paper award
Efficient & Interpretable Neural Re-Ranking
(Full) Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
(Full) Mitigating the Position Bias of Transformer Models in Passage Re-Ranking
2020TREC Evaluating Transformer-Kernel Models at TREC Deep Learning 2020
(Short) Local Self-Attention over Long Text for Efficient Document Retrieval
(Short) Learning to Re-Rank with Contextualized Stopwords
(Resource) Fine-Grained Relevance Annotations for Multi-Task Document Ranking and Question Answering
(Full) Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking
(Demo) Neural-IR-Explorer: A Content-Focused Tool to Explore Neural Re-Ranking Results
2019TREC TU Wien @ TREC Deep Learning ’19 – Simple Contextualization for Re-ranking
(Short) On the Effect of Low-Frequency Terms on Neural-IR Models
(Workshop) Let’s measure runtime!
Dense retrieval (using a nearest neighbor vector search) gained quick popularity as a promising future of search – I am focused on utilizing knowledge distillation from stronger, but slower teacher models to improve the dense retrieval quality.
Neural re-ranking models always add time to the query latency, therefore I focus on improving the efficiency for short and long text neural re-ranking models.
All my materials are available open-source on GitHub. For a detailed description of our workflow for remote teaching see:
(Full) A Time-Optimized Content Creation Workflow for Remote Teaching
Advanced Information Retrieval (Summer 2021, 2020, 2019)
Full responsibility and main lecturer for the master-level course with > 100 students on neural methods for IR. Including designing, conducting, and grading of lectures, exercises, and exams.
A playlist of all lecture recordings from 2021 is available on Youtube.
(10 lectures total) Exercise: Implement neural re-ranking models in PyTorch
🏆 Won Best Distance Learning Lecture & Best Teacher Award 2021 @ TU Wien
Introduction to Information Retrieval (Winter 2019, 2018)
2018 - presentPhD - Computer Science - TU Wien 2016 - 2018Master's - Software Engineering - TU Wien 2012 - 2016Bachelor's - Computer Science and Economics - TU Wien