My research revolves around improving data-to-text generation systems.

The projects I have worked on include:

• domain-independent data-to-text generation [1], [2],
• evaluating semantic accuracy of generated text [3], [4],
• investigating the role of data labels [5].
• software toolkit for data-to-text generation [6].

I focus on developing efficient representations of structured data, so that the data can be used as an input to pretrained language models for generating automated reports.

During my internship at Mila, I was working on applying LLMs for autonomous web navigation.

In the future, I would also like to delve deeper into model interpretability: how is the information inside the language models represented, how do language models reason, and how this all relates to human cognition.

Selected publications

TabGenie: A Toolkit for Table-to-Text Generation

Zdeněk Kasner, Ekaterina Garanina, Ondřej Plátek, Ondřej Dušek

ACL 2023 – Demo Track

We began developing this tool to play with the generative language models in real time but it soon evolved into a swiss knife for table-to-text generation. TabGenie provides interactive data visualization, unified data representation & unified programming interface for more than 15 data-to-text datasets. You can use the web interface as a dataset viewer and a model playground, the programming interface then allows to quickly prototype new models.

arXiv Poster Code Demo

Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models

Zdeněk Kasner, Ioannis Konstas, Ondřej Dušek

EACL 2023

What is a better way to learn the data semantics – memorizing an arbitrary mapping or taking the human-readable data labels into account? On the task of describing a triple (entity_1, relation, entity_2), we show that the models are able to describe previously unseen relations as long as the relation label is meaningful and unambiguous. To put it another way: if you want to train robust data-to-text systems, don't use abbreviations!

ACL Anthology arXiv Poster Code

Neural Pipeline for Zero-Shot Data-to-Text Generation

Zdeněk Kasner, Ondřej Dušek

ACL 2022

To combine the power of pretrained language models with controllability of pipeline approaches, we formulate data-to-text generation as a sequence of trainable text-to-text operations: ordering, aggregation, and paragraph compression. As a welcome side-effect, we get rid of semantically incorrect outputs arising from noisy human-written references. Is NL-only approach to data-to-text generation the way to go?

ACL Anthology arXiv Poster Code

Text-in-Context: Token-Level Error Detection for Table-to-Text Generation

Best submission at Shared Task in Evaluating Accuracy

Zdeněk Kasner, Simon Mille, Ondřej Dušek

INLG 2021

How to automatically detect which parts of the generated text do not correspond to the data? In our submission for the Shared Task in Evaluating Accuracy 2021, we devise a 3-step approach combining a rule-based system with pretrained language models. Our approach was the best out of four submitted metrics!

ACL Anthology Poster Code

Train Hard, Finetune Easy: Multilingual Denoising for RDF-to-Text Generation

2nd place in Russian RDF-to-text generation

Zdeněk Kasner, Ondřej Dušek

INLG 2020

Data == noisy text. That's definitely an overgeneralization, oversimplification... but um, it works! We succesfully generate text from DBPedia data in English and Russian just by finetuning mBART – a pretrained multilingual denoising autoencoder. This is our submission for the WebNLG Challenge 2020, presented at the 3rd Workshop on Natural Language Generation from the Semantic Web.

ACL Anthology

Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference

INLG 2020 Best Short Paper Award

Ondřej Dušek, Zdeněk Kasner

INLG 2020

Does a text contain all the information from the data? Hard to check manually, even harder to code. But what if we reformulate the question a little bit: can we infer all the data from the text and nothing else? That sounds like natural language inference. And guess what - somebody already pretrained a neural model for that!

ACL Anthology arXiv Code

Data-to-Text Generation with Iterative Text Editing

Zdeněk Kasner, Ondřej Dušek

INLG 2020

Imagine your task is to generate text from data. Which sounds easier: generating text from scratch or joining existing sentences? We propose an approach in which we iteratively join the sentences with a text-editing neural model. Since the model has a limited vocabulary, it has also a limited possibilities of introducing incorrect facts. Moreover, it also turns out that sentence fusion is a quite general task which works on multiple domains.

ACL Anthology arXiv Poster Code

Improving Fluency of Non-Autoregressive Machine Translation

Zdeněk Kasner, Jindřich Libovický, Jindřich Helcl

arXiv

In the follow-up of my master thesis, we improve translation quality of a CTC-based machine translation model. The model is non-autoregressive, i.e. faster but lacking behind autoregressive models in translation quality. To improve the translation quality, we re-score the hypotheses during the beam search decoding with an n-gram language model and several other features.

arXiv Code