Text generation with control conditions compliance
Date
2023
Authors
Konopatska, Oleksandra
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Controllable text generation has emerged as a significant research area, allowing the
production of text with desired characteristics. In this work, we investigate the controllability of text generation, exploring the challenges of controlling various aspects
of generated text, such as length, parts-of-speech (POS) structure, sentiment, and
tense; in addition, we extend our analysis to the task of multi-conditional text generation, which entails the possibility of simultaneous control of several parameters
of the generated text.
Our research is mainly based on fine-tuned GPT-2, an autoregressive transformerbased model. Using fine-tuned GPT-2, we managed to achieve notable progress in
controlling the above-mentioned text attributes; we also present the results of experiments using other approaches, such as diffusion models and ChatGPT. The models
are trained on our own dataset, meticulously curated in-house; the evaluation of
the generation results is carried out using a comprehensive set of control, fluency,
distinctiveness, and repetition metrics.
Through rigorous analysis, we assess the performance of studied models in terms
of controllability. Length control, in particular, proved to be a challenging aspect,
even when employing the largest available models. Nevertheless, our fine-tuned
GPT-2 demonstrated promising results, showcasing its capabilities in generating text
with desired characteristics.
Overall, our findings highlight the possibilities of controllable text generation
using fine-tuned GPT-2 and other models. Our work contributes to the ongoing exploration of techniques for improving controllability in text generation. As this field
continues to evolve, further research can build upon our analysis and methodologies
to enhance controllability and pave the way for more sophisticated text generation
systems.
Description
Keywords
Citation
Konopatska Oleksandra. Text generation with control conditions compliance. Ukrainian Catholic University, Faculty of Applied Sciences, Department of Computer Sciences. Lviv 2023, 41 p.