Text generation with control conditions compliance

Date

2023

Authors

Konopatska, Oleksandra

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Controllable text generation has emerged as a significant research area, allowing the production of text with desired characteristics. In this work, we investigate the controllability of text generation, exploring the challenges of controlling various aspects of generated text, such as length, parts-of-speech (POS) structure, sentiment, and tense; in addition, we extend our analysis to the task of multi-conditional text generation, which entails the possibility of simultaneous control of several parameters of the generated text. Our research is mainly based on fine-tuned GPT-2, an autoregressive transformerbased model. Using fine-tuned GPT-2, we managed to achieve notable progress in controlling the above-mentioned text attributes; we also present the results of experiments using other approaches, such as diffusion models and ChatGPT. The models are trained on our own dataset, meticulously curated in-house; the evaluation of the generation results is carried out using a comprehensive set of control, fluency, distinctiveness, and repetition metrics. Through rigorous analysis, we assess the performance of studied models in terms of controllability. Length control, in particular, proved to be a challenging aspect, even when employing the largest available models. Nevertheless, our fine-tuned GPT-2 demonstrated promising results, showcasing its capabilities in generating text with desired characteristics. Overall, our findings highlight the possibilities of controllable text generation using fine-tuned GPT-2 and other models. Our work contributes to the ongoing exploration of techniques for improving controllability in text generation. As this field continues to evolve, further research can build upon our analysis and methodologies to enhance controllability and pave the way for more sophisticated text generation systems.

Description

Keywords

Citation

Konopatska Oleksandra. Text generation with control conditions compliance. Ukrainian Catholic University, Faculty of Applied Sciences, Department of Computer Sciences. Lviv 2023, 41 p.

Collections

Endorsement

Review

Supplemented By

Referenced By