Text generation with control conditions compliance

Abstract

Controllable text generation has emerged as a significant research area, allowing the production of text with desired characteristics. In this work, we investigate the controllability of text generation, exploring the challenges of controlling various aspects of generated text, such as length, parts-of-speech (POS) structure, sentiment, and tense; in addition, we extend our analysis to the task of multi-conditional text generation, which entails the possibility of simultaneous control of several parameters of the generated text. Our research is mainly based on fine-tuned GPT-2, an autoregressive transformerbased model. Using fine-tuned GPT-2, we managed to achieve notable progress in controlling the above-mentioned text attributes; we also present the results of experiments using other approaches, such as diffusion models and ChatGPT. The models are trained on our own dataset, meticulously curated in-house; the evaluation of the generation results is carried out using a comprehensive set of control, fluency, distinctiveness, and repetition metrics. Through rigorous analysis, we assess the performance of studied models in terms of controllability. Length control, in particular, proved to be a challenging aspect, even when employing the largest available models. Nevertheless, our fine-tuned GPT-2 demonstrated promising results, showcasing its capabilities in generating text with desired characteristics. Overall, our findings highlight the possibilities of controllable text generation using fine-tuned GPT-2 and other models. Our work contributes to the ongoing exploration of techniques for improving controllability in text generation. As this field continues to evolve, further research can build upon our analysis and methodologies to enhance controllability and pave the way for more sophisticated text generation systems.