Бібліографічний опис:
Romanus, Teodor. Generalizing texture transformers for super-resolution and inpainting / Teodor Romanus; Supervisor: Roman Riazantsev; Ukrainian Catholic University, Faculty of Applied Sciences, Department of Computer Sciences. – Lviv 2022. – 46 p.
Короткий опис (реферат):
The new multi-camera smartphones and recent advancements in generalized
Machine Learning models make it possible to bring new types of photo editing neural
networks to the market. This thesis covers methods of image enhancement with
texture transfer. The known high-resolution regions (reference) can be utilized to
restore degraded areas of an image. The task of restoring partially degraded images
can be defined “partial super-resolution.” The task of restoring missing parts of
images is called inpainting. We propose to use the novel Texture Transformer Network
for Image Super-Resolution (TTSR) to solve the partial super-resolution and
inpainting tasks.
The fully convolutional networks are unable to copy image patches. This inability
forces the model to store textures using the train weights. The usage of the attention
mechanism allows taking advantage of joint feature learning in low-resolution
and high-resolution parts of images simultaneously, in which deep feature correspondences
can be discovered by attention. This approach exhibits an accurate
transfer of texture features.
The experiments confirm that the TTSR network can be used to solve the partial
super-resolution and inpainting tasks simultaneously. Modifications of the network
(different embedding sizes, soft-attention, trainable projections) study the architecture
capacity to solve the specified tasks. The evaluation of results includes comparing
the TTSR network with an inpainting network for the inpainting task.
