Hidden state refinement for optical flow forecasting

Date

2023

Authors

Babenko, Anton

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

In recent years the topic of optical flow has become well-spread due to computation power support and optical flow estimation applications used on mobile phones and edge devices: video editors, frame stabilizations, and autonomous driving feature providers. This work analyzes multiple approaches to optical flow estimation and finds the main problems of the optical flow methods: slow convergence and long execution of the prediction algorithm. We propose to solve the slow convergence and long execution time with hidden state refinement to provide the initialization for optical flow estimation based on several previous frames and their hidden state transformations, which imitates the pixel movement at the hidden state level. The proposed method uses CNN, LSTM, and Transformer blocks which help to achieve the optical flow estimation and hidden state refinement to speed up the system. We used Sintel, KITTY-15, FlyingChairs, FlyingThings, HD1K, DAVIS, and YouTubeVOS datasets for our experiment.

Description

Keywords

Citation

Babenko Anton. Hidden state refinement for optical flow forecasting. Master Thesis. Ukrainian Catholic University, Faculty of Applied Sciences, Department of Computer Sciences. Lviv 2023, 43 p.

Collections

Endorsement

Review

Supplemented By

Referenced By