The processing of a high-definition video stream in real-time is a challenging task for embedded systems. However, modern FPGA devices have both a high operating frequency and sufficient logic resources to be successfully used in these tasks. In this article, an advanced system that is able to generate and maintain a complex background model for a scene as well as segment the foreground for an HD colour video stream (1,920 × 1,080 @ 60 fps) in real-time is presented. The possible application ranges from video surveillance to machine vision systems. That is, in all cases, when information is needed about which objects are new or moving in the scene. Excellent results are obtained by using the CIE Lab colour space, advanced background representation as well as integrating information about lightness, colour and texture in the segmentation step. Finally, the complete system is implemented in a single high-end FPGA device.
@article{badap-agh:80341,title={Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device},author={Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},year={2014},journal={Journal of Real-Time Image Processing},series={Journal of Real-Time Image Processing},volume={9},number={1},pages={61},doi={10.1007/s11554-012-0290-5},issn={1861-8200},collection={Journal of Real-Time Image Processing},projects={Preludium-Kryjak},}
In the paper a system for acquisition and processing of a 3D video stream is presented. It can work with 3D HDMI cameras available on the market. In Section 2 the basic concepts of stereovision systems are described [1]. In Section 3 three distance metrics, SAD [4], ZSAD and Census [5], used for correspondence matching are discussed. Evaluation of the matching process on the Middlebury dataset [2] is also presented. The best results were obtained for the SAD and ZSAD methods and greyscale images. In Table 1 there are shown three best configurations. Figure 1 illustrates the obtained disparity maps. A description of the hardware implementation is given in Section 4. The block diagram of the system is presented in Figure 2. The proposed solution is able to process images transmitted in side by side mode, to compute two disparity maps (left to right and right to left, method from [4]), to use SAD or ZSAD cost function, to check maps consistency and execute median filtering for final image processing. The described module is highly parameterizable: different cost functions, window sizes and disparity range can be used, image size and median filtering size can be adjusted. FPGA resource utilization is presented in Table 2. A picture of the working system is shown in Figure 3 (1280 x 720 @60 fps, real-time video-stream processing). The proposed module can be used for video surveillance, pedestrian collision avoidance systems or in autonomous vehicles.
@article{badap-agh:76884,title={Implementacja obliczania map dysparycji w czasie rzeczywistym dla strumienia wizyjnego 3D zrealizowana w układzie FPGA},author={Kryjak, Tomasz and Komorkiewicz, Mateusz},year={2013},journal={Pomiary, Automatyka, Kontrola},volume={59},number={8},pages={748–750},issn={0032-4140},projects={Preludium-Kryjak},}
In this paper a human action recognition algorithm, which uses background generation with shadow elimination, silhouette description based on simple geometrical features and a finite state machine for recognizing particular actions is described. The performed tests indicate that this approach obtains a 81 % correct recognition rate allowing real-time image processing of a 360 X 288 video stream.
@article{badap-agh:80270,title={Human action recognition using simple geometric features and a finite state machine},author={Dudzinski, Damian and Kryjak, Tomasz and Mikrut, Zbigniew},year={2013},journal={Image Processing \& Communications},volume={18},number={2–3},pages={49–60},issn={1425-140X},projects={Preludium-Kryjak},}
The article presents a hardware-software system for people counting based on analysis of 3D information. It consists of a 3D video source (3D camera), an FPGA evaluation board and a PC computer. The computationaly demanding disparity map calculation and median filtering are realized in hardware. The software part is responsible for people segmentation, tracking and counting. The data between system nodes is transferred via Ethernet. In the paper the system architecture, proposed algorithms and obtained results are presented.
@article{badap-agh:80272,title={Hardware-software vision system for moving people counting based on 3D information},author={Kryjak, Tomasz and Komorkiewicz, Mateusz},year={2013},journal={Image Processing \& Communications},volume={18},number={2–3},pages={71–81},issn={1425-140X},projects={Preludium-Kryjak},}
@article{badap-agh:69876,title={Rekonfigurowalny system wizyjnego nadzoru do detekcji naruszenia obszarów chronionych},author={Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},year={2012},journal={Pomiary, Automatyka, Kontrola},volume={58},number={7},pages={584–586},issn={0032-4140},projects={Preludium-Kryjak},}
@inproceedings{badap-agh:69895,title={FPGA implementation of real-time head-shoulder detection using local binary patterns, SVM and foreground object detection},author={Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},year={2012},booktitle={dasip [Dokument elektroniczny] : proceedings of the 2012 conference on Design \& Architectures for Signal \& Image Processing : Karlsruhe, Germany, October 23–25, 2012},pages={232–239},projects={Preludium-Kryjak},}
@inproceedings{badap-agh:69897,title={FPGA implementation of camera tamper detection in real-time},author={Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},year={2012},booktitle={dasip [Dokument elektroniczny] : proceedings of the 2012 conference on Design \& Architectures for Signal \& Image Processing : Karlsruhe, Germany, October 23–25, 2012},pages={224–231},projects={Preludium-Kryjak},}
In the demo four video surveillance algorithms implemented on an FPGA platform, running in real-time are presented: detecting intrusions into protected areas, camera tamper detection, head and shoulders detection using LBP and SVM and head, people and bicycles detection using HOG and SVM.
@inproceedings{badap-agh:69900,title={Is FPGA a suitable platform for advanced video surveillance systems?},author={Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},year={2012},booktitle={dasip [Dokument elektroniczny] : proceedings of the 2012 conference on Design \& Architectures for Signal \& Image Processing : Karlsruhe, Germany, October 23–25, 2012},pages={369–370},projects={Preludium-Kryjak},}
