Marek Gorgoń

Prof Marek Gorgoń
Professor
mago@agh.edu.pl

I specialise in computer vision architectures, with particular emphasis on real-time systems, FPGA reconfigurable devices, embedded and intelligent systems. More broadly, my interests include many issues in the areas of automation and robotics, electronics and applied computer science. I am the author of 120 scientific publications.

Academic service 3

General Chair
2026

DSD 2026 – Euromicro Conference on Digital System Design, Kraków
Steering Committee member

DASIP – Workshop on Design and Architectures for Signal and Image Processing
TPC member
2026

FPL – International Conference on Field-Programmable Logic and Applications

Publications

2025

KU KDM Preludium

High-performance computing for event cameras: DIF filtering and graph convolutional networks for object classification

Marcin Kowalczyk, Kamil Jeziorek, Tomasz Kryjak, and Marek Gorgon

In KU KDM 2025 : seventeenth ACC Cyfronet AGH HPC users’ conference : Zakopane, 2-4 April 2025 : proceedings, 2025

@inproceedings{badap-agh:159338,
  title = {High-performance computing for event cameras: DIF filtering and graph convolutional networks for object classification},
  author = {Kowalczyk, Marcin and Jeziorek, Kamil and Kryjak, Tomasz and Gorgon, Marek},
  year = {2025},
  projects = {Preludium-Kowalczyk},
  series = {KU KDM~2025 [Dokument elektroniczny] : seventeenth ACC Cyfronet AGH HPC users' conference : Zakopane, 2-4 April~2025 : proceedings},
  pages = {49-50},
  collection = {KU KDM~2025 [Dokument elektroniczny] : seventeenth ACC Cyfronet AGH HPC users' conference : Zakopane, 2-4 April~2025 : proceedings},
}

DASIP
LiFT: lightweight, FPGA-tailored 3D object detection based on LiDAR data

Konrad Lis, Tomasz Kryjak, and Marek Gorgon

In Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP 2025 : Barcelona, Spain, January 20–22, 2025 : proceedings, 2025

Abstract DOI arXiv Bib

This paper presents LiFT, a lightweight, fully quantized 3D object detection algorithm for LiDAR data, optimized for real-time inference on FPGA platforms. Through an in-depth analysis of FPGA-specific limitations, we identify a set of FPGA-induced constraints that shape the algorithm’s design. These include a computational complexity limit of 30 GMACs (billion multiply-accumulate operations), INT8 quantization for weights and activations, 2D cell-based processing instead of 3D voxels, and minimal use of skip connections. To meet these constraints while maximizing performance, LiFT combines novel mechanisms with state-of-the-art techniques such as reparameterizable convolutions and fully sparse architecture. Key innovations include the Dual-bound Pillar Feature Net, which boosts performance without increasing complexity, and an efficient scheme for INT8 quantization of input features. With a computational cost of just 20.73 GMACs, LiFT stands out as one of the few algorithms targeting minimal-complexity 3D object detection. Among comparable methods, LiFT ranks first, achieving an mAP of 51.84% and an NDS of 61.01% on the challenging NuScenes validation dataset. The code will be available at https://github.com/vision-agh/lift.
@inproceedings{badap-agh:160941, title = {LiFT: lightweight, FPGA-tailored 3D object detection based on LiDAR data}, author = {Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek}, year = {2025}, series = {Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP~2025 : Barcelona, Spain, January~20–22,~2025 : proceedings}, pages = {28–40}, doi = {10.1007/978-3-031-87897-8_3}, collection = {Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP~2025 : Barcelona, Spain, January~20–22,~2025 : proceedings}, }
FPL Preludium
Preludium
Live demonstration: Continuous processing of event-data with Graph Convolutional Neural Networks implemented for SoC FPGA

Piotr Wzorek, Kamil Jeziorek, Marcin Kowalczyk, Krzysztof Blachut, Tomasz Kryjak, and 1 more author

In FPL 2025 : 35th International Conference on Field-Programmable Logic and Applications : 1–5 September 2025 Leiden, Netherlands : proceedings, 2025

Abstract DOI Bib

For perception systems in mobile robotics - particularly in demanding, highly dynamic environments - event cameras (DVS - Dynamic Vision Sensors) are being increasingly utilised as an alternative to conventional vision sensors. The real-time processing of the registered spatio-temporal sparse point cloud must satisfy stringent requirements in terms of latency, throughput, and energy efficiency. In this demonstration, we present our method for implementing Graph Convolutional Neural Networks on a heterogeneous SoC FPGA platform, aimed at meeting these constraints. We address the challenges associated with integrating event-based sensors with reconfigurable hardware, as well as the intricate relationship between latency, performance, and hardware resource utilisation.
@inproceedings{badap-agh:166922, title = {Live demonstration: Continuous processing of event-data with Graph Convolutional Neural Networks implemented for SoC FPGA}, author = {Wzorek, Piotr and Jeziorek, Kamil and Kowalczyk, Marcin and Blachut, Krzysztof and Kryjak, Tomasz and Gorgon, Marek}, year = {2025}, series = {FPL~2025 [Dokument elektroniczny] : 35th International Conference on Field-Programmable Logic and Applications : 1–5 September~2025 Leiden, Netherlands : proceedings}, pages = {359}, doi = {10.1109/FPL68686.2025.00065}, collection = {FPL~2025 [Dokument elektroniczny] : 35th International Conference on Field-Programmable Logic and Applications : 1–5 September~2025 Leiden, Netherlands : proceedings}, projects = {Preludium-Wzorek, Preludium-Kowalczyk}, }

2024

DSD
PowerYOLO: mixed precision model for hardware efficient object detection with event data

Dominika Przewlocka-Rus, Tomasz Kryjak, and Marek Gorgon

In DSD 2024 : 2024 27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings, 2024

Abstract DOI arXiv Bib

The performance of object detection systems in automotive solutions must be as high as possible, with minimal response time and, due to the often battery-powered operation, low energy consumption. When designing such solutions, we therefore face challenges typical for embedded vision systems: the problem of fitting algorithms of high memory and computational complexity into small low-power devices. In this paper we propose PowerYOLO - a mixed precision solution, which targets three essential elements of such application. First, we propose a system based on a Dynamic Vision Sensor (DVS), a novel sensor, that offers low power requirements and operates well in conditions with variable illumination. It is these features that may make event cameras a preferential choice over frame cameras in some applications. Second, to ensure high accuracy and low memory and computational complexity, we propose to use 4-bit width Powers-of- Two (PoT) quantisation for convolution weights of the YOLO detector, with all other parameters quantised linearly. Finally, we embrace from PoT scheme and replace multiplication with bit-shifting to increase the efficiency of hardware acceleration of such solution, with a special convolution-batch normalisation fusion scheme. The use of specific sensor with PoT quantisation and special batch normalisation fusion leads to a unique system with almost 8x reduction in memory complexity and vast computational simplifications, with relation to a standard approach. This efficient system achieves high accuracy of mAP 0.301 on the GENt DVS dataset, marking the new state-of-the-art for such compressed model.
@inproceedings{badap-agh:156633, title = {PowerYOLO: mixed precision model for hardware efficient object detection with event data}, author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz and Gorgon, Marek}, year = {2024}, series = {DSD~2024 [Dokument elektroniczny] : 2024~27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings}, pages = {210–217}, doi = {10.1109/DSD64264.2024.00036}, collection = {DSD~2024 [Dokument elektroniczny] : 2024~27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings}, }

2022

KU KDM

Accelerated training of object detection DNNs for embedded vision systems

Michal Machura, Piotr Wzorek, Tomasz Kryjak, and Marek Gorgon

In KU KDM 2022 : fourteenth ACC Cyfronet AGH HPC users’ conference : online, 7-8 April 2022 : proceedings, 2022

@inproceedings{badap-agh:142206,
  title = {Accelerated training of object detection DNNs for embedded vision systems},
  author = {Machura, Michal and Wzorek, Piotr and Kryjak, Tomasz and Gorgon, Marek},
  year = {2022},
  series = {KU KDM~2022 : fourteenth ACC Cyfronet AGH HPC users' conference : online, 7-8 April 2022 : proceedings},
  pages = {55–56},
  url = {http://katalog.nukat.edu.pl/lib/item?id=chamo:5338936&fromLocationLink=false&theme=nukat},
  collection = {KU KDM~2022 : fourteenth ACC Cyfronet AGH HPC users' conference : online, 7-8 April 2022 : proceedings},
}

JSPS
Implementation of the PointPillars network for 3D object detection in reprogrammable heterogeneous devices using FINN

Joanna Lis, Konrad Lis, and Marek Gorgon

Journal of Signal Processing Systems for Signal, Image, and Video Technology, 2022

Abstract DOI Bib

In this paper, we present a hardware-software implementation of a deep neural network for object detection based on a point cloud obtained by a LiDAR sensor. The PointPillars network was used in the research, as it is a reasonable compromise between detection accuracy and calculation complexity. The Brevitas / PyTorch tools were used for network quantisation (described in our previous paper) and the FINN tool for hardware implementation in the reprogrammable Zynq UltraScale+ MPSoC device. The obtained results show that quite a significant computation precision limitation along with a few network architecture simplifications allows the solution to be implemented on a heterogeneous embedded platform with maximum 19% AP loss in 3D, maximum 8% AP loss in BEV and execution time 375ms (the FPGA part takes 262ms). We have also compared our solution in terms of inference speed with a Vitis AI implementation proposed by Xilinx (19 Hz frame rate). Especially, we have thoroughly investigated the fundamental causes of differences in the frame rate of both solutions. The code is available at https://github.com/vision-agh/pp-finn .
@article{badap-agh:141196, title = {Implementation of the PointPillars network for 3D object detection in reprogrammable heterogeneous devices using FINN}, author = {Lis, Joanna and Lis, Konrad and Gorgon, Marek}, year = {2022}, journal = {Journal of Signal Processing Systems for Signal, Image, and Video Technology}, series = {Journal of Signal Processing Systems for Signal, Image, and Video Technology}, volume = {94}, pages = {659–674}, doi = {10.1007/s11265-021-01733-4}, issn = {1939-8018}, collection = {Journal of Signal Processing Systems for Signal, Image, and Video Technology}, }

Book chapter Preludium

Systemy wizyjne w zastosowaniach przemyslowych

Marek Gorgon, Maciej Aleksandrowicz, Krzysztof Blachut, Andrzej Brodzicki, Zbigniew Bublinski, and 20 more authors

In Nauka – technika – technologia : seria wydawnicza AGH, 2022

Abstract DOI Bib

Streszczenie: Systemy wizyjne to dynamicznie rozwijająca się dziedzina nauki i techniki.W niniejszym artykule przedstawiono przegląd zagadnień, nad którymi pracuje zespół Laboratorium Systemów Wizyjnych.Obejmuje on systemy wizyjne dla pojazdów autonomicznych, aplikacje analizy obrazów medycznych, systemy wspomagania szkolenia pracowników, analizę obrazów termowizyjnych, implementacje i akceleracje algorytmów wizyjnych w układach SoC FPGA i GPU, przetwarzanie strumienia wizyjnego w rozdzielczości 4K, a także analizę sygnałów z wykorzystaniem głębokich sieci neuronowych i współpracę przy eksperymencie ALICE w CERN-ie.

@inbook{badap-agh:141409,
  title = {Systemy wizyjne w zastosowaniach przemyslowych},
  author = {Gorgon, Marek and Aleksandrowicz, Maciej and Blachut, Krzysztof and Brodzicki, Andrzej and Bublinski, Zbigniew and Cyba, Artur and Danilowicz, Michal and Glowacz, Adam and Jablonski, Miroslaw and Jaworek-Korjakowska, Joanna and Kostuch, Aleksander and Kowalczyk, Marcin and Kryjak, Tomasz and Kucharski, Dariusz and Lis, Konrad and Machura, Michal and Mikrut, Zbigniew and Pawlik, Piotr and Piekarski, Michal and Przewlocka-Rus, Dominika and Lis, Joanna and Szolc, Hubert and Wasala, Mateusz and Wojcicka, Anna Malgorzata and Wzorek, Piotr},
  year = {2022},
  projects = {Preludium-Kowalczyk},
  series = {Nauka – technika – technologia : seria wydawnicza AGH},
  volume = {T.~5},
  pages = {17–31},
  doi = {10.7494/978-83-67427-00-5_1},
  collection = {Nauka – technika – technologia : seria wydawnicza AGH},
}

2021

FPL
Hardware-software implementation of a DNN for 3D object detection using FINN - a demo

Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

In FPL 2021 : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings, 2021

Abstract DOI Bib

In this demo, we present a hardware-software system for 3D object detection in LiDAR point clouds based on a deep neural network. The PointPillars architecture was used in this research, as it is a reasonable compromise between detection accuracy and computational complexity. The Brevitas tool was used for network quantisation and the FINN tool for hardware-software implementation in the reprogrammable Zynq UltraScale+ MPSoC device.
@inproceedings{badap-agh:137413, title = {Hardware-software implementation of a DNN for 3D object detection using FINN - a demo}, author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek}, year = {2021}, series = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings}, pages = {398–398}, doi = {10.1109/FPL53798.2021.00083}, collection = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings}, }
DASIP
Hardware-software implementation of the PointPillars network for 3D object detection in point clouds

Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

In DASIP 2021 : workshop on Design and Architectures for Signal and Image Processing - 14th edition : January 18–20 2021, Budapest, Hungary : proceedings, 2021

Abstract DOI Bib

In this paper, we present a hardware-software implementation of a deep neural network for object detection based on a point cloud obtained by a LiDAR sensor. The Brevitas / PyTorch tools were used for network quantisation and the FINN tool for hardware implementation in the reprogrammable Zynq UltraScale+ MPSoC device. The PointPillars network was used in the research, as it is a reasonable compromise between detection accuracy and calculation complexity. The obtained results show that quite a significant computation precision limitation along with a few network architecture simplifications allows the solution to be implemented on an heterogeneous embedded platform with reasonable detection accuracy.
@inproceedings{badap-agh:132404, title = {Hardware-software implementation of the PointPillars network for 3D object detection in point clouds}, author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek}, year = {2021}, series = {DASIP~2021 [Dokument elektroniczny] : workshop on Design and Architectures for Signal and Image Processing - 14th~edition : January 18–20 2021, Budapest, Hungary : proceedings}, pages = {44–51}, doi = {10.1145/3441110.3441150}, collection = {DASIP~2021 [Dokument elektroniczny] : workshop on Design and Architectures for Signal and Image Processing - 14th~edition : January 18–20 2021, Budapest, Hungary : proceedings}, }

2020

ICCVG
A vision based hardware-software real-time control system for the autonomous landing of an UAV

Krzysztof Blachut, Hubert Szolc, Mateusz Wasala, Tomasz Kryjak, and Marek Gorgon

In Computer Vision and Graphics : International Conference : ICCVG 2020 : Warsaw, Poland, September 14–16, 2020 : proceedings, 2020

Abstract DOI arXiv Bib

In this paper we present a vision based hardware-software control system enabling autonomous landing of a mul-tirotor unmanned aerial vehicle (UAV). It allows the detection of a marked landing pad in real-time for a 1280 x 720 @ 60 fps video stream. In addition, a LiDAR sensor is used to measure the altitude above ground. A heterogeneous Zynq SoC device is used as the computing platform. The solution was tested on a number of sequences and the landing pad was detected with 96% accuracy. This research shows that a reprogrammable heterogeneous computing system is a good solution for UAVs because it enables real-time data stream processing with relatively low energy consumption.
@inproceedings{badap-agh:130006, title = {A vision based hardware-software real-time control system for the autonomous landing of an UAV}, author = {Blachut, Krzysztof and Szolc, Hubert and Wasala, Mateusz and Kryjak, Tomasz and Gorgon, Marek}, year = {2020}, series = {Computer Vision and Graphics : International Conference : ICCVG~2020 : Warsaw, Poland, September 14–16, 2020 : proceedings}, pages = {13–24}, doi = {10.1007/978-3-030-59006-2_2}, collection = {Computer Vision and Graphics : International Conference : ICCVG~2020 : Warsaw, Poland, September 14–16, 2020 : proceedings}, }
KKA
Foreground object segmentation in RGB–D data implemented on GPU

Piotr Janus, Tomasz Kryjak, and Marek Gorgon

In Advanced, contemporary control : proceedings of KKA 2020 – the 20th Polish control conference : [14-16 October, 2020], Łódź, Poland, 2020

Abstract DOI arXiv Bib

This paper presents a GPU implementation of two foreground object segmentation algorithms: Gaussian Mixture Model (GMM) and Pixel Based Adaptive Segmenter (PBAS) modified for RGB-D data support. The simultaneous use of colour (RGB) and depth (D) data allows to improve segmentation accuracy, especially in case of colour camouflage, illumination changes and occurrence of shadows. Three GPUs were used to accelerate calculations: embedded NVIDIA Jetson TX2 (Maxwell architecture), mobile NVIDIA GeForce GTX 1050m (Pascal architecture) and efficient NVIDIA RTX 2070 (Turing architecture).Segmentation accuracy comparable to previously published works was obtained. Moreover, the use of a GPU platform allowed to get real-time image processing. In addition, the system has been adapted to work with two RGB-D sensors: RealSense D415 and D435 from Intel.
@inproceedings{badap-agh:129297, title = {Foreground object segmentation in RGB–D data implemented on GPU}, author = {Janus, Piotr and Kryjak, Tomasz and Gorgon, Marek}, year = {2020}, series = {Advanced, contemporary control : proceedings of KKA 2020 – the 20th Polish control conference : [14-16 October, 2020], \L{}\'{o}d\'{z}, Poland}, pages = {809–820}, doi = {10.1007/978-3-030-50936-1_68}, collection = {Advanced, contemporary control : proceedings of KKA 2020 – the 20th Polish control conference : [14-16 October, 2020], \L{}\'{o}d\'{z}, Poland}, }
SPA
Optimisation of the PointPillars network for 3D object detection in point clouds

Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

In SPA 2020 : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings, 2020

Abstract DOI arXiv Bib

In this paper we present our research on the optimisation of a deep neural network for 3D object detection in a point cloud. Techniques like quantisation and pruning available in the Brevitas and PyTorch tools were used. We performed the experiments for the PointPillars network, which offers a reasonable compromise between detection accuracy and calculation complexity. The aim of this work was to propose a variant of the network which we will ultimately implement in an FPGA device. This will allow for real-time LiDAR data processing with low energy consumption. The obtained results indicate that even a significant quantisation from 32-bit floating point to 2-bit integer in the main part of the algorithm, results in 5%-9% decrease of the detection accuracy, while allowing for almost a 16-fold reduction in size of the model.
@inproceedings{badap-agh:130350, title = {Optimisation of the PointPillars network for 3D object detection in point clouds}, author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek}, year = {2020}, series = {SPA~2020 [Dokument elektroniczny] : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings}, pages = {122–127}, doi = {10.23919/SPA50552.2020.9241265}, collection = {SPA~2020 [Dokument elektroniczny] : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings}, }

2019

DASIP

Hardware-software implementation of car detection system based on LiDAR sensor data - a demo

Joanna Stanisz, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

In dasip : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montréal, Canada, 2019

The article presents an object detection system based on LiDAR sensor data. The solution allows real-time detection of cars using information about the distance and intensity of the reflection obtained from LiDAR. The module has been implemented in programmable logic resources of the Xilinx Zynq SoC (System on Chip) device. The detected objects were projected onto the camera image for visualization. The system is dedicated to Velodyne HDL-64E LiDAR.

@inproceedings{badap-agh:126248,
  title = {Hardware-software implementation of car detection system based on LiDAR sensor data - a demo},
  author = {Stanisz, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
  year = {2019},
  series = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
  pages = {1–2},
  url = {https://dasip-conference.org/wp-content/uploads/sites/2/2019/10/DASIP2019_paper_30.pdf},
  collection = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
}

2018

DASIP

Hardware implementation of multi-scale Lucas-Kanade optical flow computation algorithm – a demo

Krzysztof Blachut, Tomasz Kryjak, and Marek Gorgon

In dasip 2018 : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal, 2018

@inproceedings{badap-agh:118435,
  title = {Hardware implementation of multi-scale Lucas-Kanade optical flow computation algorithm – a~demo},
  author = {Blachut, Krzysztof and Kryjak, Tomasz and Gorgon, Marek},
  year = {2018},
  series = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
  pages = {1–2},
  url = {https://web.fe.up.pt/~specs/events/dasip2018/files/paper_49.pdf},
  collection = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
}

DASIP
Hardware – software implementation of a SFM module for navigation an unmanned aerial vehicles – a demo

Karol Radwan, Tomasz Kryjak, and Marek Gorgon

In dasip : the 2018 conference on Design & Architectures for Signal & Image Processing : Porto, Portugal, October 10–12, 2018, 2018

Abstract DOI Bib

In this paper the design of an embedded vision system for structure from motion (SFM) computation is presented. The solution allows for scene depth reconstruction based on two consecutive video frames registered by a moving camera. The module is developed for a Xilinx Zynq SoC (System on Chip) device and is divided into a hardware and software part. The programmable logic (PL) is used mainly for detect and match features. Fundamental matrix estimation and triangulation are executed in the processing system (PS-ARM based). The module is dedicated to navigation of unmanned aerial vehicles (UAV)-autonomous flight and landing. The system is adapted to process a 1280×720 pixel video stream @ 60 frames per second in realtime.
@inproceedings{badap-agh:119436, title = {Hardware – software implementation of a SFM module for navigation an unmanned aerial vehicles – a~demo}, author = {Radwan, Karol and Kryjak, Tomasz and Gorgon, Marek}, year = {2018}, series = {dasip [Dokument elektroniczny] : the 2018~conference on Design \& Architectures for Signal \& Image Processing : Porto, Portugal, October 10–12, 2018}, pages = {60–61}, doi = {10.1109/DASIP.2018.8596937}, collection = {dasip [Dokument elektroniczny] : the 2018~conference on Design \& Architectures for Signal \& Image Processing : Porto, Portugal, October 10–12, 2018}, }
JRTIP
Real-time hardware-software embedded vision system for ITS smart camera implemented in Zynq SoC

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

Journal of Real-Time Image Processing, 2018

Abstract DOI Bib

The article demonstrates the usefulness of heterogeneous System on Chip (SoC) devices in smart cameras used in intelligent transportation systems (ITS). In a compact, energy efficient system the following exemplary algorithms were implemented: vehicle queue length estimation, vehicle detection, vehicle counting and speed estimation (using multiple virtual detection lines), as well as vehicle type (local binary features and SVM classifier) and colour (k-means classifier and YCbCr colourspace analysis) recognition. The solution exploits the hardware–software architecture, i.e. the combination of reconfigurable resources and the efficient ARM processor. Most of the modules were implemented in hardware, using Verilog HDL, taking full advantage of the possible parallelization and pipeline, which allowed to obtain real-time image processing. The ARM processor is responsible for executing some parts of the algorithm, i.e. high-level image processing and analysis, as well as for communication with the external systems (e.g. traffic lights controllers). The demonstrated results indicate that modern SoC systems are a very interesting platform for advanced ITS systems and other advanced embedded image processing, analysis and recognition applications.
@article{badap-agh:104286, title = {Real-time hardware-software embedded vision system for ITS smart camera implemented in Zynq SoC}, author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek}, year = {2018}, journal = {Journal of Real-Time Image Processing}, series = {Journal of Real-Time Image Processing}, volume = {15}, number = {1}, pages = {123}, doi = {10.1007/s11554-016-0588-9}, issn = {1861-8200}, collection = {Journal of Real-Time Image Processing}, }

2017

DASIP

Demo: embedded vision system for pedestrian detection based on HoG+SVM and use of motion information implemented in Zynq heterogeneous device

Bartosz Meus, Tomasz Kryjak, and Marek Gorgon

In dasip 2017 : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden, 2017

@inproceedings{badap-agh:109751,
  title = {Demo: embedded vision system for pedestrian detection based on HoG+SVM and use of motion information implemented in Zynq heterogeneous device},
  author = {Meus, Bartosz and Kryjak, Tomasz and Gorgon, Marek},
  year = {2017},
  series = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden},
  pages = {1–2},
  collection = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden},
}

SPA
Embedded vision system for pedestrian detection based on HOG+SVM and use of motion information implemented in Zynq heterogeneous device

Bartosz Meus, Tomasz Kryjak, and Marek Gorgon

In SPA 2017 : Signal Processing : Algorithms, Architectures, Arrangements, and Applications : Poznan, 20–22nd September 2017 : conference proceedings, 2017

Abstract DOI Bib

Pedestrian detection is a very important application of embedded real-time vision systems. It is essential in Advanced Driver Assistance Systems (ADAS) and Advanced Video Surveillance Systems (AVSS). The most widely used method involves a combination of Histogram of Oriented Gradients (HOG) features and Support Vector Machine (SVM) classifier. It offers quite high detection accuracy at the cost of high computational complexity. Therefore, it is impossible to use GPP-based (General Purpose Processor) embedded systems for this application. In this paper a hardware-software HOG+SVM pedestrian detection system implemented in a heterogeneous Xilinx Zynq device is presented. The base algorithm is supported with detection grouping and tracking procedures. They increase the performance of the solution, especially in case of temporal partially occluded or vague visible objects. The system is able to process a 1280 × 720 @ 60 fps video stream in real-time.
@inproceedings{badap-agh:108956, title = {Embedded vision system for pedestrian detection based on HOG+SVM and use of motion information implemented in Zynq heterogeneous device}, author = {Meus, Bartosz and Kryjak, Tomasz and Gorgon, Marek}, year = {2017}, series = {SPA~2017 : Signal Processing : Algorithms, Architectures, Arrangements, and Applications : Poznan, 20–22nd September 2017 : conference proceedings}, pages = {406–411}, doi = {10.23919/SPA.2017.8166901}, collection = {SPA~2017 : Signal Processing : Algorithms, Architectures, Arrangements, and Applications : Poznan, 20–22nd September 2017 : conference proceedings}, }
DASIP
Hardware-software abandoned object detection vision system in heterogeneous Zynq device

Tomasz Kryjak, Artur Skirzynski, and Marek Gorgon

In dasip 2017 : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden, 2017

Abstract DOI Bib

In this paper a hardware-software abandoned object detection vision system implemented in the Zynq SoC (System on Chip) device is presented. First, the solution was implemented in C++ and run as a bare metal application on the ARM processor core of the Zynq (using floating and fixed-point computations). For the target video stream 1280 χ 720 @ 50 fps (74.25 MHz pixel clock) it reached only 2 fps. Therefore, to speed-up the application, it was decided to move some of the image processing and analysis operations to the programmable logic. This allowed to obtain real-time image processing i.e. 50 fps, with power consumption of less than 4W.
@inproceedings{badap-agh:109748, title = {Hardware-software abandoned object detection vision system in heterogeneous Zynq device}, author = {Kryjak, Tomasz and Skirzynski, Artur and Gorgon, Marek}, year = {2017}, series = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden}, pages = {1–2}, doi = {10.1109/DASIP.2017.8122122}, collection = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden}, }
DASIP
Object tracking with the use of a moving camera implemented in heterogeneous Zynq SoC – a demo

Marcin Kowalczyk, Tomasz Kryjak, and Marek Gorgon

In dasip 2017 : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden, 2017

Abstract Bib

In this paper a hardware-software design of an object tracking system, which uses a moving camera is presented. The solution is implemented on the Zybo development board with the Zynq SoC (System on Chip) device from Xilinx. The object’s position is used to control two servomotors, which constitute a pan-tilt mounting of the camera. The proposed system is able to process a 1280 x 720 @ 60 fps video stream in real-time and track moving objects.
@inproceedings{badap-agh:109750, title = {Object tracking with the use of a moving camera implemented in heterogeneous Zynq SoC – a~demo}, author = {Kowalczyk, Marcin and Kryjak, Tomasz and Gorgon, Marek}, year = {2017}, series = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden}, pages = {1–2}, url = {https://ieeexplore.ieee.org/document/8122123}, collection = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden}, }

2016

ARC
An efficient hardware architecture for block based image processing algorithms

Tomasz Kryjak, Marek Gorgon, and Mateusz Komorkiewicz

In Applied Reconfigurable Computing : 12th international symposium, ARC 2016 : Mangaratiba, RJ, Brazil, March 22–24, 2016 : proceedings, 2016

Abstract DOI Bib

In this paper a novel method for data reordering in a video stream is presented. It can be used in high performance vision algorithms that require block based image processing. For case study a block based optical flow histogram computation application was selected. The proposed solution allows for a 2.3x and 6.3x speed-up for fixed-point and floating-point calculations respectively. This enables real-time operations for 1920 × 1080 pixels or higher resolutions
@inproceedings{badap-agh:97049, title = {An efficient hardware architecture for block based image processing algorithms}, author = {Kryjak, Tomasz and Gorgon, Marek and Komorkiewicz, Mateusz}, year = {2016}, series = {Applied Reconfigurable Computing : 12th international symposium, ARC~2016 : Mangaratiba, RJ, Brazil, March 22–24, 2016 : proceedings}, pages = {54–65}, doi = {10.1007/978-3-319-30481-6_5}, collection = {Applied Reconfigurable Computing : 12th international symposium, ARC~2016 : Mangaratiba, RJ, Brazil, March 22–24, 2016 : proceedings}, }

DASIP

Demo: FPGA implementation of the flux tensor moving object detection method

Piotr Janus, Kamil Piszczek, Tomasz Kryjak, and Marek Gorgon

In dasip : the 2016 conference on Design & Architectures for Signal & Image Processing : Rennes, France, October 12–14, 2016 : proceedings, 2016

@inproceedings{badap-agh:101813,
  title = {Demo: FPGA implementation of the flux tensor moving object detection method},
  author = {Janus, Piotr and Piszczek, Kamil and Kryjak, Tomasz and Gorgon, Marek},
  year = {2016},
  series = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
  pages = {286–287},
  collection = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
}

DASIP

Demo: an embedded vision system supporting the home care for convalescent or elderly people

Piotr Konopka, Tomasz Jonak, Tomasz Kryjak, and Marek Gorgon

In dasip : the 2016 conference on Design & Architectures for Signal & Image Processing : Rennes, France, October 12–14, 2016 : proceedings, 2016

@inproceedings{badap-agh:101812,
  title = {Demo: an embedded vision system supporting the home care for convalescent or elderly people},
  author = {Konopka, Piotr and Jonak, Tomasz and Kryjak, Tomasz and Gorgon, Marek},
  year = {2016},
  series = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
  pages = {288–289},
  collection = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
}

DASIP

Demo: autonomous vehicle model with vision-based tracking system

Krzysztof Mazur, Tomasz Kryjak, and Marek Gorgon

In dasip : the 2016 conference on Design & Architectures for Signal & Image Processing : Rennes, France, October 12–14, 2016 : proceedings, 2016

@inproceedings{badap-agh:101815,
  title = {Demo: autonomous vehicle model with vision-based tracking system},
  author = {Mazur, Krzysztof and Kryjak, Tomasz and Gorgon, Marek},
  year = {2016},
  series = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
  pages = {293–294},
  collection = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
}

IPC
FPGA implementation of multi-scale pedestrian detection in thermal images

Tomasz Kanka, Tomasz Kryjak, and Marek Gorgon

Image Processing & Communications, 2016

Abstract Bib

In this paper an embedded vision system for human silhouette detection in thermal images is presented. As the computing platform a reprogrammable device (FPGA - Field Programmable Gate Array) is used. The detection algorithm is based on a sliding window approach, which content is compared with a probabilistic template. Moreover, detection is four scales in supported. On the used test database, the proposed method obtained 97% accuracy, with average one false detection per frame. Due to the used parallelization and pipelining real-time processing for 720 × 480 @ 50 fps and 1280 × 720 @ 50 fps video streams was achieved. The system has been practically verified in a test setup with a thermal camera.
@article{badap-agh:112592, title = {FPGA implementation of multi-scale pedestrian detection in thermal images}, author = {Kanka, Tomasz and Kryjak, Tomasz and Gorgon, Marek}, year = {2016}, journal = {Image Processing \& Communications}, series = {Image Processing \& Communications}, volume = {21}, pages = {55}, issn = {1425-140X}, url = {https://goo.gl/jj6BRX}, collection = {Image Processing \& Communications}, }
DASIP
FPGA-based hardware-in-the-loop environment using video injection concept for camera-based systems in automotive applications

Mateusz Komorkiewicz, Pawel Skruch, Tomasz Kryjak, Krzysztof Turek, and Marek Gorgon

In dasip : the 2016 conference on Design & Architectures for Signal & Image Processing : Rennes, France, October 12–14, 2016 : proceedings, 2016

Abstract Bib

In this paper an FPGA-based video HiL (Hardwarein-the-Loop) solution is presented. It was designed to evaluate an of-the-shelf ADAS (Advanced Driver Assistance System) smart camera consisting of a video sensor (so-called forward looking camera (FLC)) and video processing unit. It allows to directly capture and store on a PC the video stream from the sensor. Then, the pre-recorded sequences can be directly injected into the video processing unit. Therefore, it is possible to evaluate the vision system in the laboratory under conditions almost identical to those present during real test drives. Moreover, all experiments are fully repeatable. The proposed system supports video streams with resolution 1280×960 @ 30 fps in the RCCC (Red/Clear)1 colour system. It is used in research and development projects in Delphi Technical Center Krakow.
@inproceedings{badap-agh:101814, title = {FPGA-based hardware-in-the-loop environment using video injection concept for camera-based systems in automotive applications}, author = {Komorkiewicz, Mateusz and Skruch, Pawel and Kryjak, Tomasz and Turek, Krzysztof and Gorgon, Marek}, year = {2016}, series = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings}, pages = {179–185}, collection = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings}, }

2015

DASIP
FPGA based system for real-time structure from motion computation

Mateusz Komorkiewicz, Tomasz Kryjak, Katarzyna Chuchacz-Kowalczyk, Pawel Skruch, and Marek Gorgon

In dasip : proceedings of the 2015 conference on Design & Architectures for Signal & Image Processing : Cracow, Poland, September 23–25, 2015, 2015

Abstract Bib

In this paper a system for real-time structure from motion computation is presented. The module is dedicated for a video based Advanced Driver Assistance System, where information about the surrounding environment and scene depth is often required to facilitate advanced applications. A heterogeneous System on a Chip solution, based on a FPGA device and an ARM processor is presented. The described hardware-software co-design is able to perform all steps required for structure from motion computation: feature extraction for every video frame, feature matching between two consecutive frames, finding the fundamental matrix, recovering the projection matrix and points triangulation. The design space exploration for the hardware part of the system is presented, which clearly points out the tradeoffs between resource usage, maximum clock and results accuracy. The system can process in real-time the images of 720x576 pixel resolution @ 50 fps. While the triangulation module allows for even high definition images processing and is able to compute results for up to 388 million points per second.
@inproceedings{badap-agh:93233, title = {FPGA based system for real-time structure from motion computation}, author = {Komorkiewicz, Mateusz and Kryjak, Tomasz and Chuchacz-Kowalczyk, Katarzyna and Skruch, Pawel and Gorgon, Marek}, year = {2015}, series = {dasip [Dokument elektroniczny] : proceedings of the 2015~conference on Design \& Architectures for Signal \& Image Processing : Cracow, Poland, September 23–25, 2015}, pages = {28–34}, collection = {dasip [Dokument elektroniczny] : proceedings of the 2015~conference on Design \& Architectures for Signal \& Image Processing : Cracow, Poland, September 23–25, 2015}, }
ICIAP
Real-time implementation of background modelling algorithms in FPGA devices

Tomasz Kryjak and Marek Gorgon

In ICIAP 2015 : international workshops BioFor, CTMR, RHEUMA, ISCA, MADiMa, SBMI, and QoEM : new trends in image analysis and processing : Genoa, Italy, September 7–8, 2015 : proceedings, 2015

Abstract DOI Bib

The article discusses the possibilities of hardware implementation of foreground object segmentation and background modelling algorithms in FPGA. The potential benefits, as well as challenges and problems associated with porting algorithms from general-purpose processors (CPU) to reconfigurable logic (FPGA) are presented. Also several hardware implementation of well known method are reviewed: GMM, Codebook, Clustering, ViBE and PBAS. The last algorithm was also evaluated on the SBI dataset.
@inproceedings{badap-agh:92036, title = {Real-time implementation of background modelling algorithms in FPGA devices}, author = {Kryjak, Tomasz and Gorgon, Marek}, year = {2015}, series = {ICIAP~2015 [Dokument elektroniczny] : international workshops BioFor, CTMR, RHEUMA, ISCA, MADiMa, SBMI, and QoEM : new trends in image analysis and processing : Genoa, Italy, September 7–8, 2015 : proceedings}, pages = {519–526}, doi = {10.1007/978-3-319-23222-5_63}, collection = {ICIAP~2015 [Dokument elektroniczny] : international workshops BioFor, CTMR, RHEUMA, ISCA, MADiMa, SBMI, and QoEM : new trends in image analysis and processing : Genoa, Italy, September 7–8, 2015 : proceedings}, }
IPC
Segmentation of dishes for customer service automation in a self-service canteen

Tomasz Kryjak, Damian Król, and Marek Gorgon

Image Processing & Communications, 2015

Abstract DOI Bib

The article describes research on dishes segmentation for the purposes of customer service process automation in a self-service canteen. The project assumptions and a prototype test stand are presented. The developed empty workspace detection and tray position determination algorithms are discussed. Finally, the chosen dishes segmentation algorithm is described and justified.
@article{badap-agh:96348, title = {Segmentation of dishes for customer service automation in a self-service canteen}, author = {Kryjak, Tomasz and Kr\'{o}l, Damian and Gorgon, Marek}, year = {2015}, journal = {Image Processing \& Communications}, series = {Image Processing \& Communications}, volume = {20}, pages = {5}, doi = {10.1515/ipc-2015-0030}, issn = {1425-140X}, collection = {Image Processing \& Communications}, }

Conference

Segmentation of dishes for the purposes of customer service process automation in a self-service canteen

Tomasz Kryjak, Damian Król, and Marek Gorgon

In Przetwarzanie i analiza sygnałów w systemach wizji i sterowania : Słok, 17–18 czerwca 2015 : [streszczenia artykułów], 2015

@inproceedings{badap-agh:97542,
  title = {Segmentation of dishes for the purposes of customer service process automation in a self-service canteen},
  author = {Kryjak, Tomasz and Kr\'{o}l, Damian and Gorgon, Marek},
  year = {2015},
  series = {Przetwarzanie i~analiza sygna\l{}\'{o}w w~systemach wizji i~sterowania [Dokument elektroniczny] : S\l{}ok, 17–18 czerwca 2015 : [streszczenia artyku\l{}\'{o}w]},
  pages = {2–2},
  url = {http://www.kis.p.lodz.pl/g/content/file/slok/Streszczenia_AGH.pdf},
  collection = {Przetwarzanie i~analiza sygna\l{}\'{o}w w~systemach wizji i~sterowania [Dokument elektroniczny] : S\l{}ok, 17–18 czerwca 2015 : [streszczenia artyku\l{}\'{o}w]},
}

2014

Sensors
Efficient hardware implementation of the Horn-Schunck algorithm for high-resolution real-time dense optical flow sensor

Mateusz Komorkiewicz, Tomasz Kryjak, and Marek Gorgon

Sensors, 2014

Abstract DOI Bib

This article presents an efficient hardware implementation of the Horn-Schunck algorithm that can be used in an embedded optical flow sensor. An architecture is proposed, that realises the iterative Horn-Schunck algorithm in a pipelined manner. This modification allows to achieve data throughput of 175 MPixels/s and makes processing of Full HD video stream (1; 920 × 1; 080 @ 60 fps) possible. The structure of the optical flow module as well as pre- and post-filtering blocks and a flow reliability computation unit is described in details. Three versions of optical flow modules, with different numerical precision, working frequency and obtained results accuracy are proposed. The errors caused by switching from floating- to fixed-point computations are also evaluated. The described architecture was tested on popular sequences from an optical flow dataset of the Middlebury University. It achieves state-of-the-art results among hardware implementations of single scale methods. The designed fixed-point architecture achieves performance of 418 GOPS with power efficiency of 34 GOPS/W. The proposed floating-point module achieves 103 GFLOPS, with power efficiency of 24 GFLOPS/W. Moreover, a 100 times speedup compared to a modern CPU with SIMD support is reported. A complete, working vision system realized on Xilinx VC707 evaluation board is also presented. It is able to compute optical flow for Full HD video stream received from an HDMI camera in real-time. The obtained results prove that FPGA devices are an ideal platform for embedded vision systems.
@article{badap-agh:80299, title = {Efficient hardware implementation of the Horn-Schunck algorithm for high-resolution real-time dense optical flow sensor}, author = {Komorkiewicz, Mateusz and Kryjak, Tomasz and Gorgon, Marek}, year = {2014}, journal = {Sensors}, series = {Sensors}, volume = {14}, number = {2}, pages = {2860}, doi = {10.3390/s140202860}, issn = {1424-8220}, collection = {Sensors}, }
DASIP
Hardware-software implemantation of vehicle detection and counting using virtual detection lines

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

In dasip : proceedings of the 2014 conference on Design & Architectures for Signal & Image Processing : Madrid, Spain, October 8–10, 2014, 2014

Abstract DOI Bib

In this paper a hardware-software system for vehicle detection and counting at intersections is presented. It has been implemented and evaluated on the heterogeneous Zynq platform. The vehicle detection is based on the concept of virtual detection lines (VDL). Differences in colour, horizontal edges and Census transform results between areas around the VDL for two consecutive frames are used to disclosure particular vehicles. This part of the system is designed in hardware description language and implemented in the reconfigurable resources of the Zynq platform. Information about the vehicle presence, along with the time-spatial image obtained at the VDL are transmitted and processed on the ARM core integrated in the Zynq device. This allows to perform further analysis and eliminate false or multiple detections. The solution has been evaluated on several video sequences recorded in various conditions: sunny day (the occurrence of deep shadows), cloudy day, rain and night-time. The system could be an important element of an intelligent transportation system (ITS) i.e. applied in a smart camera.
@inproceedings{badap-agh:85563, title = {Hardware-software implemantation of vehicle detection and counting using virtual detection lines}, author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek}, year = {2014}, series = {dasip [Dokument elektroniczny] : proceedings of the 2014 conference on Design \& Architectures for Signal \& Image Processing : Madrid, Spain, October 8–10, 2014}, pages = {1–8}, doi = {10.1109/DASIP.2014.7115618}, collection = {dasip [Dokument elektroniczny] : proceedings of the 2014 conference on Design \& Architectures for Signal \& Image Processing : Madrid, Spain, October 8–10, 2014}, }
ICCVG
Implementation of advanced foreground segmentation algorithms GMM, ViBE and PBAS in FPGA and GPU – a comparison

Bartlomiej Bulat, Tomasz Kryjak, and Marek Gorgon

In Computer Vision and Graphics : International Conference : ICCVG 2014 : Warsaw, Poland, September 15–17, 2014 : proceedings, 2014

Abstract DOI Bib

The article presents the results of implementing advanced foreground object segmentation algorithms: GMM (Gaussian Mixture Model), ViBE (Visual Background Extractor) and PBAS (Pixel-Based Adaptive Segmenter) on different hardware platforms: CPU, GPU and FPGA. The influence of the architecture on the segmentation accuracy and feasibility to perform real-time video stream processing was analysed. Also the limitations resulting from the specific features of GPU and FPGA were pointed out. Furthermore, the possible use of different platforms in advanced vision systems was discussed.
@inproceedings{badap-agh:84669, title = {Implementation of advanced foreground segmentation algorithms GMM, ViBE and PBAS in FPGA and GPU – a~comparison}, author = {Bulat, Bartlomiej and Kryjak, Tomasz and Gorgon, Marek}, year = {2014}, series = {Computer Vision and Graphics : International Conference : ICCVG 2014 : Warsaw, Poland, September 15–17, 2014 : proceedings}, pages = {124–131}, doi = {10.1007/978-3-319-11331-9_16}, collection = {Computer Vision and Graphics : International Conference : ICCVG 2014 : Warsaw, Poland, September 15–17, 2014 : proceedings}, }
JRTIP Preludium
Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

Journal of Real-Time Image Processing, 2014

Abstract DOI Bib

The processing of a high-definition video stream in real-time is a challenging task for embedded systems. However, modern FPGA devices have both a high operating frequency and sufficient logic resources to be successfully used in these tasks. In this article, an advanced system that is able to generate and maintain a complex background model for a scene as well as segment the foreground for an HD colour video stream (1,920 × 1,080 @ 60 fps) in real-time is presented. The possible application ranges from video surveillance to machine vision systems. That is, in all cases, when information is needed about which objects are new or moving in the scene. Excellent results are obtained by using the CIE Lab colour space, advanced background representation as well as integrating information about lightness, colour and texture in the segmentation step. Finally, the complete system is implemented in a single high-end FPGA device.
@article{badap-agh:80341, title = {Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device}, author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek}, year = {2014}, journal = {Journal of Real-Time Image Processing}, series = {Journal of Real-Time Image Processing}, volume = {9}, number = {1}, pages = {61}, doi = {10.1007/s11554-012-0290-5}, issn = {1861-8200}, collection = {Journal of Real-Time Image Processing}, projects = {Preludium-Kryjak}, }
IJET
Real-time foreground object detection combining the PBAS background modelling algorithm and feedback from scene analysis module

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

International Journal of Electronics and Telecommunications, 2014

Abstract DOI Bib

The article presents a hardware implementation of the foreground object detection algorithm PBAS (Pixel-Based Adaptive Segmenter) with a scene analysis module. A mechanism for static object detection is proposed, which is based on consecutive frame differencing. The method allows to distinguish stopped foreground objects (e.g. a car at the intersection, abandoned luggage) from false detections (so-called ghosts) using edge similarity. The improved algorithm was compared with the original version on popular test sequences from the changedetection.net dataset. The obtained results indicate that the proposed approach allows to improve the performance of the method for sequences with the stopped objects. The algorithm has been implemented and successfully verified on a hardware platform with Virtex 7 FPGA device. The PBAS segmentation, consecutive frame differencing, Sobel edge detection and advanced one-pass connected component analysis modules were designed. The system is capable of processing 50 frames with a resolution of 720 × 576 pixels per second
@article{badap-agh:81415, title = {Real-time foreground object detection combining the PBAS background modelling algorithm and feedback from scene analysis module}, author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek}, year = {2014}, journal = {International Journal of Electronics and Telecommunications}, series = {International Journal of Electronics and Telecommunications}, volume = {60}, pages = {61}, doi = {10.2478/eletel-2014-0006}, issn = {2081-8491}, collection = {International Journal of Electronics and Telecommunications}, }

DASIP

Video surveillance algorithms implemented on the heterogeneous Zynq platform : [poster]

Tomasz Kryjak, Mateusz Zun, and Marek Gorgon

In dasip : proceedings of the 2014 conference on Design & Architectures for Signal & Image Processing : Madrid, Spain, October 8–10, 2014, 2014

@inproceedings{badap-agh:85623,
  title = {Video surveillance algorithms implemented on the heterogeneous Zynq platform : [poster]},
  author = {Kryjak, Tomasz and Zun, Mateusz and Gorgon, Marek},
  year = {2014},
  booktitle = {dasip [Dokument elektroniczny] : proceedings of the 2014 conference on Design \& Architectures for Signal \& Image Processing : Madrid, Spain, October 8–10, 2014},
  pages = {1},
}

Journal
Real-time implementation of foreground object detection from a moving camera using ViBE algorithm

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

Computer Science and Information Systems, 2014

Abstract DOI Bib

The article presents a real-time hardware implementation of a foreground object detection for a non-static camera setup. The system consists of two parts: the calculation of the displacement between two consecutive frames using a correlation based corner tracker and background generation method ViBE (Visual Background Extractor). The paper discusses details of the used hardware modules, resource utilization, computing performance and power dissipation. The solution was evaluated on sequences recorded with a static and moving camera. The system was successfully tested on a hardware platform with an FPGA device. It allows to process a 720x576 pixels and 50 frames per second video stream in real-time.
@article{badap-agh:86089, title = {Real-time implementation of foreground object detection from a moving camera using ViBE algorithm}, author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek}, year = {2014}, journal = {Computer Science and Information Systems}, volume = {11}, number = {4}, pages = {1617–1637}, doi = {10.2298/CSIS131218055K}, issn = {1820-0214}, }

2013

Conference
Hardware implementation of the PBAS foreground detection method in FPGA

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

In MIXDES 2013 : mixed design of integrated circuits and systems : Gdynia, June 20–22, 2013 : book of abstracts of the 20th international conference, 2013

Abstract Bib

In this paper a hardware implementation of the state-of-the-art PBAS (Pixel-Based Adaptive Segmenter) foreground object detection algorithm in FPGA is presented. This background modelling and subtraction technique is using novel ideas which improve the detection results. In this work a research is presented that determines which parts of the original algorithm need to be adjusted and modified to allow hardware implementation. A comparison is presented how this modifications influence the results accuracy compared to the original software implementation and other algorithms. An example working system based on the proposed module using the evaluation board with Virtex 7 device is also presented. This is the first reported FPGA implementation of this algorithm.
@inproceedings{badap-agh:74690, title = {Hardware implementation of the PBAS foreground detection method in FPGA}, author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek}, year = {2013}, booktitle = {MIXDES 2013 : mixed design of integrated circuits and systems : Gdynia, June 20–22, 2013 : book of abstracts of the 20th international conference}, pages = {129}, }
FedCSIS
Real-time implementation of the ViBe foreground object segmentation algorithm

Tomasz Kryjak and Marek Gorgon

In FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : September 8–11, 2013, Kraków, Poland, 2013

Abstract Bib

This paper presents a novel real-time hardware implementation of the ViBe (VIsual Background Extractor) background generation algorithm in reconfigurable FPGA device. This novel method combines the advantages of typical recursive and non-recursive approaches and achieves very good foreground object segmentation results. In this work the issue of porting ViBe to a FPGA hardware platform is discussed, two modification to the original approach are proposed and a detailed description of the implemented system is presented. This is the first, known to the authors, FPGA implementation of this algorithm
@inproceedings{badap-agh:76794, title = {Real-time implementation of the ViBe foreground object segmentation algorithm}, author = {Kryjak, Tomasz and Gorgon, Marek}, year = {2013}, booktitle = {FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : September 8–11, 2013, Kraków, Poland}, pages = {56}, }
PAK
Implementacja sprzętowa odejmowania tła metodą ViBe w układzie FPGA

Tomasz Kryjak and Marek Gorgon

Pomiary, Automatyka, Kontrola, 2013

Abstract Bib

This paper presents a hardware implementation in the FPGA reconfigurable device of ViBe - a novel background subtraction algorithm. The method is based on a different, from those previously described and implemented, background modelling concept. It partly uses random numbers, which allowed us to significantly reduce the buffer size in relation to the standard methods like mean or median form a buffer. A detailed description of ViBe can be found in papers [6, 7, 8]. In this paper the role of background generation algorithms in image processing and analysis systems, with particular emphasis on hardware implementations is discussed (Section 1). The ViBe algorithm is described in Section 2. Then an analysis of the possibility of implementing ViBe in FPGA is presented (Section 3). Section 4 describes two proposed modifications: the use of the CIE Lab colour space and the enhanced flashing pixels detection method. Their desirability has been confirmed quantitatively using the "ChangeDetection" database [9]. A detailed description of the designed ViBe hardware module and image processing system is presented in Section 5. The scheme of the ViBe module is shown in Figure 5 and the whole system in Figure 4. Table 3 summarizes the hardware resource utilization. The proposed solution enables the detection of objects using the method ViBe and enables realtime processing of a colour 640 x 480 video stream at 60 frames per second. The obtained results confirm the high usefulness of FPGA in the implementation of advanced image processing and analysis algorithms.
@article{badap-agh:78883, title = {Implementacja sprzętowa odejmowania tła metodą ViBe w układzie FPGA}, author = {Kryjak, Tomasz and Gorgon, Marek}, year = {2013}, journal = {Pomiary, Automatyka, Kontrola}, volume = {59}, number = {11}, pages = {1191–1194}, issn = {0032-4140}, }

2012

Rekonfigurowalny system wizyjnego nadzoru do detekcji naruszenia obszarów chronionych

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

Pomiary, Automatyka, Kontrola, 2012

@article{badap-agh:69876,
  title = {Rekonfigurowalny system wizyjnego nadzoru do detekcji naruszenia obszarów chronionych},
  author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
  year = {2012},
  journal = {Pomiary, Automatyka, Kontrola},
  volume = {58},
  number = {7},
  pages = {584–586},
  issn = {0032-4140},
  projects = {Preludium-Kryjak},
}

DASIP Preludium

FPGA implementation of real-time head-shoulder detection using local binary patterns, SVM and foreground object detection

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

In dasip : proceedings of the 2012 conference on Design & Architectures for Signal & Image Processing : Karlsruhe, Germany, October 23–25, 2012, 2012

@inproceedings{badap-agh:69895,
  title = {FPGA implementation of real-time head-shoulder detection using local binary patterns, SVM and foreground object detection},
  author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
  year = {2012},
  booktitle = {dasip [Dokument elektroniczny] : proceedings of the 2012 conference on Design \& Architectures for Signal \& Image Processing : Karlsruhe, Germany, October 23–25, 2012},
  pages = {232–239},
  projects = {Preludium-Kryjak},
}

DASIP Preludium

FPGA implementation of camera tamper detection in real-time

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

In dasip : proceedings of the 2012 conference on Design & Architectures for Signal & Image Processing : Karlsruhe, Germany, October 23–25, 2012, 2012

@inproceedings{badap-agh:69897,
  title = {FPGA implementation of camera tamper detection in real-time},
  author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
  year = {2012},
  booktitle = {dasip [Dokument elektroniczny] : proceedings of the 2012 conference on Design \& Architectures for Signal \& Image Processing : Karlsruhe, Germany, October 23–25, 2012},
  pages = {224–231},
  projects = {Preludium-Kryjak},
}

DASIP Preludium
Is FPGA a suitable platform for advanced video surveillance systems?

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

In dasip : proceedings of the 2012 conference on Design & Architectures for Signal & Image Processing : Karlsruhe, Germany, October 23–25, 2012, 2012

Abstract Bib

In the demo four video surveillance algorithms implemented on an FPGA platform, running in real-time are presented: detecting intrusions into protected areas, camera tamper detection, head and shoulders detection using LBP and SVM and head, people and bicycles detection using HOG and SVM.
@inproceedings{badap-agh:69900, title = {Is FPGA a suitable platform for advanced video surveillance systems?}, author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek}, year = {2012}, booktitle = {dasip [Dokument elektroniczny] : proceedings of the 2012 conference on Design \& Architectures for Signal \& Image Processing : Karlsruhe, Germany, October 23–25, 2012}, pages = {369–370}, projects = {Preludium-Kryjak}, }

2011

KU KDM

Accelerating computing tasks in video surveillance systems using FPGA

Tomasz Kryjak and Marek Gorgon

In KU KDM 2011 : fourth ACC Cyfronet AGH users’ conference : Zakopane, March 09–11, 2011 : proceedings, 2011

@inproceedings{badap-agh:58732,
  title = {Accelerating computing tasks in video surveillance systems using FPGA},
  author = {Kryjak, Tomasz and Gorgon, Marek},
  year = {2011},
  booktitle = {KU KDM 2011 : fourth ACC Cyfronet AGH users' conference : Zakopane, March 09–11, 2011 : proceedings},
  pages = {38–40},
}

DASIP

Real-time moving object detection for video surveillance system in FPGA

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

In dasip : conference on Design & Architectures for Signal & Image Processing : Tampere, Finland, November 2nd–4th, 2011, 2011

@inproceedings{badap-agh:62870,
  title = {Real-time moving object detection for video surveillance system in FPGA},
  author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
  year = {2011},
  booktitle = {dasip [Dokument elektroniczny] : conference on Design \& Architectures for Signal \& Image Processing : Tampere, Finland, November 2nd–4th, 2011},
  pages = {209–216},
}

FPL

Single low-power FPGA implementation of moving object detection

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

In FPL 2011 : proceedings of the FPL2011 workshop on Computer vision on low-power reconfigurable architectures : September 4, 2011, Chania, Crete, Greese, 2011

@inproceedings{badap-agh:62871,
  title = {Single low-power FPGA implementation of moving object detection},
  author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
  year = {2011},
  booktitle = {FPL 2011 : proceedings of the FPL2011 workshop on Computer vision on low-power reconfigurable architectures : September 4, 2011, Chania, Crete, Greese},
  pages = {7–8},
}

Journal
Real-time implementation of moving object detection in video surveillance systems using FPGA

Tomasz Kryjak and Marek Gorgon

Computer Science, 2011

Abstract Bib

The article presents the concept of real-time implementation computing tasks in video surveillance systems. A pipeline implementation of a multimodal background generation algorithm for colour video stream and a moving objects segmentation based on brightness, colour and textural information in reconfigurable resources of FPGA device is described. System architecture, resource usage and segmentation results are presented.
@article{badap-agh:63727, title = {Real-time implementation of moving object detection in video surveillance systems using FPGA}, author = {Kryjak, Tomasz and Gorgon, Marek}, year = {2011}, journal = {Computer Science}, volume = {12}, pages = {162}, issn = {1508-2806}, }
Automatyka
Implementacja algorytmu generacji tła wraz z modułem segmentacji obiektów ruchomych i eliminacji cieni w układach FPGA serii Spartan 6

Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

Automatyka, 2011

Abstract Bib

The article describes an implementation of a moving object detection system consisting of a digital camera, a Spartan 6 FPGA device and a LCD monitor. The object detection method is based on differential images. It uses the following algorithms: background generation, background subtraction and shadow removal. The basis of the system are two modules designed in VHDL: advanced background generation and moving objects segmentation based on brightness, colour and texture analysis. In addition the construction of modules allowing communication with the camera, execution of the Bayer transform, RGB to CIE Lab colour space conversion and fast interface to the external DDR3 RAM is described. The paper also presents the usage of FPGA resources and tests results of the proposed system.
@article{badap-agh:64605, title = {Implementacja algorytmu generacji tła wraz z modułem segmentacji obiektów ruchomych i eliminacji cieni w układach FPGA serii Spartan 6}, author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek}, year = {2011}, journal = {Automatyka}, volume = {15}, number = {3}, pages = {197–217}, issn = {1429-3447}, }

2010

KU KDM
Pipeline implementation of peer group filtering in FPGA

Tomasz Kryjak and Marek Gorgon

In KU KDM 2010 : third ACC Cyfronet AGH users’ conference : Zakopane March 18–19, 2010 : proceedings, 2010

Abstract Bib

In the paper a parallel FPGA implementation of the Peer Group Filtering algorithm is described. Implementation details, results, performance of the design and FPGA logic resources are discussed. The PGF algorithm customized for FPGA is compared with the original one and Vector Median Filtering.
@inproceedings{badap-agh:51482, title = {Pipeline implementation of peer group filtering in FPGA}, author = {Kryjak, Tomasz and Gorgon, Marek}, year = {2010}, booktitle = {KU KDM 2010 : third ACC Cyfronet AGH users' conference : Zakopane March 18–19, 2010 : proceedings}, pages = {27–29}, }
Journal
Parallel implementation of local thresholding in Mitrion-C

Tomasz Kryjak and Marek Gorgon

International Journal of Applied Mathematics and Computer Science, 2010

Abstract DOI Bib

Mitrion-C based implementations of three image processing algorithms: a look-up table operation, simple local thresholding and Sauvola’s local thresholding are described. Implementation results, performance of the design and FPGA logic utilization are discussed.
@article{badap-agh:55263, title = {Parallel implementation of local thresholding in Mitrion-C}, author = {Kryjak, Tomasz and Gorgon, Marek}, year = {2010}, journal = {International Journal of Applied Mathematics and Computer Science}, volume = {20}, number = {3}, pages = {571–580}, doi = {10.2478/v10006-010-0042-2}, issn = {1641-876X}, }

2009

KU KDM

Parallel implementation of local thresholding in Mitrion-C

Tomasz Kryjak and Marek Gorgon

In KU KDM 2009 : druga konferencja użytkowników komputerów dużej mocy : Zakopane, March 12–13, 2009 : abstracts, 2009

@inproceedings{badap-agh:44944,
  title = {Parallel implementation of local thresholding in Mitrion-C},
  author = {Kryjak, Tomasz and Gorgon, Marek},
  year = {2009},
  booktitle = {KU KDM 2009 : druga konferencja użytkowników komputerów dużej mocy : Zakopane, March 12–13, 2009 : abstracts},
  pages = {21–23},
}

FPL
Pipeline implementation of the 128-bit block cipher CLEFIA in FPGA

Tomasz Kryjak and Marek Gorgon

In FPL09 Prague : 19th international conference on Field Programmable Logic and Applications (FPL) : Prague, Czech Republic August 31 – September 2, 2009 : proceedings, 2009

Abstract DOI Bib

The article presents a pipeline implementation of the block cipher CLEFIA. The article examines three known methods of implementing a single encryption round and proposes a new fourth method. The article proposes the implementation of a key scheduler, which is highly compatible with pipeline encryption. The article contains a detailed analysis of the data processing path for the 128-bit key version of the algorithm and verifies its operation on two FPGA cards in practice. On the basis of one of these cards, the article proposes a prototype of an effective supercomputer-compatible hardware accelerator (High Performance Computing Application).
@inproceedings{badap-agh:51633, title = {Pipeline implementation of the 128-bit block cipher CLEFIA in FPGA}, author = {Kryjak, Tomasz and Gorgon, Marek}, year = {2009}, booktitle = {FPL09 Prague : 19th international conference on Field Programmable Logic and Applications (FPL) : Prague, Czech Republic August 31 – September 2, 2009 : proceedings}, pages = {373–378}, doi = {10.1109/FPL.2009.5272264}, }

2008

Automatyka
Akcelerator sprzętowy do szyfrowania strumienia danych

Tomasz Kryjak and Marek Gorgon

Automatyka, 2008

Abstract Bib

The paper describes a data stream encryption hardware accelerator. The used DES algorithm and its implementation in VHDL language have been discussed. Different FPGA platforms have been evaluated to determine the most suitable one for creating an accelerator cooperating with a PC computer. The software and hardware parts of the presented solution, as well as problems with data transfer form the PC computer to the FPGA device have been described. Correctness and speed of the implemented application have been tested. Finally, the performance of the presented hardware solution and software solutions has been compared.
@article{badap-agh:42953, title = {Akcelerator sprzętowy do szyfrowania strumienia danych}, author = {Kryjak, Tomasz and Gorgon, Marek}, year = {2008}, journal = {Automatyka}, volume = {12}, number = {3}, pages = {695–707}, issn = {1429-3447}, }