Tomasz Kryjak

2026

1. TRETS

   Hardware-accelerated graph neural networks: an alternative approach for event-based audio classification and keyword spotting on SoC FPGA

   Kamil Jeziorek, Piotr Wzorek, Krzysztof Blachut, Hiroshi Nakano, Manon Dampfhoffer, and 4 more authors

   ACM Transactions on Reconfigurable Technology and Systems, 2026

   Abstract DOI Bib

   As the volume of data recorded by embedded edge sensors increases, particularly from neuromorphic devices producing discrete event streams, there is a need for hardware-aware neural architectures that enable efficient, low-latency and energy-conscious local processing. To address this research gap, we present an FPGA implementation of event-graph neural networks for audio processing. We utilise an artificial cochlea that converts time-series signals into sparse event data, reducing memory and computation costs. Our architecture was implemented on an SoC FPGA and evaluated on two open source datasets. For the classification task, our baseline floating-point model achieves 92.7% accuracy on the SHD dataset—only 2.4% and 2% below the state-of-the-art—while requiring 10 and 67 fewer parameters, respectively. Compared to FPGA-based spiking neural networks, our quantised model reaches 92.3% accuracy, outperforming them by 19.3 pps while reducing resource usage and latency. For the SSC dataset, we report the first hardware-accelerated evaluation, and our models achieve 66.9–71.0% accuracy. Extending our conference work, we further demonstrate the first end-to-end FPGA implementation of the event-audio keyword spotting task, combining graph convolutional layers with recurrent sequence modelling for continuous, streaming operation. The system achieves 95% word-end detection accuracy, with only 10.53 µs latency and 1.18 W power consumption, establishing a strong benchmark for energy-efficient event-driven KWS.

   @article{badap-agh:170304,
     title = {Hardware-accelerated graph neural networks: an alternative approach for event-based audio classification and keyword spotting on SoC FPGA},
     author = {Jeziorek, Kamil and Wzorek, Piotr and Blachut, Krzysztof and Nakano, Hiroshi and Dampfhoffer, Manon and Mesquida, Thomas and Nishi, Hiroaki and Dalgaty, Thomas and Kryjak, Tomasz},
     year = {2026},
     journal = {ACM Transactions on Reconfigurable Technology and Systems},
     volume = {19},
     pages = {45},
     doi = {10.1145/3838800},
   }

2. Automation

   Vision-based automatic fruit counting with UAV

   Hubert Szolc, Mateusz Wasala, Remigiusz Mietla, Kacper Iwicki, and Tomasz Kryjak

   In Automation 2025: recent advances in automation, robotics and measurement techniques, 2026

   Abstract DOI arXiv Bib

   The use of unmanned aerial vehicles (UAVs) for smart agriculture is becoming increasingly popular. This is evidenced by recent scientific works, as well as the various competitions organised on this topic. Therefore, in this work we present a system for automatic fruit counting using UAVs. To detect them, our solution uses a vision algorithm that processes streams from an RGB camera and a depth sensor using classical image operations. Our system also allows the planning and execution of flight trajectories, taking into account the minimisation of flight time and distance covered. We tested the proposed solution in simulation and obtained an average score of 87.27/100 points from a total of 500 missions. We also submitted it to the UAV Competition organised as part of the ICUAS 2024 conference, where we achieved an average score of 84.83/100 points, placing 6th in a field of 23 teams and advancing to the finals.

   @inproceedings{badap-agh:165951,
     title = {Vision-based automatic fruit counting with UAV},
     author = {Szolc, Hubert and Wasala, Mateusz and Mietla, Remigiusz and Iwicki, Kacper and Kryjak, Tomasz},
     year = {2026},
     series = {Automation~2025: recent advances in automation, robotics and measurement techniques},
     pages = {243–252},
     doi = {10.1007/978-3-032-08359-3_23},
     collection = {Automation~2025: recent advances in automation, robotics and measurement techniques},
   }

3. KU KDM ARTIQ
   Preludium
   

   Towards event-based in-cabin sensing: a synchronized dual-camera dataset pipeline

   Marcin Kowalczyk, Marek Dlugosz, Kamil Jeziorek, Mateusz Komorkiewicz, Tomasz Kryjak, and 4 more authors

   In KU KDM 2026 : eighteenth ACC Cyfronet AGH HPC users conference : Zakopane, 13–15 April 2026 : proceedings, 2026

   Bib

   @inproceedings{badap-agh:167206,
     title = {Towards event-based in-cabin sensing: a synchronized dual-camera dataset pipeline},
     author = {Kowalczyk, Marcin and Dlugosz, Marek and Jeziorek, Kamil and Komorkiewicz, Mateusz and Kryjak, Tomasz and Marchewka, Dariusz and Szelest, Marcin and Wzorek, Piotr and Skruch, Pawel},
     year = {2026},
     series = {KU KDM~2026 : eighteenth ACC Cyfronet AGH HPC users conference : Zakopane, 13–15 April~2026 : proceedings},
     pages = {41–42},
     collection = {KU KDM~2026 : eighteenth ACC Cyfronet AGH HPC users conference : Zakopane, 13–15 April~2026 : proceedings},
     projects = {ARTIQ, Preludium-Kowalczyk},
   }

4. TRETS

   SIRA: scaled-integer range analysis for optimizing FPGA dataflow neural network accelerators

   Yaman Umuroglu, Christoph Berganski, Felix Jentzsch, Michal Danilowicz, Tomasz Kryjak, and 6 more authors

   ACM Transactions on Reconfigurable Technology and Systems, 2026

   Abstract Collaboration DOI arXiv Bib

   • Computer Engineering Group
     Paderborn University, Germany
   • AMD Research & Advanced Development
   • Computer Architecture Laboratory (CAL)
     Norwegian University of Science and Technology (NTNU), Norway

   All our collaborations

   While neural network quantization effectively reduces the cost of matrix multiplications, aggressive quantization can expose non-matrix-multiply operations as significant performance and resource bottlenecks on embedded systems. Addressing such bottlenecks requires a comprehensive approach to tailoring the precision across operations in the inference computation. To this end, we introduce scaled-integer range analysis ( SIRA ), a static analysis technique employing interval arithmetic to determine the range, scale, and bias for tensors in quantized neural networks. We show how this information can be exploited to reduce the resource footprint of FPGA dataflow neural network accelerators via tailored bitwidth adaptation for accumulators and downstream operations, aggregation of scales and biases, and conversion of consecutive elementwise operations to thresholding operations. We integrate SIRA -driven optimizations into the open source FINN framework, then evaluate their effectiveness across a range of quantized neural network workloads and compare implementation alternatives for non-matrix-multiply operations. We demonstrate an average reduction of 17% for LUTs, 66% for DSPs, and 22% for accumulator bitwidths with SIRA optimizations, providing detailed benchmark analysis and analytical models to guide the implementation style for non-matrix layers. Finally, we open source SIRA to facilitate community exploration of its benefits across various applications and hardware platforms.

   @article{badap-agh:168428,
     title = {SIRA: scaled-integer range analysis for optimizing FPGA dataflow neural network accelerators},
     author = {Umuroglu, Yaman and Berganski, Christoph and Jentzsch, Felix and Danilowicz, Michal and Kryjak, Tomasz and Bezaitis, Charalampos and Sj\"{a}lander, Magnus and Colbert, Ian and Preusser, Thomas and Petri-Koenig, Jakoba and Blott, Michaela},
     year = {2026},
     journal = {ACM Transactions on Reconfigurable Technology and Systems},
     series = {ACM Transactions on Reconfigurable Technology and Systems},
     volume = {19},
     number = {2},
     pages = {23},
     doi = {10.1145/3807510},
     issn = {1936-7406},
     collection = {ACM Transactions on Reconfigurable Technology and Systems},
   }

5. JSA Preludium
   

   Embedded graph convolutional networks for real-time event data processing on SoC FPGAs

   Kamil Jeziorek, Piotr Wzorek, Krzysztof Blachut, Andrea Pinna, and Tomasz Kryjak

   Journal of Systems Architecture, 2026

   Abstract Collaboration DOI arXiv Bib

   • SYEL (Systèmes Électroniques), LIP6 laboratory
     Sorbonne Université, France

   All our collaborations

   The utilisation of event cameras represents an important and swiftly evolving trend aimed at addressing the constraints of traditional video systems. Particularly within the automotive domain, these cameras find significant relevance for their integration into embedded real-time systems due to lower latency and power consumption. One effective approach to ensure the necessary throughput and latency for event processing is through the utilisation of graph convolutional networks (GCNs). In this study, we introduce a custom EFGCN (Event-based FPGA-accelerated Graph Convolutional Network) designed with a series of hardware-aware optimisations tailored for PointNetConv, a graph convolution designed for point cloud processing. The proposed techniques result in up to 100-fold reduction in model size compared to Asynchronous Event-based GNN (AEGNN), one of the most recent works in the field, with a relatively small decrease in accuracy (2.9% for the N-Caltech101 classification task, 2.2% for the N-Cars classification task), thus following the TinyML trend. We implemented EFGCN on a ZCU104 SoC FPGA platform without any off-chip external memory resources, achieving a throughput of 13.3 million events per second (MEPS) and real-time partially asynchronous processing with low latency. Across multiple event-based classification benchmarks, our approach achieves competitive accuracy while providing state-of-the-art computational efficiency per event, small model size, and high scalability, customisability and resource efficiency. We publish both software and hardware source code in an open repository: https://github.com/vision-agh/gcnn-dvs-fpga .

   @article{badap-agh:168430,
     title = {Embedded graph convolutional networks for real-time event data processing on SoC FPGAs},
     author = {Jeziorek, Kamil and Wzorek, Piotr and Blachut, Krzysztof and Pinna, Andrea and Kryjak, Tomasz},
     year = {2026},
     journal = {Journal of Systems Architecture},
     series = {Journal of Systems Architecture},
     volume = {177},
     pages = {103850},
     doi = {10.1016/j.sysarc.2026.103850},
     issn = {1383-7621},
     collection = {Journal of Systems Architecture},
     projects = {Preludium-Wzorek},
   }

6. ARC Preludium
   

   End-to-end keyword spotting on FPGA using graph neural networks with a neuromorphic auditory sensor

   Wiktor Matykiewicz, Piotr Wzorek, Kamil Jeziorek, Tomás Muñoz, Antonio Rios-Navarro, and 2 more authors

   In Applied Reconfigurable Computing : architectures, tools, and applications : 22nd international symposium, ARC 2026 : Cagliari, Sardinia, Italy, April 8–10, 2026 : proceedings, 2026

   Abstract Collaboration DOI arXiv Bib

   • Robotics and Technology of Computers Lab (RTC)
     Universidad de Sevilla, Spain

   All our collaborations

   With the rapid growth of mobile robotics and embedded intelligence, there is an increasing demand for efficient on-device data processing on edge platforms. A promising research direction is the use of neuromorphic sensors inspired by human sensory systems, which generate sparse, event-based data encoding changes in the environment. In this work, we present the first end-to-end FPGA implementation of a keyword spotting system that integrates a Neuromorphic Auditory Sensor (NAS) and a graph neural network (GNN) on a single FPGA device, enabling real-time processing of raw audio data. The proposed architecture eliminates conventional signal preprocessing and operates directly on event-based audio streams. Leveraging a compute-near-memory network architecture, the system achieves efficient inference with low latency and low power consumption. Experimental results demonstrate an accuracy of 87.43% after quantization on the Google Speech Commands v2 dataset processed through the neuromorphic sensor, with end-to-end latency below 35 us and average power consumption of 1.12 W. The processed datasets, software models, and hardware modules are available at https://github.com/vision-agh/NAS-GNN-KWS.

   @inproceedings{badap-agh:168992,
     title = {End-to-end keyword spotting on FPGA using graph neural networks with a neuromorphic auditory sensor},
     author = {Matykiewicz, Wiktor and Wzorek, Piotr and Jeziorek, Kamil and Mu\~{n}oz, Tom\'{a}s and Rios-Navarro, Antonio and Jim\'{e}nez-Fern\'{a}ndez, Angel and Kryjak, Tomasz},
     year = {2026},
     series = {Applied Reconfigurable Computing : architectures, tools, and applications : 22nd international symposium, ARC~2026 : Cagliari, Sardinia, Italy, April 8–10,~2026 : proceedings},
     pages = {119–136},
     doi = {10.1007/978-3-032-29365-7_8},
     collection = {Applied Reconfigurable Computing : architectures, tools, and applications : 22nd international symposium, ARC~2026 : Cagliari, Sardinia, Italy, April 8–10,~2026 : proceedings},
     projects = {Preludium-Wzorek},
   }

7. ARC Preludium
   

   FPGA-based hardware architecture for contrast maximization in event-based vision

   Michal Filipkowski, Marcin Kowalczyk, and Tomasz Kryjak

   In Applied Reconfigurable Computing : architectures, tools, and applications : 22nd international symposium, ARC 2026 : Cagliari, Sardinia, Italy, April 8–10, 2026 : proceedings, 2026

   Abstract DOI arXiv Bib

   This paper presents a hardware architecture that implements the Contrast Maximization (CM) algorithm in Field-Programmable Gate Array (FPGA) resources for event-based vision systems. CM estimates motion parameters by maximizing the contrast of an Image of Warped Events (IWE) reconstructed from asynchronous event streams. Event-based vision sensors generate sparse data with high temporal resolution and low spatial redundancy, which makes them well suited for hardware processing. The deterministic, massively parallel structure of the FPGA is leveraged to design a deeply pipelined architecture capable of high-throughput, energy-efficient processing suitable for real-time embedded applications. This paper details the hardware modules responsible for event warping, contrast computation, and iterative optimization, discusses key implementation decisions, and presents the hardware-aware optimization method used in the design. Experimental results demonstrate a substantial speed and efficiency improvement over CPU- and GPU-based implementations, with motion parameter estimation executing over 200 times faster. To the best of our knowledge, this is the first hardware architecture enabling acceleration of CM algorithm computations. Its performance is evaluated in terms of processing speed, energy efficiency, and hardware resource utilization. The proposed design is validated using an event-based object tracking application. The results confirm that the architecture provides a solid foundation for real-time motion estimation in high-speed, low-power embedded systems.

   @inproceedings{badap-agh:169140,
     title = {FPGA-based hardware architecture for contrast maximization in event-based vision},
     author = {Filipkowski, Michal and Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2026},
     projects = {Preludium-Kowalczyk},
     series = {Applied Reconfigurable Computing : architectures, tools, and applications : 22nd international symposium, ARC~2026 : Cagliari, Sardinia, Italy, April 8–10,~2026 : proceedings},
     pages = {309–325},
     doi = {10.1007/978-3-032-29365-7_19},
     collection = {Applied Reconfigurable Computing : architectures, tools, and applications : 22nd international symposium, ARC~2026 : Cagliari, Sardinia, Italy, April 8–10,~2026 : proceedings},
   }

2025

1. KU KDM Preludium
   

   High-performance computing for event cameras: DIF filtering and graph convolutional networks for object classification

   Marcin Kowalczyk, Kamil Jeziorek, Tomasz Kryjak, and Marek Gorgon

   In KU KDM 2025 : seventeenth ACC Cyfronet AGH HPC users’ conference : Zakopane, 2-4 April 2025 : proceedings, 2025

   Bib

   @inproceedings{badap-agh:159338,
     title = {High-performance computing for event cameras: DIF filtering and graph convolutional networks for object classification},
     author = {Kowalczyk, Marcin and Jeziorek, Kamil and Kryjak, Tomasz and Gorgon, Marek},
     year = {2025},
     projects = {Preludium-Kowalczyk},
     series = {KU KDM~2025 [Dokument elektroniczny] : seventeenth ACC Cyfronet AGH HPC users' conference : Zakopane, 2-4 April~2025 : proceedings},
     pages = {49-50},
     collection = {KU KDM~2025 [Dokument elektroniczny] : seventeenth ACC Cyfronet AGH HPC users' conference : Zakopane, 2-4 April~2025 : proceedings},
   }

2. ARC

   Real-time multi-object tracking using YOLOv8 and SORT on a SoC FPGA

   Michal Danilowicz and Tomasz Kryjak

   In Applied Reconfigurable Computing : architectures, tools, and applications : 21st international symposium, ARC 2025 : Seville, Spain, April 9–11, 2025 : proceedings, 2025

   Abstract DOI arXiv Bib

   Multi-object tracking (MOT) is one of the most important problems in computer vision and a key component of any vision-based perception system used in advanced autonomous mobile robotics. Therefore, its implementation on low-power and real-time embedded platforms is highly desirable. Modern MOT algorithms should be able to track objects of a given class (e.g. people or vehicles). In addition, the number of objects to be tracked is not known in advance, and they may appear and disappear at any time, as well as be obscured. For these reasons, the most popular and successful approaches have recently been based on the tracking paradigm. Therefore, the presence of a high quality object detector is essential, which in practice accounts for the vast majority of the computational and memory complexity of the whole MOT system. In this paper, we propose an FPGA (Field-Programmable Gate Array) implementation of an embedded MOT system based on a quantized YOLOv8 detector and the SORT (Simple Online Realtime Tracker) tracker. We use a modified version of the FINN framework to utilize external memory for model parameters and to support operations necessary required by YOLOv8. We discuss the evaluation of detection and tracking performance using the COCO and MOT15 datasets, where we achieve 0.21 mAP and 38.9 MOTA respectively. As the computational platform, we use an MPSoC system (Zynq UltraScale+ device from AMD/Xilinx) where the detector is deployed in reprogrammable logic and the tracking algorithm is implemented in the processor system.

   @inproceedings{badap-agh:159463,
     title = {Real-time multi-object tracking using YOLOv8 and SORT on a SoC FPGA},
     author = {Danilowicz, Michal and Kryjak, Tomasz},
     year = {2025},
     series = {Applied Reconfigurable Computing : architectures, tools, and applications : 21st international symposium, ARC~2025 : Seville, Spain, April 9–11,~2025 : proceedings},
     pages = {214–230},
     doi = {10.1007/978-3-031-87995-1_13},
     collection = {Applied Reconfigurable Computing : architectures, tools, and applications : 21st international symposium, ARC~2025 : Seville, Spain, April 9–11,~2025 : proceedings},
   }

3. ARC

   Hardware-accelerated event-graph neural networks for low-latency time-series classification on SoC FPGA

   Hiroshi Nakano, Krzysztof Blachut, Kamil Jeziorek, Piotr Wzorek, Manon Dampfhoffer, and 4 more authors

   In Applied Reconfigurable Computing : architectures, tools, and applications : 21st international symposium, ARC 2025 : Seville, Spain, April 9–11, 2025 : proceedings, 2025

   Abstract Collaboration DOI arXiv Bib

   • Digital Systems and Integrated Circuits Division
     CEA-List Grenoble, France
   • Faculty of Science and Technology, Department of System Design Engineering
     Keio University, Japan

   All our collaborations

   As the quantities of data recorded by embedded edge sensors grow, so too does the need for intelligent local processing. Such data often comes in the form of time-series signals, based on which real-time predictions can be made locally using an AI model. However, a hardware-software approach capable of making low-latency predictions with low power consumption is required. In this paper, we present a hardware implementation of an event-graph neural network for time-series classification. We leverage an artificial cochlea model to convert the input time-series signals into a sparse event-data format that allows the event-graph to drastically reduce the number of calculations relative to other AI methods. We implemented the design on a SoC FPGA and applied it to the real-time processing of the Spiking Heidelberg Digits (SHD) dataset to benchmark our approach against competitive solutions. Our method achieves a floating-point accuracy of 92.7% on the SHD dataset for the base model, which is only 2.4% and 2% less than the state-of-the-art models with over 10% and 67% fewer model parameters, respectively. It also outperforms FPGA-based spiking neural network implementations by 19.3% and 4.5%, achieving 92.3% accuracy for the quantised model while using fewer computational resources and reducing latency.

   @inproceedings{badap-agh:160129,
     title = {Hardware-accelerated event-graph neural networks for low-latency time-series classification on SoC FPGA},
     author = {Nakano, Hiroshi and Blachut, Krzysztof and Jeziorek, Kamil and Wzorek, Piotr and Dampfhoffer, Manon and Mesquida, Thomas and Nishi, Hiroaki and Kryjak, Tomasz and Dalgaty, Thomas},
     year = {2025},
     series = {Applied Reconfigurable Computing : architectures, tools, and applications : 21st international symposium, ARC~2025 : Seville, Spain, April 9–11,~2025 : proceedings},
     pages = {51–68},
     doi = {10.1007/978-3-031-87995-1_4},
     collection = {Applied Reconfigurable Computing : architectures, tools, and applications : 21st international symposium, ARC~2025 : Seville, Spain, April 9–11,~2025 : proceedings},
   }

4. SPIE RTIPDVI

   Self-supervised event representations: towards accurate, real-time perception on SoC FPGAs

   Kamil Jeziorek and Tomasz Kryjak

   In Real-time processing of image, depth, and video information 2025 : 7–10 April 2025, Prague, Czech Republic, 2025

   Abstract DOI arXiv Bib

   Event cameras offer significant advantages over traditional frame-based sensors. These include microsecond temporal resolution, robustness under varying lighting conditions and low power consumption. Nevertheless, the effective processing of their sparse, asynchronous event streams remains challenging. Existing approaches to this problem can be categorised into two distinct groups. The first group involves the direct processing of event data with neural models, such as Spiking Neural Networks or Graph Convolutional Neural Networks. However, this approach is often accompanied by a compromise in terms of qualitative performance. The second group involves the conversion of events into dense representations with handcrafted aggregation functions, which can boost accuracy at the cost of temporal fidelity. This paper introduces a novel Self-Supervised Event Representation (SSER) method leveraging Gated Recurrent Unit (GRU) networks to achieve precise per-pixel encoding of event timestamps and polarities without temporal discretisation. The recurrent layers are trained in a self-supervised manner to maximise the fidelity of event-time encoding. The inference is performed with event representations generated asynchronously, thus ensuring compatibility with high-throughput sensors. The experimental validation demonstrates that SSER outperforms aggregation-based baselines, achieving improvements of 2.4% mAP and 0.6% on the Gen1 and 1 Mpx object detection datasets. Furthermore, the paper presents the first hardware implementation of recurrent representation for event data on a System-on-Chip FPGA, achieving submicrosecond latency and power consumption between 1–2 W, suitable for real-time, power-efficient applications. Code is available at https://github.com/vision-agh/RecRepEvent.

   @inproceedings{badap-agh:160301,
     title = {Self-supervised event representations: towards accurate, real-time perception on SoC FPGAs},
     author = {Jeziorek, Kamil and Kryjak, Tomasz},
     year = {2025},
     series = {Real-time processing of image, depth, and video information~2025 : 7–10 April~2025, Prague, Czech Republic},
     pages = {1352602-1–1352602–14},
     doi = {10.1117/12.3056056},
     collection = {Real-time processing of image, depth, and video information~2025 : 7–10 April~2025, Prague, Czech Republic},
   }

5. MICPRO Preludium
   

   High throughput event filtering: the interpolation-based DIF algorithm hardware architecture

   Marcin Kowalczyk and Tomasz Kryjak

   Microprocessors and Microsystems, 2025

   Abstract DOI arXiv Bib

   In recent years, there has been rapid development in the field of event vision. It manifests itself both on the technical side, as better and better event sensors are available, and on the algorithmic side, as more and more applications of this technology are proposed and scientific papers are published. However, the data stream from these sensors typically contains a significant amount of noise, which varies depending on factors such as the degree of illumination in the observed scene or the temperature of the sensor. We propose a hardware architecture of the Distance-based Interpolation with Frequency Weights(DIF) filter and implement it on an FPGA chip. To evaluate the algorithm and compare it with other solutions, we have prepared a new high-resolution event dataset, which we are also releasing to the community. Our architecture achieved a throughput of 403.39 million events per second (MEPS) for a sensor resolution of 1280 × 720 and 428.45 MEPS for a resolution of 640 × 480. The averagevalues of the Area Under the Receiver Operating Characteristic (AUROC) index ranged from 0.844 to 0.999, depending on the dataset, which is comparable to the state-of-the-art filtering solutions, but with much higher throughput and better operation over a wide range of noise levels.

   @article{badap-agh:160777,
     title = {High throughput event filtering: the interpolation-based DIF algorithm hardware architecture},
     author = {Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2025},
     projects = {Preludium-Kowalczyk},
     journal = {Microprocessors and Microsystems},
     series = {Microprocessors and Microsystems},
     volume = {117},
     pages = {105171},
     doi = {10.1016/j.micpro.2025.105171},
     issn = {0141-9331},
     collection = {Microprocessors and Microsystems},
   }

6. DASIP

   LiFT: lightweight, FPGA-tailored 3D object detection based on LiDAR data

   Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP 2025 : Barcelona, Spain, January 20–22, 2025 : proceedings, 2025

   Abstract DOI arXiv Bib

   This paper presents LiFT, a lightweight, fully quantized 3D object detection algorithm for LiDAR data, optimized for real-time inference on FPGA platforms. Through an in-depth analysis of FPGA-specific limitations, we identify a set of FPGA-induced constraints that shape the algorithm’s design. These include a computational complexity limit of 30 GMACs (billion multiply-accumulate operations), INT8 quantization for weights and activations, 2D cell-based processing instead of 3D voxels, and minimal use of skip connections. To meet these constraints while maximizing performance, LiFT combines novel mechanisms with state-of-the-art techniques such as reparameterizable convolutions and fully sparse architecture. Key innovations include the Dual-bound Pillar Feature Net, which boosts performance without increasing complexity, and an efficient scheme for INT8 quantization of input features. With a computational cost of just 20.73 GMACs, LiFT stands out as one of the few algorithms targeting minimal-complexity 3D object detection. Among comparable methods, LiFT ranks first, achieving an mAP of 51.84% and an NDS of 61.01% on the challenging NuScenes validation dataset. The code will be available at https://github.com/vision-agh/lift.

   @inproceedings{badap-agh:160941,
     title = {LiFT: lightweight, FPGA-tailored 3D object detection based on LiDAR data},
     author = {Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2025},
     series = {Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP~2025 : Barcelona, Spain, January~20–22,~2025 : proceedings},
     pages = {28–40},
     doi = {10.1007/978-3-031-87897-8_3},
     collection = {Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP~2025 : Barcelona, Spain, January~20–22,~2025 : proceedings},
   }

7. CVPRW Preludium
   

   Learning from noise: enhancing DNNs for event-based vision through controlled noise injection

   Marcin Kowalczyk, Kamil Jeziorek, and Tomasz Kryjak

   In CVPRW 2025 : IEEE/CVF Computer Vision and Pattern Recognition Workshops : 11–15 June 2025, Nashville, [USA] : proceedings, 2025

   Abstract DOI arXiv Bib

   Event-based sensors offer significant advantages over traditional frame-based cameras, especially in scenarios involving rapid motion or challenging lighting conditions. However, event data frequently suffers from considerable noise, negatively impacting the performance and robustness of deep learning models. Traditionally, this problem has been addressed by applying filtering algorithms to the event stream, but this may also remove some of relevant data. In this paper, we propose a novel noise-injection training methodology designed to enhance the neural networks robustness against varying levels of event noise. Our approach introduces controlled noise directly into the training data, enabling models to learn noise-resilient representations. We have conducted extensive evaluations of the proposed method using multiple benchmark datasets (N-Caltech101, N-Cars, and Mini N-ImageNet) and various network architectures, including Convolutional Neural Networks, Vision Transformers, Spiking Neural Networks, and Graph Convolutional Networks. Experimental results show that our noise-injection training strategy achieves stable performance over a range of noise intensities, consistently outperforms event-filtering techniques, and achieves the highest average classification accuracy, making it a viable alternative to traditional event-data filtering methods in an object classification system. Code: https://github.com/vision-agh/DVS_Filtering

   @inproceedings{badap-agh:162623,
     title = {Learning from noise: enhancing DNNs for event-based vision through controlled noise injection},
     author = {Kowalczyk, Marcin and Jeziorek, Kamil and Kryjak, Tomasz},
     year = {2025},
     projects = {Preludium-Kowalczyk},
     series = {CVPRW~2025 [Dokument elektroniczny] : IEEE/CVF Computer Vision and Pattern Recognition Workshops : 11–15 June~2025, Nashville, [USA] : proceedings},
     pages = {5092–5102},
     doi = {10.1109/CVPRW67362.2025.00504},
     collection = {CVPRW~2025 [Dokument elektroniczny] : IEEE/CVF Computer Vision and Pattern Recognition Workshops : 11–15 June~2025, Nashville, [USA] : proceedings},
   }

8. CVPRW Preludium
   

   Live demonstration: real-time event-data processing with Graph Convolutional Neural Networks and SoC FPGA

   Piotr Wzorek, Krzysztof Blachut, Kamil Jeziorek, and Tomasz Kryjak

   In CVPRW 2025 : IEEE/CVF Computer Vision and Pattern Recognition Workshops : 11–15 June 2025, Nashville, [USA] : proceedings, 2025

   Abstract DOI Bib

   With growing demand for robust low-power and lowlatency computer vision, event cameras are becoming increasingly common in mobile robotics and autonomous vehicles due to their high temporal resolution, ability to operate in adverse lightning conditions and reduction of redundant information. However, fully exploiting the benefits of these sensors requires algorithms capable of processing sparse data efficiently and optimised for a specialised target platform. To address this demand, we present a hardware-aware model of Graph Convolutional Neural Networks (GCNNs) implemented on an embedded heterogeneous SoC FPGA. Leveraging the temporal sparsity of event data, our approach integrates partially asynchronous processing and 3D MaxPool layers to achieve real-time, high-accuracy object classification on embedded platforms. A key strength of our architecture is its scalability – customisable hardware modules enable flexible configurations to balance resource utilisation and latency. In this realtime demonstration we will showcase the capabilities of our GCNN model, address its limitations, and discuss future development directions. We will use the system operating on a heterogeneous SoC FPGA to classify event data captured directly by the integrated camera or read from an SD card with simulated time intervals between events. Our work has been shared in a open-source repository: https://github.com/vision-agh/gcnn-dvs-fpga

   @inproceedings{badap-agh:162625,
     title = {Live demonstration: real-time event-data processing with Graph Convolutional Neural Networks and SoC FPGA},
     author = {Wzorek, Piotr and Blachut, Krzysztof and Jeziorek, Kamil and Kryjak, Tomasz},
     year = {2025},
     series = {CVPRW~2025 [Dokument elektroniczny] : IEEE/CVF Computer Vision and Pattern Recognition Workshops : 11–15 June~2025, Nashville, [USA] : proceedings},
     pages = {5103–5104},
     doi = {10.1109/CVPRW67362.2025.00505},
     collection = {CVPRW~2025 [Dokument elektroniczny] : IEEE/CVF Computer Vision and Pattern Recognition Workshops : 11–15 June~2025, Nashville, [USA] : proceedings},
     projects = {Preludium-Wzorek},
   }

9. SPA

   Embedded perception and control system for an autonomous surface vehicle using visual-LiDAR fusion and SLAM

   Olivier Lichota, Mateusz Wasala, Hubert Szolc, and Tomasz Kryjak

   In SPA 2025 : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September 2025 : conference proceedings, 2025

   Abstract DOI Bib

   The paper presents the design and implementation of a modular perception and control system for an autonomous surface vehicle (ASV), developed on a System-on-Module (SoM) embedded platform. The system was created with the goal of participating in the international RoboBoat competition. The high-level architecture integrates data from a stereo camera, LiDAR sensor, and a GNSS unit with an Inertial Measurement Unit (IMU), enabling object detection, simultaneous localization and mapping (SLAM), and autonomous navigation. We evaluated the use of a quantized convolutional neural network (YOLOv11) for object classification, obstacle detection based on LiDAR data, and control module operation using NAV2 within the ROS 2 environment under limited computational resources. Accelerating visual detection algorithms with CUDA and TensorRT libraries on the onboard NVIDIA Jetson Orin Nano platform enabled effective real-time and low-energy classification. Tests carried out in simulation demonstrated the correct operation of the perception and control system. The average runtime of the system using the quantised YOLO model with GPU was 26.01 ms. This equates to an approximate speed increase of 6.7 times compared to inference performed on an ARM processor, and an approximate speed increase of 5.5 times compared to a standard CPU.

   @inproceedings{badap-agh:163124,
     title = {Embedded perception and control system for an autonomous surface vehicle using visual-LiDAR fusion and SLAM},
     author = {Lichota, Olivier and Wasala, Mateusz and Szolc, Hubert and Kryjak, Tomasz},
     year = {2025},
     series = {SPA~2025 [Dokument elektroniczny] : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September~2025 : conference proceedings},
     pages = {153–158},
     doi = {10.23919/SPA65537.2025.11215136},
     collection = {SPA~2025 [Dokument elektroniczny] : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September~2025 : conference proceedings},
   }

10. SPA Preludium
    

    Hardware-aware Graph Neural Networks prunning for embedded event-based vision

    Piotr Wzorek, Kamil Jeziorek, and Tomasz Kryjak

    In SPA 2025 : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September 2025 : conference proceedings, 2025

    Abstract DOI arXiv Bib

    Event-based cameras are gaining popularity as the sensor of choice for mobile robotics, due to their high performance in dynamic environments. However, these applications require efficient real-time data processing with low latency and power consumption. One strategy to meet these stringent requirements is hardware acceleration of efficient algorithms that preserve the temporal sparsity of event data.In this work, we propose an optimization strategy for Graph Convolutional Neural Networks models aimed at adapting their architecture to the limited resources of embedded heterogeneous FPGA platforms. Our method incorporates hardware-aware pruning and quantization, taking into account the trade-off between on-chip memory savings and inference accuracy. Strategic exploration of the design space with Fine Grid Search and Greedy layer-wise Iterative Deepening Search methods enables flexible adaptation of the model architecture to the target platform. Our approach was evaluated across various network configurations and multiple datasets, resulting in BRAM memory reductions of 28.8% for CIFAR-10 (with a 1.65% decrease in accuracy), 31.4% for MNIST-DVS (accuracy drop of 3.55%), and 26.5% for N-Caltech101 (with a 5.18% accuracy reduction).

    @inproceedings{badap-agh:163126,
      title = {Hardware-aware Graph Neural Networks prunning for embedded event-based vision},
      author = {Wzorek, Piotr and Jeziorek, Kamil and Kryjak, Tomasz},
      year = {2025},
      series = {SPA~2025 [Dokument elektroniczny] : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September~2025 : conference proceedings},
      pages = {182–187},
      doi = {10.23919/SPA65537.2025.11215154},
      collection = {SPA~2025 [Dokument elektroniczny] : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September~2025 : conference proceedings},
      projects = {Preludium-Wzorek},
    }

11. SPA

    A model-driven TTFS translation for ReLU neural networks

    Michal Michniak, Kamil Jeziorek, and Tomasz Kryjak

    In SPA 2025 : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September 2025 : conference proceedings, 2025

    Abstract DOI Bib

    Spiking neural networks (SNNs) offer a promising avenue for energy-efficient and biologically inspired computation, with the potential to run effectively on neuromorphic hardware. Recent advances have focused on translating conventional artificial neural networks (ANNs) into their spiking counterparts, particularly using time-to-first-spike (TTFS) coding to reduce the number of spikes and improve efficiency. However, most existing TTFS translation methods rely heavily on data-driven threshold estimation and multi-phase operations. In this work, we present a novel, fully model-driven TTFS translation approach for ReLU-PSP (Rectified Linear Postsynaptic Potential function) networks, eliminating the need for training data and employing a single, fixed activation threshold. We introduce a structural upper bound based on the ReLU1 function to control numerical stability and enable accurate inference, even with complex architectures containing residual connections. Our method is evaluated on multiple event-based datasets, demonstrating equivalent classification accuracy to conventional ANNs.

    @inproceedings{badap-agh:163127,
      title = {A model-driven TTFS translation for ReLU neural networks},
      author = {Michniak, Michal and Jeziorek, Kamil and Kryjak, Tomasz},
      year = {2025},
      series = {SPA~2025 [Dokument elektroniczny] : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September~2025 : conference proceedings},
      pages = {188–193},
      doi = {10.23919/SPA65537.2025.11215160},
      collection = {SPA~2025 [Dokument elektroniczny] : Signal Processing - algorithms, architectures, arrangements, and applications : Poznan, 17th–19th September~2025 : conference proceedings},
    }

12. DSD

    Hardware-aware feature extraction quantisation for real-time visual odometry on FPGA platforms

    Mateusz Wasala, Mateusz Smolarczyk, Michal Danilowicz, and Tomasz Kryjak

    In DSD 2025 : 28th euromicro conference on Digital System Design : 10–12 September 2025, Salerno, Italy, 2025

    Abstract DOI arXiv Bib

    Accurate position estimation is essential for modern navigation systems deployed in autonomous platforms, including ground vehicles, marine vessels, and aerial drones. In this context, Visual Simultaneous Localisation and Mapping (VSLAM) which includes Visual Odometry - relies heavily on the reliable extraction of salient feature points from the visual input data. In this work, we propose an embedded implementation of an unsupervised architecture capable of detecting and describing feature points. It is based on a quantised SuperPoint convolutional neural network. Our objective is to minimise the computational demands of the model while preserving high detection quality, thus facilitating efficient deployment on platforms with limited resources, such as mobile or embedded systems. We implemented the solution on an FPGA System-on-Chip (SoC) platform, specifically the AMD/Xilinx Zynq UltraScale+, where we evaluated the performance of Deep Learning Processing Units (DPUs) and we also used the Brevitas library and the FINN framework to perform model quantisation and hardware-aware optimisation. This allowed us to process 640 × 480 pixel images at up to 54 fps on an FPGA platform, outperforming state-of-the-art solutions in the field. We conducted experiments on the TUM dataset to demonstrate and discuss the impact of different quantisation techniques on the accuracy and performance of the model in a visual odometry task.

    @inproceedings{badap-agh:165139,
      title = {Hardware-aware feature extraction quantisation for real-time visual odometry on FPGA platforms},
      author = {Wasala, Mateusz and Smolarczyk, Mateusz and Danilowicz, Michal and Kryjak, Tomasz},
      year = {2025},
      series = {DSD~2025 [Dokument elektroniczny] : 28th euromicro conference on Digital System Design : 10–12 September~2025, Salerno, Italy},
      pages = {539–548},
      doi = {10.1109/DSD67783.2025.00080},
      collection = {DSD~2025 [Dokument elektroniczny] : 28th euromicro conference on Digital System Design : 10–12 September~2025, Salerno, Italy},
    }

13. FPL Preludium
    Preludium
    

    Live demonstration: Continuous processing of event-data with Graph Convolutional Neural Networks implemented for SoC FPGA

    Piotr Wzorek, Kamil Jeziorek, Marcin Kowalczyk, Krzysztof Blachut, Tomasz Kryjak, and 1 more author

    In FPL 2025 : 35th International Conference on Field-Programmable Logic and Applications : 1–5 September 2025 Leiden, Netherlands : proceedings, 2025

    Abstract DOI Bib

    For perception systems in mobile robotics - particularly in demanding, highly dynamic environments - event cameras (DVS - Dynamic Vision Sensors) are being increasingly utilised as an alternative to conventional vision sensors. The real-time processing of the registered spatio-temporal sparse point cloud must satisfy stringent requirements in terms of latency, throughput, and energy efficiency. In this demonstration, we present our method for implementing Graph Convolutional Neural Networks on a heterogeneous SoC FPGA platform, aimed at meeting these constraints. We address the challenges associated with integrating event-based sensors with reconfigurable hardware, as well as the intricate relationship between latency, performance, and hardware resource utilisation.

    @inproceedings{badap-agh:166922,
      title = {Live demonstration: Continuous processing of event-data with Graph Convolutional Neural Networks implemented for SoC FPGA},
      author = {Wzorek, Piotr and Jeziorek, Kamil and Kowalczyk, Marcin and Blachut, Krzysztof and Kryjak, Tomasz and Gorgon, Marek},
      year = {2025},
      series = {FPL~2025 [Dokument elektroniczny] : 35th International Conference on Field-Programmable Logic and Applications : 1–5 September~2025 Leiden, Netherlands : proceedings},
      pages = {359},
      doi = {10.1109/FPL68686.2025.00065},
      collection = {FPL~2025 [Dokument elektroniczny] : 35th International Conference on Field-Programmable Logic and Applications : 1–5 September~2025 Leiden, Netherlands : proceedings},
      projects = {Preludium-Wzorek, Preludium-Kowalczyk},
    }

2024

1. FPT

   Increasing the scalability of graph convolution for FPGA-implemented event-based vision

   Piotr Wzorek, Kamil Jeziorek, Tomasz Kryjak, and Andrea Pinna

   In ICFPT 2024 : International Conference on Field Programmable Technology 2024 : Sydney, Australia, 10-12 December 2024 : proceedings, 2024

   Abstract Collaboration DOI arXiv Bib

   • SYEL (Systèmes Électroniques), LIP6 laboratory
     Sorbonne Université, France

   All our collaborations

   Event cameras are becoming increasingly popular as an alternative to traditional frame-based vision sensors, especially in mobile robotics. Taking full advantage of their high temporal resolution, high dynamic range, low power consumption and sparsity of event data, which only reflects changes in the observed scene, requires both an efficient algorithm and a specialised hardware platform. A recent trend involves using Graph Convolutional Neural Networks (GCNNs) implemented on a heterogeneous SoC FPGA. In this paper we focus on optimising hardware modules for graph convolution to allow flexible selection of the FPGA resource (BlockRAM, DSP and LUT) for their implementation. We propose a “two-step convolution” approach that utilises additional BRAM buffers in order to reduce up to 94% of LUT usage for multiplications. This method significantly improves the scalability of GCNNs, enabling the deployment of models with more layers, larger graphs sizes and their application for more dynamic scenarios.

   @inproceedings{badap-agh:162122,
     title = {Increasing the scalability of graph convolution for FPGA-implemented event-based vision},
     author = {Wzorek, Piotr and Jeziorek, Kamil and Kryjak, Tomasz and Pinna, Andrea},
     year = {2024},
     series = {ICFPT~2024 [Dokument elektroniczny] : International Conference on Field Programmable Technology~2024 : Sydney, Australia, 10-12 December~2024 : proceedings},
     pages = {[1-4]},
     doi = {10.1109/ICFPT64416.2024.11113443},
     collection = {ICFPT~2024 [Dokument elektroniczny] : International Conference on Field Programmable Technology~2024 : Sydney, Australia, 10-12 December~2024 : proceedings},
   }

2. SPA

   

   Tangled Program Graphs as an alternative to DRL-based control algorithms for UAVs

   Hubert Szolc, Karol Desnos, and Tomasz Kryjak

   In 2024 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA) , Sep 2024

   Abstract Collaboration DOI arXiv Bib

   • VAADER team, Institute of Electronics and Digital Technologies (IETR)
     National Institute of Applied Sciences (INSA) of Rennes, France

   All our collaborations

   Deep reinforcement learning (DRL) is currently the most popular AI-based approach to autonomous vehicle control. An agent, trained for this purpose in simulation, can interact with the real environment with a human-level performance. Despite very good results in terms of selected metrics, this approach has some significant drawbacks: high computational requirements and low explainability. Because of that, a DRL-based agent cannot be used in some control tasks, especially when safety is the key issue. Therefore we propose to use Tangled Program Graphs (TPGs) as an alternative for deep reinforcement learning in control-related tasks. In this approach, input signals are processed by simple programs that are combined in a graph structure. As a result, TPGs are less computationally demanding and their actions can be explained based on the graph structure. In this paper, we present our studies on the use of TPGs as an alternative for DRL in control-related tasks. In particular, we consider the problem of navigating an unmanned aerial vehicle (UAV) through the unknown environment based solely on the onboard LiDAR sensor. The results of our work show promising prospects for the use of TPGs in control related-tasks.

   @inproceedings{10715635,
     author = {Szolc, Hubert and Desnos, Karol and Kryjak, Tomasz},
     booktitle = {2024 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA)},
     title = {Tangled Program Graphs as an alternative to DRL-based control algorithms for UAVs},
     year = {2024},
     volume = {},
     number = {},
     pages = {36--41},
     keywords = {Measurement;Laser radar;Navigation;Signal processing algorithms;Process control;Computer architecture;Deep reinforcement learning;Autonomous aerial vehicles;Safety;Tangled Program Graphs;TPG;Deep Reinforcement Learning;DRL;control algorithms;unmanned aerial vehicle;UAV},
     doi = {10.23919/SPA61993.2024.10715635},
     issn = {2326-0319},
     month = sep,
     acronym = {SPA}
   }

3. DSD

   Event-based vision on FPGAs – a survey

   Tomasz Kryjak

   In DSD 2024 : 2024 27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings, Sep 2024

   Abstract DOI arXiv Bib

   In recent years there has been a growing interest in event cameras, i.e. vision sensors that record changes in illumination independently for each pixel. This type of operation ensures that acquisition is possible in very adverse lighting conditions, both in low light and high dynamic range, and reduces average power consumption. In addition, the independent operation of each pixel results in low latency, which is desirable for robotic solutions. Nowadays, Field Programmable Gate Arrays (FPGAs), along with general-purpose processors (GPPs/CPUs) and programmable graphics processing units (GPUs), are popular architectures for implementing and accelerating computing tasks. In particular, their usefulness in the embedded vision domain has been repeatedly demonstrated over the past 30 years, where they have enabled fast data processing (even in real-time) and energy efficiency. Hence, the combination of event cameras and reconfigurable devices seems to be a good solution, especially in the context of energy-efficient real-time embedded systems. This paper gives an overview of the most important works, where FPGAs have been used in different contexts to process event data. It covers applications in the following areas: filtering, stereovision, optical flow, acceleration of AI-based algorithms (including spiking neural networks) for object classification, detection and tracking, and applications in robotics and inspection systems. Current trends and challenges for such systems are also discussed.

   @inproceedings{badap-agh:156634,
     title = {Event-based vision on FPGAs – a survey},
     author = {Kryjak, Tomasz},
     year = {2024},
     series = {DSD~2024 [Dokument elektroniczny] : 2024~27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings},
     pages = {541–550},
     doi = {10.1109/DSD64264.2024.00078},
     collection = {DSD~2024 [Dokument elektroniczny] : 2024~27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings},
   }

4. DASIP

   Optimising graph representation for hardware implementation of graph convolutional networks for event-based vision

   Kamil Jeziorek, Piotr Wzorek, Krzysztof Blachut, Andrea Pinna, and Tomasz Kryjak

   In Design and Architectures for Signal and Image Processing : 17th international workshop, DASIP 2024 : Munich, Germany, January 17–19, 2024 : proceedings, Sep 2024

   Abstract Collaboration DOI arXiv Bib

   • SYEL (Systèmes Électroniques), LIP6 laboratory
     Sorbonne Université, France

   All our collaborations

   Event-based vision is an emerging research field involving processing data generated by Dynamic Vision Sensors (neuromorphic cameras). One of the latest proposals in this area are Graph Convolutional Networks (GCNs), which allow to process events in its original sparse form while maintaining high detection and classification performance. In this paper, we present the hardware implementation of a graph generation process from an event camera data stream, taking into account both the advantages and limitations of FPGAs. We propose various ways to simplify the graph representation and use scaling and quantisation of values. We consider both undirected and directed graphs that enable the use of PointNet convolution. The results obtained show that by appropriately modifying the graph representation, it is possible to create a hardware module for graph generation. Moreover, the proposed modifications have no significant impact on object detection performance, only 0.08% mAP less for the base model and the N-Caltech data set.Finally, we describe the proposed hardware architecture of the graph generation module.

   @inproceedings{badap-agh:154027,
     title = {Optimising graph representation for hardware implementation of graph convolutional networks for event-based vision},
     author = {Jeziorek, Kamil and Wzorek, Piotr and Blachut, Krzysztof and Pinna, Andrea and Kryjak, Tomasz},
     year = {2024},
     series = {Design and Architectures for Signal and Image Processing : 17th international workshop, DASIP~2024 : Munich, Germany, January 17–19, 2024 : proceedings},
     pages = {110–122},
     doi = {10.1007/978-3-031-62874-0_9},
     collection = {Design and Architectures for Signal and Image Processing : 17th international workshop, DASIP~2024 : Munich, Germany, January 17–19, 2024 : proceedings},
   }

5. DSD

   PowerYOLO: mixed precision model for hardware efficient object detection with event data

   Dominika Przewlocka-Rus, Tomasz Kryjak, and Marek Gorgon

   In DSD 2024 : 2024 27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings, Sep 2024

   Abstract DOI arXiv Bib

   The performance of object detection systems in automotive solutions must be as high as possible, with minimal response time and, due to the often battery-powered operation, low energy consumption. When designing such solutions, we therefore face challenges typical for embedded vision systems: the problem of fitting algorithms of high memory and computational complexity into small low-power devices. In this paper we propose PowerYOLO - a mixed precision solution, which targets three essential elements of such application. First, we propose a system based on a Dynamic Vision Sensor (DVS), a novel sensor, that offers low power requirements and operates well in conditions with variable illumination. It is these features that may make event cameras a preferential choice over frame cameras in some applications. Second, to ensure high accuracy and low memory and computational complexity, we propose to use 4-bit width Powers-of- Two (PoT) quantisation for convolution weights of the YOLO detector, with all other parameters quantised linearly. Finally, we embrace from PoT scheme and replace multiplication with bit-shifting to increase the efficiency of hardware acceleration of such solution, with a special convolution-batch normalisation fusion scheme. The use of specific sensor with PoT quantisation and special batch normalisation fusion leads to a unique system with almost 8x reduction in memory complexity and vast computational simplifications, with relation to a standard approach. This efficient system achieves high accuracy of mAP 0.301 on the GENt DVS dataset, marking the new state-of-the-art for such compressed model.

   @inproceedings{badap-agh:156633,
     title = {PowerYOLO: mixed precision model for hardware efficient object detection with event data},
     author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz and Gorgon, Marek},
     year = {2024},
     series = {DSD~2024 [Dokument elektroniczny] : 2024~27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings},
     pages = {210–217},
     doi = {10.1109/DSD64264.2024.00036},
     collection = {DSD~2024 [Dokument elektroniczny] : 2024~27th Euromicro Conference on Digital System Design : 28–30 August 2024, Paris, France : proceedings},
   }

2023

1. MMAR

   Comparative study of subset selection methods for rapid prototyping of 3D object detection algorithms

   Konrad Lis and Tomasz Kryjak

   In MMAR 2023 : 27th international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Międzyzdroje, Poland : on line proceedings : technical papers, Sep 2023

   Abstract DOI arXiv Bib

   Object detection in 3D is a crucial aspect in the context of autonomous vehicles and drones. However, prototyping detection algorithms is time-consuming and costly in terms of energy and environmental impact. To address these challenges, one can check the effectiveness of different models by training on a subset of the original training set. In this paper, we present a comparison of three algorithms for selecting such a subset – random sampling, random per class sampling, and our proposed MONSPeC (Maximum Object Number Sampling per Class). We provide empirical evidence for the superior effectiveness of random per class sampling and MONSPeC over basic random sampling. By replacing random sampling with one of the more efficient algorithms, the results obtained on the subset are more likely to transfer to the results on the entire dataset. The code is available at: https://github.com/vision-agh/monspec.

   @inproceedings{badap-agh:148559,
     title = {Comparative study of subset selection methods for rapid prototyping of 3D object detection algorithms},
     author = {Lis, Konrad and Kryjak, Tomasz},
     year = {2023},
     series = {MMAR~2023 [Dokument elektroniczny] : 27th~international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Mi\k{e}dzyzdroje, Poland : on line proceedings : technical papers},
     pages = {344–349},
     doi = {10.1109/MMAR58394.2023.10242454},
     collection = {MMAR~2023 [Dokument elektroniczny] : 27th~international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Mi\k{e}dzyzdroje, Poland : on line proceedings : technical papers},
   }

2. MMAR

   Detection-segmentation convolutional neural network for autonomous vehicle perception

   Maciej Baczmanski, Robert Synoczek, Mateusz Wasala, and Tomasz Kryjak

   In MMAR 2023 : 27th international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Międzyzdroje, Poland : on line proceedings : technical papers, Sep 2023

   Abstract DOI arXiv Bib

   Object detection and segmentation are two core modules of an autonomous vehicle perception system. They should have high efficiency and low latency while reducing computational complexity. Currently, the most commonly used algorithms are based on deep neural networks, which guarantee high efficiency but require high-performance computing platforms. In the case of autonomous vehicles, i.e. cars, but also drones, it is necessary to use embedded platforms with limited computing power, which makes it difficult to meet the requirements described above. A reduction in the complexity of the network can be achieved by using an appropriate: architecture, representation (reduced numerical precision, quantisation, pruning), and computing platform. In this paper, we focus on the first factor – the use of so-called detection-segmentation networks as a component of a perception system. We considered the task of segmenting the drivable area and road markings in combination with the detection of selected objects (pedestrians, traffic lights, and obstacles). We compared the performance of three different architectures described in the literature: MultiTask V3, HybridNets, and YOLOP. We conducted the experiments on a custom dataset consisting of approximately 500 images of the drivable area and lane markings, and 250 images of detected objects. Of the three methods analysed, MultiTask V3 proved to be the best, achieving 99% mAP50for detection, 97% MIoU for drivable area segmentation, and 91% MIoU for lane segmentation, as well as 124 fps on the RTX 3060 graphics card. This architecture is a good solution for embedded perception systems for autonomous vehicles. The code is available at: https://github.com/vision-agh/MMAR_2023.

   @inproceedings{badap-agh:148543,
     title = {Detection-segmentation convolutional neural network for autonomous vehicle perception},
     author = {Baczmanski, Maciej and Synoczek, Robert and Wasala, Mateusz and Kryjak, Tomasz},
     year = {2023},
     series = {MMAR~2023 [Dokument elektroniczny] : 27th~international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Mi\k{e}dzyzdroje, Poland : on line proceedings : technical papers},
     pages = {117–122},
     doi = {10.1109/MMAR58394.2023.10242398},
     collection = {MMAR~2023 [Dokument elektroniczny] : 27th~international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Mi\k{e}dzyzdroje, Poland : on line proceedings : technical papers},
   }

3. ICCVG

   Energy efficient hardware acceleration of neural networks with power-of-two quantisation

   Dominika Przewlocka-Rus and Tomasz Kryjak

   In Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw], Sep 2023

   Abstract DOI arXiv Bib

   Deep neural networks virtually dominate the domain of most modern vision systems, providing high performance at a cost of increased computational complexity.Since for those systems it is often required to operate both in real-time and with minimal energy consumption (e.g., for wearable devices or autonomous vehicles, edge Internet of Things (IoT), sensor networks), various network optimisation techniques are used, e.g., quantisation, pruning, or dedicated lightweight architectures. Due to the logarithmic distribution of weights in neural network layers, a method providing high performance with significant reduction in computational precision (for 4-bit weights and less) is the Power-of-Two (PoT) quantisation (and therefore also with a logarithmic distribution). This method introduces additional possibilities of replacing the typical for neural networks Multiply and ACcumulate (MAC – performing, e.g., convolution operations) units, with more energy-efficient Bitshift and ACcumulate (BAC). In this paper, we show that a hardware neural network accelerator with PoT weights implemented on the Zynq UltraScale + MPSoC ZCU104 SoC FPGA can be at least 1.4x more energy efficient than the uniform quantisation version. To further reduce the actual power requirement by omitting part of the computation for zero weights, we also propose a new pruning method adapted to logarithmic quantisation.

   @inproceedings{badap-agh:145407,
     title = {Energy efficient hardware acceleration of neural networks with power-of-two quantisation},
     author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz},
     year = {2023},
     series = {Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw]},
     pages = {225–236},
     doi = {10.1007/978-3-031-22025-8_16},
     collection = {Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw]},
   }

4. SPA

   High-definition event frame generation using SoC FPGA devices

   Krzysztof Blachut and Tomasz Kryjak

   In SPA 2023 : Signal Processing Algorithms, Architectures, Arrangements, and Applications : Poznan, 20th - 22nd September 2023, Sep 2023

   Abstract DOI arXiv Bib

   In this paper we have addressed the implementation of the accumulation and projection of high-resolution event data stream (HD – 1280×720 pixels) onto the image plane in FPGA devices. The results confirm the feasibility of this approach, but there are a number of challenges, limitations and trade-offs to be considered. The required hardware resources of selected data representations, such as binary frame, event frame, exponentially decaying time surface and event frequency, were compared with those available on several popular platforms from AMD Xilinx. The resulting event frames can be used for typical vision algorithms, such as object classification and detection, using both classical and deep neural network methods.

   @inproceedings{badap-agh:150127,
     title = {High-definition event frame generation using SoC FPGA devices},
     author = {Blachut, Krzysztof and Kryjak, Tomasz},
     year = {2023},
     series = {SPA~2023 : Signal Processing Algorithms, Architectures, Arrangements, and Applications : Poznan, 20th - 22nd September 2023},
     pages = {106–111},
     doi = {10.23919/SPA59660.2023.10274447},
     collection = {SPA~2023 : Signal Processing Algorithms, Architectures, Arrangements, and Applications : Poznan, 20th - 22nd September 2023},
   }

5. ARC

   Implementation of a perception system for autonomous vehicles using a detection-segmentation network in SoC FPGA

   Maciej Baczmanski, Mateusz Wasala, and Tomasz Kryjak

   In Applied Reconfigurable Computing : architectures, tools, and applications : 19th international symposium, ARC 2023 : Cottbus, Germany, September 27-29, 2023 : proceedings, Sep 2023

   Abstract DOI arXiv Bib

   Perception and control systems for autonomous vehicles are an active area of scientific and industrial research. These solutions should be characterised by high efficiency in recognising obstacles and other environmental elements in different road conditions, real-time capability, and energy efficiency. Achieving such functionality requires an appropriate algorithm and a suitable computing platform. In this paper, we have used the MultiTaskV3 detection-segmentation network as the basis for a perception system that can perform both functionalities within a single architecture. It was appropriately trained, quantised, and implemented on the AMD Xilinx Kria KV260 Vision AI embedded platform. By using this device, it was possible to parallelise and accelerate the computations. Furthermore, the whole system consumes relatively little power compared to a CPU-based implementation (an average of 5 watts, compared to the minimum of 55 watts for weaker CPUs, and the small size (119mm x 140mm x 36mm) of the platform allows it to be used in devices where the amount of space available is limited. It also achieves an accuracy higher than 97% of the mAP (mean average precision) for object detection and above 90% of the mIoU (mean intersection over union) for image segmentation. The article also details the design of the Mecanum wheel vehicle, which was used to test the proposed solution in a mock-up city.

   @inproceedings{badap-agh:149007,
     title = {Implementation of a perception system for autonomous vehicles using a detection-segmentation network in SoC FPGA},
     author = {Baczmanski, Maciej and Wasala, Mateusz and Kryjak, Tomasz},
     year = {2023},
     series = {Applied Reconfigurable Computing : architectures, tools, and applications : 19th~international symposium, ARC~2023 : Cottbus, Germany, September 27-29, 2023 : proceedings},
     pages = {200–211},
     doi = {10.1007/978-3-031-42921-7_14},
     collection = {Applied Reconfigurable Computing : architectures, tools, and applications : 19th~international symposium, ARC~2023 : Cottbus, Germany, September 27-29, 2023 : proceedings},
   }

6. CVPRW Preludium
   

   Interpolation-based event visual data filtering algorithms

   Marcin Kowalczyk and Tomasz Kryjak

   In CVPRW 2023 : 2023 IEEE/CVF conference on Computer Vision and Pattern Recognition Workshops : Vancouver, Canada 18-22 June 2023 : proceedings, Sep 2023

   Abstract DOI arXiv Bib

   The field of neuromorphic vision is developing rapidly, and event cameras are finding their way into more and more applications. However, the data stream from these sensors is characterised by significant noise. In this paper, we propose a method for event data that is capable of removing approximately 99% of noise while preserving the majority of the valid signal. We have proposed four algorithms based on the matrix of infinite impulse response (IIR) filters method. We compared them on several event datasets that were further modified by adding artificially generated noise and noise recorded with dynamic vision sensor. The proposed methods use about 30KB of memory for a sensor with a resolution of 1280 × 720 and is therefore well suited for implementation in embedded devices.

   @inproceedings{badap-agh:148668,
     title = {Interpolation-based event visual data filtering algorithms},
     author = {Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2023},
     projects = {Preludium-Kowalczyk},
     series = {CVPRW~2023 [Dokument elektroniczny] : 2023 IEEE/CVF conference on Computer Vision and Pattern Recognition Workshops : Vancouver, Canada 18-22 June 2023 : proceedings},
     pages = {4056–4064},
     doi = {10.1109/CVPRW59228.2023.00425},
     collection = {CVPRW~2023 [Dokument elektroniczny] : 2023 IEEE/CVF conference on Computer Vision and Pattern Recognition Workshops : Vancouver, Canada 18-22 June 2023 : proceedings},
   }

7. SPA

   Memory-efficient graph convolutional networks for object classification and detection with event cameras

   Kamil Jeziorek, Andrea Pinna, and Tomasz Kryjak

   In SPA 2023 : Signal Processing Algorithms, Architectures, Arrangements, and Applications : Poznan, 20th - 22nd September 2023, Sep 2023

   Abstract Collaboration DOI arXiv Bib

   • SYEL (Systèmes Électroniques), LIP6 laboratory
     Sorbonne Université, France

   All our collaborations

   Recent advances in event camera research emphasize processing data in its original sparse form, which allows the use of its unique features such as high temporal resolution, high dynamic range, low latency, and resistance to image blur. One promising approach for analyzing event data is through graph convolutional networks (GCNs). However, current research in this domain primarily focuses on optimizing computational costs, neglecting the associated memory costs. In this paper, we consider both factors together in order to achieve satisfying results and relatively low model complexity. For this purpose, we performed a comparative analysis of different graph convolution operations, considering factors such as execution time, the number of trainable model parameters, data format requirements, and training outcomes. Our results show a 450-fold reduction in the number of parameters for the feature extraction module and a 4.5-fold reduction in the size of the data representation while maintaining a classification accuracy of 52.3%, which is 6.3% higher compared to the operation used in state-of-the-art approaches. To further evaluate performance, we implemented the object detection architecture and evaluated its performance on the N-Caltech101 dataset. The results showed an accuracy of 53.7% mAP@0.5 and reached an execution rate of 82 graphs per second.

   @inproceedings{badap-agh:150128,
     title = {Memory-efficient graph convolutional networks for object classification and detection with event cameras},
     author = {Jeziorek, Kamil and Pinna, Andrea and Kryjak, Tomasz},
     year = {2023},
     series = {SPA~2023 : Signal Processing Algorithms, Architectures, Arrangements, and Applications : Poznan, 20th - 22nd September 2023},
     pages = {160–165},
     doi = {10.23919/SPA59660.2023.10274464},
     collection = {SPA~2023 : Signal Processing Algorithms, Architectures, Arrangements, and Applications : Poznan, 20th - 22nd September 2023},
   }

8. PP-RAI

   Pedestrian detection with high-resolution event camera

   Piotr Wzorek and Tomasz Kryjak

   In Progress in Polish artificial intelligence research 4, Sep 2023

   Abstract DOI arXiv Bib

   Despite the dynamic development of computer vision algorithms, the implementation of perception and control systems for autonomous vehicles such as drones and self-driving cars still poses many challenges. A video stream captured by traditional cameras is often prone to problems such as motion blur or degraded image quality due to challenging lighting conditions. In addition, the frame rate - typically 30 or 60 frames per second - can be a limiting factor in certain scenarios. Event cameras (DVS – Dynamic Vision Sensor) are a potentially interesting technology to address the above mentioned problems. In this paper, we compare two methods of processing event data by means of deep learning for the task of pedestrian detection. We used a representation in the form of video frames, convolutional neural networks and asynchronous sparse convolutional neural networks. The results obtained illustrate the potential of event cameras and allow the evaluation of the accuracy and efficiency of the methods used for high-resolution (1280 x 720 pixels) footage.

   @inbook{badap-agh:149063,
     title = {Pedestrian detection with high-resolution event camera},
     author = {Wzorek, Piotr and Kryjak, Tomasz},
     year = {2023},
     series = {Progress in Polish artificial intelligence research 4 [Dokument elektroniczny]},
     pages = {55–60},
     doi = {10.34658/9788366741928.7},
     collection = {Progress in Polish artificial intelligence research 4 [Dokument elektroniczny]},
   }

9. ICCVG

   PointPillars backbone type selection for fast and accurate LiDAR object detection

   Konrad Lis and Tomasz Kryjak

   In Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw], Sep 2023

   Abstract DOI arXiv Bib

   3D object detection from LiDAR sensor data is an important topic in the context of autonomous cars and drones. In this paper, we present the results of experiments on the impact of backbone selection of a deep convolutional neural network on detection accuracy and computation speed. We chose the PointPillars network, which is characterised by a simple architecture, high speed, and modularity that allows for easy expansion. During the experiments, we paid particular attention to the change in detection efficiency (measured by the mAP metric) and the total number of multiply-addition operations needed to process one point cloud. We tested 10 different convolutional neural network architectures that are widely used in image-based detection problems. For a backbone like MobilenetV1, we obtained an almost 4x speedup at the cost of a 1.13% decrease in mAP. On the other hand, for CSPDarknet we got an acceleration of more than 1.5x at an increase in mAP of 0.33%. We have thus demonstrated that it is possible to significantly speed up a 3D object detector in LiDAR point clouds with a small decrease in detection efficiency. This result can be used when PointPillars or similar algorithms are implemented in embedded systems, including SoC FPGAs. The code is available at https://github.com/vision-agh/pointpillars_backbone.

   @inproceedings{badap-agh:145406,
     title = {PointPillars backbone type selection for fast and accurate LiDAR object detection},
     author = {Lis, Konrad and Kryjak, Tomasz},
     year = {2023},
     series = {Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw]},
     pages = {99–119},
     doi = {10.1007/978-3-031-22025-8_8},
     collection = {Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw]},
   }

10. DSD

    Power-of-two quantized YOLO network for pedestrian detection with dynamic vision sensor

    Dominika Przewlocka-Rus and Tomasz Kryjak

    In DSD 2023 : 2023 26th euromicro conference on Digital System Design : 6–8 September 2023, Durres, Albania : proceedings, Sep 2023

    Abstract DOI Bib

    Pedestrian detection algorithms have a wide range of applications: from video surveillance, to driver assistance systems and autonomous vehicles. The performance of these systems must be as high as possible, with minimal response time and, due to the often battery-powered operation (like in electric vehicles), low energy consumption. When designing such solutions, we therefore face challenges typical for embedded vision systems: the problem of fitting algorithms of high memory and computational complexity into small low-power devices. In this paper, we propose a system based on a Dynamic Vision Sensor (DVS), which has low power requirements and operates well in conditions with variable illumination. It is these features that may make event cameras a preferential choice over frame cameras in some applications. To ensure high accuracy, for pedestrian detection we use the YOLO (You Only Look Once) deep network on event data representation. Due to the high complexity of the applied algorithm, we propose a low precision architecture: the weights of the convolution layers are quantized logarithmically to 4 bits powers-of-two values (PoT quantization). Such compression reduces not only the memory complexity by almost 8x, but also the computational complexity by replacing most multiplication operations with bit-shifting, due to use of powers-of-two weights. At the same time, the proposed system achieves the accuracy on par with the floating point baseline, of mAP0.5 0.708.

    @inproceedings{badap-agh:152559,
      title = {Power-of-two quantized YOLO network for pedestrian detection with dynamic vision sensor},
      author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz},
      year = {2023},
      series = {DSD~2023 [Dokument elektroniczny] : 2023~26th euromicro conference on Digital System Design : 6–8 September 2023, Durres, Albania : proceedings},
      pages = {39–45},
      doi = {10.1109/DSD60849.2023.00016},
      collection = {DSD~2023 [Dokument elektroniczny] : 2023~26th euromicro conference on Digital System Design : 6–8 September 2023, Durres, Albania : proceedings},
    }

11. DASIP Sonata
    

    Real-time FPGA implementation of the semi-global matching stereo vision algorithm for a 4K/UHD video stream

    Mariusz Grabowski and Tomasz Kryjak

    In Design and Architecture for Signal and Image Processing : 16th international workshop, DASIP 2023 : Toulouse, France, January 16–18, 2023 : proceedings, Sep 2023

    Abstract DOI arXiv Bib

    In this paper, we propose a real-time FPGA implementation of the Semi-Global Matching (SGM) stereo vision algorithm. The designed module supports a 4K/Ultra HD (3840 x 2160 pixels @ 30 frames per second) video stream in a 4 pixel per clock (ppc) format and a 64-pixel disparity range. The baseline SGM implementation had to be modified to process pixels in the 4ppc format and meet the timing constrains, however, our version provides results comparable to the original design. The solution has been positively evaluated on the Xilinx VC707 development board with a Virtex-7 FPGA device.

    @inproceedings{badap-agh:146665,
      title = {Real-time FPGA implementation of the semi-global matching stereo vision algorithm for a 4K/UHD video stream},
      projects = {Sonata},
      author = {Grabowski, Mariusz and Kryjak, Tomasz},
      year = {2023},
      series = {Design and Architecture for Signal and Image Processing : 16th~international workshop, DASIP~2023 : Toulouse, France, January 16–18, 2023 : proceedings},
      pages = {70–81},
      doi = {10.1007/978-3-031-29970-4_6},
      collection = {Design and Architecture for Signal and Image Processing : 16th~international workshop, DASIP~2023 : Toulouse, France, January 16–18, 2023 : proceedings},
    }

12. PAR

    Sterowanie autonomicznym bezzalogowym statkiem powietrznym z wykorzystaniem uczenia przez wzmacnianie

    Pawel Miera, Hubert Szolc, and Tomasz Kryjak

    PAR Pomiary Automatyka Robotyka, Sep 2023

    Abstract DOI arXiv Bib

    Reinforcement learning is of increasing importance in the field of robot control and simulation plays a key role in this process. In the unmanned aerial vehicles (UAVs, drones), there is also an increase in the number of published scientific papers involving this approach. In this work, an autonomous drone control system was prepared to fly forward (according to its coordinates system) and pass the trees encountered in the forest based on the data from a rotating LiDAR sensor. The Proximal Policy Optimization (PPO) algorithm, an example of reinforcement learning (RL), was used to prepare it. A custom simulator in the Python language was developed for this purpose. The Gazebo environment, integrated with the Robot Operating System (ROS), was also used to test the resulting control algorithm. Finally, the prepared solution was implemented in the Nvidia Jetson Nano eGPU and verified in the real tests scenarios. During them, the drone successfully completed the set task and was able to repeatable avoid trees and fly through the forest.

    @article{badap-agh:151231,
      title = {Sterowanie autonomicznym bezzalogowym statkiem powietrznym z wykorzystaniem uczenia przez wzmacnianie},
      author = {Miera, Pawel and Szolc, Hubert and Kryjak, Tomasz},
      year = {2023},
      journal = {PAR Pomiary Automatyka Robotyka},
      series = {PAR Pomiary Automatyka Robotyka},
      volume = {27},
      pages = {85},
      doi = {10.14313/PAR_250/85},
      issn = {1427-9126},
      collection = {PAR Pomiary Automatyka Robotyka},
    }

13. ICCVG

    Traffic sign classification using deep and quantum neural networks

    Sylwia Kuros and Tomasz Kryjak

    In Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw], Sep 2023

    Abstract DOI arXiv Bib

    Quantum Neural Networks (QNNs) are an emerging technology that can be used in many applications including computer vision. In this paper, we presented a traffic sign classification system implemented using a hybrid quantum-classical convolutional neural network. Experiments on the German Traffic Sign Recognition Benchmark dataset indicate that currently QNN do not outperform classical DCNN (Deep Convolutuional Neural Networks), yet still provide an accuracy of over 90% and are a definitely promising solution for advanced computer vision.

    @inproceedings{badap-agh:145405,
      title = {Traffic sign classification using deep and quantum neural networks},
      author = {Kuros, Sylwia and Kryjak, Tomasz},
      year = {2023},
      series = {Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw]},
      pages = {43–55},
      doi = {10.1007/978-3-031-22025-8_4},
      collection = {Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw]},
    }

14. PAR Preludium
    

    Zliczanie szybkich obiektów z wykorzystaniem kamery zdarzeniowej

    Kamil Bialik, Marcin Kowalczyk, Krzysztof Blachut, and Tomasz Kryjak

    PAR Pomiary Automatyka Robotyka, Sep 2023

    Abstract DOI arXiv Bib

    This paper proposes the use of an event camera as a component of a vision system that enables counting of fast-moving objects – in this case, falling corn grains. These cameras transmit, in an asynchronous manner, information about the change in brightness of individual pixels and are characterised by low latency, no motion blur, and correct operation in different lighting conditions, as well as very low power consumption. The proposed counting algorithm processes events in real time. The operation of the solution was demonstrated on a stand consisting of a chute with a vibrating feeder, which allowed the number of falling grains to be adjusted. The objective of the control system with a PID controller was to maintain a constant average number of falling objects. The proposed solution was subjected to a series of tests to determine the correctness of the developed method operation. On their basis, the validity of using an event camera to count small, fast-moving objects and the associated wide range of potential industrial applications can be confirmed.

    @article{badap-agh:145827,
      title = {Zliczanie szybkich obiekt\'{o}w z wykorzystaniem kamery zdarzeniowej},
      author = {Bialik, Kamil and Kowalczyk, Marcin and Blachut, Krzysztof and Kryjak, Tomasz},
      year = {2023},
      projects = {Preludium-Kowalczyk},
      journal = {PAR Pomiary Automatyka Robotyka},
      series = {PAR Pomiary Automatyka Robotyka},
      volume = {27},
      number = {1},
      pages = {79},
      doi = {10.14313/PAR_247/79},
      issn = {1427-9126},
      collection = {PAR Pomiary Automatyka Robotyka},
    }

2022

1. KU KDM

   Accelerated training of object detection DNNs for embedded vision systems

   Michal Machura, Piotr Wzorek, Tomasz Kryjak, and Marek Gorgon

   In KU KDM 2022 : fourteenth ACC Cyfronet AGH HPC users’ conference : online, 7-8 April 2022 : proceedings, Sep 2022

   Bib

   @inproceedings{badap-agh:142206,
     title = {Accelerated training of object detection DNNs for embedded vision systems},
     author = {Machura, Michal and Wzorek, Piotr and Kryjak, Tomasz and Gorgon, Marek},
     year = {2022},
     series = {KU KDM~2022 : fourteenth ACC Cyfronet AGH HPC users' conference : online, 7-8 April 2022 : proceedings},
     pages = {55–56},
     url = {http://katalog.nukat.edu.pl/lib/item?id=chamo:5338936&fromLocationLink=false&theme=nukat},
     collection = {KU KDM~2022 : fourteenth ACC Cyfronet AGH HPC users' conference : online, 7-8 April 2022 : proceedings},
   }

2. Electronics

   An efficient real-time FPGA-based ORB feature extraction for an UHD video stream for embedded visual SLAM

   Mateusz Wasala, Hubert Szolc, and Tomasz Kryjak

   Electronics, Sep 2022

   Abstract DOI Bib

   The detection and description of feature points are important components of many computer vision systems. For example, in the field of autonomous unmanned aerial vehicles (UAV), these methods form the basis of so-called Visual Odometry (VO) and Simultaneous Localisation and Mapping (SLAM) algorithms. In this paper, we present a hardware feature points detection system able to process a 4K video stream in real-time. We use the ORB algorithm—Oriented FAST (Features from Accelerated Segment Test) and Rotated BRIEF (Binary Robust Independent Elementary Features)—to detect and describe feature points in the images. We make numerous modifications to the original ORB algorithm (among others, we use the RS-BRIEF instead of classic R-BRIEF) to adapt it to the high video resolution, make it computationally efficient, reduce the resource utilisation and achieve lower power consumption. Our hardware implementation supports a 4 ppc (pixels per clock) format (with simple adaptation to 2 ppc, 8 ppc, and more) and real-time processing of a 4K video stream (UHD—Ultra High Definition, 3840×2160 pixels) @ 60 frames per second (150 MHz clock). We verify our system using simulations in the Vivado IDE and implement it in hardware on the ZCU 104 evaluation board with the AMD Xilinx Zynq UltraScale+ MPSoC device. The proposed design consumes only 5 watts.

   @article{badap-agh:141501,
     title = {An efficient real-time FPGA-based ORB feature extraction for an UHD video stream for embedded visual SLAM},
     author = {Wasala, Mateusz and Szolc, Hubert and Kryjak, Tomasz},
     year = {2022},
     journal = {Electronics},
     series = {Electronics},
     volume = {11},
     number = {14},
     pages = {2259},
     doi = {10.3390/electronics11142259},
     issn = {2079-9292},
     collection = {Electronics},
   }

3. JSPS

   Automotive perception system evaluation with reference data from a UAV’s camera using ArUco markers and DCNN

   Krzysztof Blachut, Michal Danilowicz, Hubert Szolc, Mateusz Wasala, Tomasz Kryjak, and 1 more author

   Journal of Signal Processing Systems for Signal, Image, and Video Technology, Sep 2022

   Abstract DOI Bib

   Testing and evaluation of an automotive perception system is a complicated task which requires special equipment and infrastructure. To compute key performance indicators and compare the results with real-world situation, some additional sensors and manual data labelling are often required. In this article, we propose a different approach, which is based on a UAV equipped with a 4K camera flying above a test track. Two computer vision methods are used to precisely determine the positions of the objects around the car – one based on ArUco markers and the other on a DCNN (we provide the algorithms used on GitHub). The detections are then correlated with the perception system readings. For the static and dynamic experiments, the differences between various systems are mostly below 0.5 m. The results of the experiments performed indicate that this approach could be an interesting alternative to existing evaluation solutions.

   @article{badap-agh:141200,
     title = {Automotive perception system evaluation with reference data from a UAV's camera using ArUco markers and DCNN},
     author = {Blachut, Krzysztof and Danilowicz, Michal and Szolc, Hubert and Wasala, Mateusz and Kryjak, Tomasz and Komorkiewicz, Mateusz},
     year = {2022},
     journal = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
     series = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
     volume = {94},
     number = {7},
     pages = {675},
     doi = {10.1007/s11265-021-01734-3},
     issn = {1939-8018},
     collection = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
   }

4. JLPEA

   Embedded object detection with custom LittleNet, FINN and vitis AI DCNN accelerators

   Michal Machura, Michal Danilowicz, and Tomasz Kryjak

   Journal of Low Power Electronics and Applications, Sep 2022

   Abstract DOI Bib

   Object detection is an essential component of many systems used, for example, in advanced driver assistance systems (ADAS) or advanced video surveillance systems (AVSS). Currently, the highest detection accuracy is achieved by solutions using deep convolutional neural networks (DCNN). Unfortunately, these come at the cost of a high computational complexity; hence, the work on the widely understood acceleration of these algorithms is very important and timely. In this work, we compare three different DCNN hardware accelerator implementation methods: coarse-grained (a custom accelerator called LittleNet), fine-grained (FINN) and sequential (Vitis AI). We evaluate the approaches in terms of object detection accuracy, throughput and energy usage on the VOT and VTB datasets. We also present the limitations of each of the methods considered. We describe the whole process of DNNs implementation, including architecture design, training, quantisation and hardware implementation. We used two custom DNN architectures to obtain a higher accuracy, higher throughput and lower energy consumption. The first was implemented in SystemVerilog and the second with the FINN tool from AMD Xilinx. Next, both approaches were compared with the Vitis AI tool from AMD Xilinx. The final implementations were tested on the Avnet Ultra96-V2 development board with the Zynq UltraScale+ MPSoC ZCU3EG device. For two different DNNs architectures, we achieved a throughput of 196 fps for our custom accelerator and 111 fps for FINN. The same networks implemented with Vitis AI achieved 123.3 fps and 53.3 fps, respectively.

   @article{badap-agh:140225,
     title = {Embedded object detection with custom LittleNet, FINN and vitis AI DCNN accelerators},
     author = {Machura, Michal and Danilowicz, Michal and Kryjak, Tomasz},
     year = {2022},
     journal = {Journal of Low Power Electronics and Applications},
     series = {Journal of Low Power Electronics and Applications},
     volume = {12},
     number = {2},
     pages = {30},
     doi = {10.3390/jlpea12020030},
     issn = {2079-9268},
     collection = {Journal of Low Power Electronics and Applications},
   }

5. Conference Preludium
   

   Exploration of hardware acceleration methods for an XNOR traffic signs classifier

   Dominika Przewlocka-Rus, Marcin Kowalczyk, and Tomasz Kryjak

   In Progress in image processing, pattern recognition and communication systems : proceedings of the conference (CORES, IP&C, ACS) – June 28–30 2021, Sep 2022

   Abstract DOI arXiv Bib

   Deep learning algorithms are a key component of many state-of-the-art vision systems, especially as Convolutional Neural Networks (CNN) outperform most solutions in the sense of accuracy. To apply such algorithms in real-time applications, one has to address the challenges of memory and computational complexity. To deal with the first issue, we use networks with reduced precision, specifically a binary neural network (also known as XNOR). To satisfy the computational requirements, we propose to use highly parallel and low-power FPGA devices. In this work, we explore the possibility of accelerating XNOR networks for traffic sign classification. The trained binary networks are implemented on the ZCU 104 development board, equipped with a Zynq UltraScale+ MPSoC device using two different approaches. Firstly, we propose a custom HDL accelerator for XNOR networks, which enables the inference with almost 450 fps. Even better results are obtained with the second method - the Xilinx FINN accelerator - enabling to process input images with around 550 frame rate. Both approaches provide over 96% accuracy on the test set.

   @inproceedings{badap-agh:136486,
     title = {Exploration of hardware acceleration methods for an XNOR traffic signs classifier},
     author = {Przewlocka-Rus, Dominika and Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2022},
     projects = {Preludium-Kowalczyk},
     series = {Progress in image processing, pattern recognition and communication systems : proceedings of the conference (CORES, IP\&C, ACS) – June 28–30 2021},
     pages = {34–45},
     doi = {10.1007/978-3-030-81523-3_4},
     collection = {Progress in image processing, pattern recognition and communication systems : proceedings of the conference (CORES, IP\&C, ACS) – June 28–30 2021},
   }

6. JSPS

   Guest Editorial: special issue on Design and Architectures for Signal and Image Processing 2021

   Tomasz Kryjak and Andrea Pinna

   Journal of Signal Processing Systems for Signal, Image, and Video Technology, Sep 2022

   DOI Bib

   @article{badap-agh:141199,
     title = {Guest Editorial: special issue on Design and Architectures for Signal and Image Processing 2021},
     author = {Kryjak, Tomasz and Pinna, Andrea},
     year = {2022},
     journal = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
     series = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
     volume = {94},
     pages = {621},
     doi = {10.1007/s11265-022-01789-w},
     issn = {1939-8018},
     collection = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
   }

7. DSD Preludium
   

   Hardware architecture for high throughput event visual data filtering with matrix of IIR filters algorithm

   Marcin Kowalczyk and Tomasz Kryjak

   In DSD 2022 : 25th Euromicro Conference on Digital System Design : 31 August – 2 September 2022, Maspalomas, Spain : proceedings, Sep 2022

   Abstract DOI arXiv Bib

   Neuromorphic vision is a rapidly growing field with numerous applications in the perception systems of autonomous vehicles. Unfortunately, due to the sensors working principle, there is a significant amount of noise in the event stream. In this paper we present a novel algorithm based on an IIR filter matrix for filtering this type of noise and a hardware architecture that allows its acceleration using an SoC FPGA. Our method has a very good filtering efficiency for uncorrelated noise - over 99% of noisy events are removed. It has been tested for several event data sets with added random noise. We designed the hardware architecture in such a way as to reduce the utilisation of the FPGA’s internal BRAM resources. This enabled a very low latency and a throughput of up to 385.8 MEPS million events per second. The proposed hardware architecture was verified in simulation and in hardware on the Xilinx Zynq Ultrascale+ MPSoC chip on the Mercury+ XU9 module with the Mercury+ ST1 base board.

   @inproceedings{badap-agh:145044,
     title = {Hardware architecture for high throughput event visual data filtering with matrix of IIR filters algorithm},
     author = {Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2022},
     projects = {Preludium-Kowalczyk},
     series = {DSD~2022 [Dokument elektroniczny] : 25th~Euromicro Conference on Digital System Design : 31 August – 2 September 2022, Maspalomas, Spain : proceedings},
     pages = {284–291},
     doi = {10.1109/DSD57027.2022.00046},
     collection = {DSD~2022 [Dokument elektroniczny] : 25th~Euromicro Conference on Digital System Design : 31 August – 2 September 2022, Maspalomas, Spain : proceedings},
   }

8. SPA

   Hardware-in-the-loop simulation of a UAV autonomous landing algorithm implemented in SoC FPGA

   Hubert Szolc and Tomasz Kryjak

   In SPA 2022 : Signal Processing Algorithms, Architectures, Arrangements, and Applications : 25th IEEE SPA conference : Poznan, 21st - 22nd September 2022 : conference proceedings, Sep 2022

   Abstract DOI arXiv Bib

   This paper presents a system for hardware-in-the-loop (HiL) simulation of unmanned aerial vehicle (UAV) control algorithms implemented on a heterogeneous SoC FPGA computing platforms. The AirSim simulator running on a PC and an Arty Z7 development board with a Zynq SoC chip from AMD Xilinx were used. Communication was carried out via a serial USB link. An application for autonomous landing on a specially marked landing strip was selected as a case study. A landing site detection algorithm was implemented on the Zynq SoC platform. This allowed processing a 1280 × 720 @ 60 fps video stream in real time. Performed tests showed that the system works correctly and there are no delays that could negatively affect the stability of the control. The proposed concept is characterised by relative simplicity and low implementation cost. At the same time, it can be applied to test various types of high-level perception and control algorithms for UAV implemented on embedded platforms. We provide the code developed on GitHub, which includes both Python scripts running on the PC and C code running on Arty Z7.

   @inproceedings{badap-agh:143873,
     title = {Hardware-in-the-loop simulation of a UAV autonomous landing algorithm implemented in SoC FPGA},
     author = {Szolc, Hubert and Kryjak, Tomasz},
     year = {2022},
     series = {SPA~2022 [Dokument elektroniczny] : Signal Processing Algorithms, Architectures, Arrangements, and Applications : 25th IEEE SPA conference : Poznan, 21st - 22nd September 2022 : conference proceedings},
     pages = {135–140},
     doi = {10.23919/SPA53010.2022.9927847},
     collection = {SPA~2022 [Dokument elektroniczny] : Signal Processing Algorithms, Architectures, Arrangements, and Applications : 25th IEEE SPA conference : Poznan, 21st - 22nd September 2022 : conference proceedings},
   }

9. Elektronika

   Najmłodszy oddział IEEE w Polsce - co można zdziałać w 3 lata?

   Marek Dlugosz, Jacek Izydorczyk, Adrian Kliks, Mateusz Komorkiewicz, Tomasz Kryjak, and 2 more authors

   Elektronika : Konstrukcje, Technologie, Zastosowania, Sep 2022

   Abstract DOI Bib

   The work presents a short history of the Polish Chapter of the Vehicular Technology Society (VTS), which is currently the youngest IEEE chapter in Poland. The VTS chapter began its operations on January 7, 2020 after collecting the appropriate number of signatures and obtaining the required approvals. The purpose of establishing the new chapter is to promote Poland not only as a place where factories of cars or automotive components are located, but also as a country of highly qualified scientists and engineers involved in the automotive research and development. The chapter enables its members to develop and improve their skills after graduation, transfer knowledge and experience between older and younger generations of engineers by organizing workshops and seminars. In addition, the organization should be a partner for government organizations in giving opinions on standards and regulations related to the automotive sector. The paper summarizes the period of the chapter’s activity over the last 3 years, presents key achievements and events, as well as refers to what has not been fully implemented.

   @article{badap-agh:144092,
     title = {Najm\l{}odszy oddzia\l{} IEEE w Polsce - co mo\.{z}na zdzia\l{}a\'{c} w 3 lata?},
     author = {Dlugosz, Marek and Izydorczyk, Jacek and Kliks, Adrian and Komorkiewicz, Mateusz and Kryjak, Tomasz and Skruch, Pawel and Szelest, Marcin},
     year = {2022},
     journal = {Elektronika : Konstrukcje, Technologie, Zastosowania},
     series = {Elektronika : Konstrukcje, Technologie, Zastosowania},
     volume = {63},
     pages = {70},
     doi = {10.15199/13.2022.11.12},
     issn = {0033-2089},
     collection = {Elektronika : Konstrukcje, Technologie, Zastosowania},
   }

10. PP-RAI

    Playing cards and bidding calls detection for automatic registration of a duplicate bridge game

    Piotr Wzorek and Tomasz Kryjak

    In PP-RAI’2022 : proceedings of the 3rd Polish conference on Artificial intelligence : April 25–27, 2022, Gdynia, Poland, Sep 2022

    Abstract Bib

    In this work, the implementation of a playing cards and bidding calls detection system for the automatic registration of a duplicate bridge game is presented. For this purpose, two YOLOv4 deep convolutional neural networks were used. During the training process, which was carried out using an automatically generated dataset, we obtained a detector characterised by more than 99.9% accuracy on the test set. The prepared system has been evaluated on videos collected during bridge competitions and is characterised by high accuracy. The application has been implemented and evaluated on a CPU and an embedded GPU

    @inproceedings{badap-agh:139996,
      title = {Playing cards and bidding calls detection for automatic registration of a duplicate bridge game},
      author = {Wzorek, Piotr and Kryjak, Tomasz},
      year = {2022},
      series = {PP-RAI'2022 [Dokument elektroniczny] : proceedings of the 3rd Polish conference on Artificial intelligence : April 25–27, 2022, Gdynia, Poland},
      pages = {41–44},
      url = {https://wydawnictwo.umg.edu.pl/pp-rai2022/pdfs/ProceedingsPP-RAI2022.pdf},
      collection = {PP-RAI'2022 [Dokument elektroniczny] : proceedings of the 3rd Polish conference on Artificial intelligence : April 25–27, 2022, Gdynia, Poland},
    }

11. Electronics Sonata
    

    Real-time CLAHE algorithm implementation in SoC FPGA device for 4K UHD video stream

    Tomasz Kryjak, Krzysztof Blachut, Hubert Szolc, and Mateusz Wasala

    Electronics, Sep 2022

    Abstract DOI Bib

    One of the problems encountered in the field of computer vision and video data analysis is the extraction of information from low-contrast images. This problem can be addressed in several ways, including the use of histogram equalisation algorithms. In this work, a method designed for this purpose—the Contrast-Limited Adaptive Histogram Equalization (CLAHE) algorithm—is implemented in hardware. An FPGA platform is used for this purpose due to the ability to run parallel computations and very low power consumption. To enable the processing of a 4K resolution (UHD, 3840 × 2160 pixels) video stream at 60 fps (frames per second) by using the CLAHE method, it is necessary to use a vector data format and process multiple pixels simultaneously. The algorithm realised in this work can be a component of a larger vision system, such as in autonomous vehicles or drones, but it can also support the analysis of underwater, thermal, or medical images both by humans and in an automated system.

    @article{badap-agh:141231,
      title = {Real-time CLAHE algorithm implementation in SoC FPGA device for 4K UHD video stream},
      projects = {Sonata},
      author = {Kryjak, Tomasz and Blachut, Krzysztof and Szolc, Hubert and Wasala, Mateusz},
      year = {2022},
      journal = {Electronics},
      series = {Electronics},
      volume = {11},
      number = {14},
      pages = {2248},
      doi = {10.3390/electronics11142248},
      issn = {2079-9292},
      collection = {Electronics},
    }

12. MECO Sonata
    

    Real-time HOG+SVM based object detection using SoC FPGA for a UHD video stream

    Mateusz Wasala and Tomasz Kryjak

    In 11th Mediterranean Conference on Embedded Computing (MECO) : 7–10 June 2022, Budva, Montenegro, Sep 2022

    Abstract DOI arXiv Bib

    Object detection is an essential component of many vision systems. For example, pedestrian detection is used in advanced driver assistance systems (ADAS) and advanced video surveillance systems (AVSS). Currently, most detectors (e.g. the YOLO - You Only Look Once - family) use deep convolutional neural networks. However, due to their high computational complexity, they are not able to process a very high-resolution video stream in real-time, especially within a limited energy budget. In this paper we present a hardware implementation of the well-known pedestrian detector with HOG (Histogram of Oriented Gradients) feature extraction and SVM (Support Vector Machine) classification. Our system running onAMD Xilinx Zynq UltraScale+ MPSoC(Multiprocessor System on Chip) device allows real-time processing of 4K resolution (UHD - Ultra High Definition,3840 × 2160 pixels) video for 60 frames per second. The system is capable of detecting a pedestrian in a single scale. The results obtained confirm the high suitability of reprogrammable devices in the real-time implementation of embedded vision systems.

    @inproceedings{badap-agh:140732,
      title = {Real-time HOG+SVM based object detection using SoC FPGA for a UHD video stream},
      projects = {Sonata},
      author = {Wasala, Mateusz and Kryjak, Tomasz},
      year = {2022},
      series = {11th~Mediterranean Conference on Embedded Computing (MECO) [Dokument elektroniczny] : 7–10 June 2022, Budva, Montenegro},
      pages = {1–6},
      doi = {10.1109/MECO55406.2022.9797113},
      collection = {11th~Mediterranean Conference on Embedded Computing (MECO) [Dokument elektroniczny] : 7–10 June 2022, Budva, Montenegro},
    }

13. Sensors Sonata
    

    Real-time efficient FPGA implementation of the multi-scale Lucas-Kanade and Horn-Schunck optical flow algorithms for a 4K video stream

    Krzysztof Blachut and Tomasz Kryjak

    Sensors, Sep 2022

    Abstract DOI Bib

    The information about optical flow, i.e., the movement of pixels between two consecutive images from a video sequence, is used in many vision systems, both classical and those based on deep neural networks. In some robotic applications, e.g., in autonomous vehicles, it is necessary to calculate the flow in real time. This represents a challenging task, especially for high-resolution video streams. In this work, two gradient-based algorithms-Lucas-Kanade and Horn-Schunck-were implemented on a ZCU 104 platform with Xilinx Zynq UltraScale+ MPSoC FPGA. A vector data format was used to enable flow calculation for a 4K (Ultra HD, 3840 × 2160 pixels) video stream at 60 fps. In order to detect larger pixel displacements, a multi-scale approach was used in both algorithms. Depending on the scale, the calculations were performed for different data formats, allowing for more efficient processing by reducing resource utilisation. The presented solution allows real-time optical flow determination in multiple scales for a 4K resolution with estimated energy consumption below 6 W. The algorithms realised in this work can be a component of a larger vision system in advanced surveillance systems or autonomous vehicles.

    @article{badap-agh:140919,
      title = {Real-time efficient FPGA implementation of the multi-scale Lucas-Kanade and Horn-Schunck optical flow algorithms for a~4K video stream},
      projects = {Sonata},
      author = {Blachut, Krzysztof and Kryjak, Tomasz},
      year = {2022},
      journal = {Sensors},
      series = {Sensors},
      volume = {22},
      number = {13},
      pages = {5017},
      doi = {10.3390/s22135017},
      issn = {1424-8220},
      collection = {Sensors},
    }

14. ARC

    Real-time embedded object tracking with discriminative correlation filters using convolutional features

    Michal Danilowicz and Tomasz Kryjak

    In Applied Reconfigurable Computing : architectures, tools, and applications : 18th international symposium, ARC 2022 : virtual event : September 19–20, 2022 : proceedings, Sep 2022

    Abstract DOI Bib

    Object tracking is an essential element of visual perception systems. It is used in advanced video surveillance systems (AVSS), autonomous vehicles, robotics, and many more. For applications such as autonomous robots, the system must be implemented on some embedded platform with limited computing performance and power. Furthermore, sufficiently fast response is required from the tracking system in order to perform some real-time tasks. Discriminative Correlation Filter (DCF) based tracking algorithms are popular for such applications, as they offer state-of-the-art performance while not being too computationally complex. In this paper, an FPGA implementation of the DCF tracking algorithm using convolutional features is presented. The ZCU104 board is used as a platform, and the performance is evaluated on the VOT2015 dataset. In contrast to other implementations that use HOG (Histogram of Oriented Gradients) features, this implementation achieves better results for 64×64 filter size while being able to potentially operate at higher speeds (over 467 fps per scale).

    @inproceedings{badap-agh:143788,
      title = {Real-time embedded object tracking with discriminative correlation filters using convolutional features},
      author = {Danilowicz, Michal and Kryjak, Tomasz},
      year = {2022},
      series = {Applied Reconfigurable Computing : architectures, tools, and applications : 18th~international symposium, ARC~2022 : virtual event : September 19–20, 2022 : proceedings},
      pages = {166–180},
      doi = {10.1007/978-3-031-19983-7_12},
      collection = {Applied Reconfigurable Computing : architectures, tools, and applications : 18th~international symposium, ARC~2022 : virtual event : September 19–20, 2022 : proceedings},
    }

15. Book chapter Preludium
    

    Systemy wizyjne w zastosowaniach przemyslowych

    Marek Gorgon, Maciej Aleksandrowicz, Krzysztof Blachut, Andrzej Brodzicki, Zbigniew Bublinski, and 20 more authors

    In Nauka – technika – technologia : seria wydawnicza AGH, Sep 2022

    Abstract DOI Bib

    Streszczenie: Systemy wizyjne to dynamicznie rozwijająca się dziedzina nauki i techniki.W niniejszym artykule przedstawiono przegląd zagadnień, nad którymi pracuje zespół Laboratorium Systemów Wizyjnych.Obejmuje on systemy wizyjne dla pojazdów autonomicznych, aplikacje analizy obrazów medycznych, systemy wspomagania szkolenia pracowników, analizę obrazów termowizyjnych, implementacje i akceleracje algorytmów wizyjnych w układach SoC FPGA i GPU, przetwarzanie strumienia wizyjnego w rozdzielczości 4K, a także analizę sygnałów z wykorzystaniem głębokich sieci neuronowych i współpracę przy eksperymencie ALICE w CERN-ie.

    @inbook{badap-agh:141409,
      title = {Systemy wizyjne w zastosowaniach przemyslowych},
      author = {Gorgon, Marek and Aleksandrowicz, Maciej and Blachut, Krzysztof and Brodzicki, Andrzej and Bublinski, Zbigniew and Cyba, Artur and Danilowicz, Michal and Glowacz, Adam and Jablonski, Miroslaw and Jaworek-Korjakowska, Joanna and Kostuch, Aleksander and Kowalczyk, Marcin and Kryjak, Tomasz and Kucharski, Dariusz and Lis, Konrad and Machura, Michal and Mikrut, Zbigniew and Pawlik, Piotr and Piekarski, Michal and Przewlocka-Rus, Dominika and Lis, Joanna and Szolc, Hubert and Wasala, Mateusz and Wojcicka, Anna Malgorzata and Wzorek, Piotr},
      year = {2022},
      projects = {Preludium-Kowalczyk},
      series = {Nauka – technika – technologia : seria wydawnicza AGH},
      volume = {T.~5},
      pages = {17–31},
      doi = {10.7494/978-83-67427-00-5_1},
      collection = {Nauka – technika – technologia : seria wydawnicza AGH},
    }

16. BAMS

    The bioinspired traffic sign classifier

    Dominika Przewlocka-Rus and Tomasz Kryjak

    Bio-Algorithms and Med-Systems, Sep 2022

    Abstract DOI Bib

    Objectives In this paper the research on developing convolutional spiking neural networks for traffic signs classification is presented. Unlike classical ones, spiking networks reflect the behaviour of biological neurons much more closely, by taking into account the time dimension and event-based operation. Spiking networks running on dedicated neuromorphic platforms, such as Intel Loihi, can operate with greater energy efficiency, hence they are an interesting approach for embedded solutions. Methods For convolutional spiking neural networks’ design and simulation, Nengo and NengoDL libraries for Python language were used. Numerous experiments using the Leaky-Integrate-and-Fire (LIF) neuron model were conducted. The training results, with different augmentation methods and number of time steps for input image presentation were compared. Results Finally, an accuracy of up to 97% on the test set was achieved, depending on the number of time steps the input was presented to the SNN. Conclusions The proposed experiments show that using simple convolutional spiking neural network, one can achieve accuracy comparable to the classical network with the same architecture and trained on the same dataset. At the same time, running on dedicated neuromorphic hardware, such solution should be characterized by low latency and low energy consumption.

    @article{badap-agh:139732,
      title = {The bioinspired traffic sign classifier},
      author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz},
      year = {2022},
      journal = {Bio-Algorithms and Med-Systems},
      series = {Bio-Algorithms and Med-Systems},
      volume = {18},
      number = {1},
      pages = {29},
      doi = {10.1515/bams-2021-0159},
      issn = {1895-9091},
      collection = {Bio-Algorithms and Med-Systems},
    }

17. DASIP

    Towards real-time and energy efficient Siamese tracking – a hardware-software approach

    Dominika Przewlocka-Rus and Tomasz Kryjak

    In DASIP 2022 : 15th international workshop on Design and Architectures for Signal and Image Processing : Budapest, Hungary, June 20–22, 2022 : proceedings, Sep 2022

    Abstract DOI arXiv Bib

    . Siamese trackers have been among the state-of-the-art solutions in each Visual Object Tracking (VOT) challenge over the past few years. However, with great accuracy comes great computational complexity: to achieve real-time processing, these trackers have to be massively parallelised and are usually run on high-end GPUs. Easy to implement, this approach is energy consuming, and thus cannot be used in many low-power applications. To overcome this, one can use energy-efficient embedded devices, such as heterogeneous platforms joining the ARM processor system with programmable logic (FPGA). In this work, we propose a hardware-software implementation of the well-known fully connected Siamese tracker (SiamFC). We have developed a quantised Siamese network for the FINN accelerator, using algorithm-accelerator co-design, and performed design space exploration to achieve the best efficiency-to-energy ratio (determined by FPS and used resources). For our network, running in the programmable logic part of the Zynq UltraScale+ MPSoC ZCU104, we achieved the processing of almost 50 frames-per-second with tracker accuracy on par with its floating point counterpart, as well as the original SiamFC network. The complete tracking system, implemented in ARM with the network accelerated on FPGA, achieves up to 17 fps. These results bring us towards bridging the gap between the highly accurate but energy-demanding algorithms and energy-efficient solutions ready to be used in low-power, edge systems.

    @inproceedings{badap-agh:141976,
      title = {Towards real-time and energy efficient Siamese tracking – a hardware-software approach},
      author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz},
      year = {2022},
      series = {DASIP~2022 : 15th~international workshop on Design and Architectures for Signal and Image Processing : Budapest, Hungary, June 20–22, 2022 : proceedings},
      pages = {162–173},
      doi = {10.1007/978-3-031-12748-9_13},
      collection = {DASIP~2022 : 15th~international workshop on Design and Architectures for Signal and Image Processing : Budapest, Hungary, June 20–22, 2022 : proceedings},
    }

18. ZSTN

    Traffic sign detection and recognition using event camera image reconstruction

    Kamil Jeziorek and Tomasz Kryjak

    Zeszyty Studenckiego Towarzystwa Naukowego, Sep 2022

    Abstract arXiv Bib

    This paper presents a method for detection and recognition of traffic signs based on information extracted from an event camera. The solution used a FireNet deep convolutional neural network to reconstruct events into greyscale frames. Two YOLOv4 network models were trained, one based on greyscale images and the other on colour images. The best result was achieved for the model trained on the basis of greyscale images, achieving an efficiency of 87.03%.

    @article{badap-agh:143548,
      title = {Traffic sign detection and recognition using event camera image reconstruction},
      author = {Jeziorek, Kamil and Kryjak, Tomasz},
      year = {2022},
      journal = {Zeszyty Studenckiego Towarzystwa Naukowego},
      series = {Zeszyty Studenckiego Towarzystwa Naukowego},
      pages = {127},
      issn = {1732-0925},
      url = {https://skn.agh.edu.pl/strony/59-hutnicza-konferencja-studenckich-kol-naukowych-agh/},
      collection = {Zeszyty Studenckiego Towarzystwa Naukowego},
    }

19. SPA

    Traffic sign detection with event cameras and DCNN

    Piotr Wzorek and Tomasz Kryjak

    In SPA 2022 : Signal Processing Algorithms, Architectures, Arrangements, and Applications : 25th IEEE SPA conference : Poznan, 21st - 22nd September 2022 : conference proceedings, Sep 2022

    Abstract DOI arXiv Bib

    In recent years, event cameras (DVS - Dynamic Vision Sensors) have been used in vision systems as an alternative or supplement to traditional cameras. They are characterised by high dynamic range, high temporal resolution, low latency, and reliable performance in limited lighting conditions - parameters that are particularly important in the context of advanced driver assistance systems (ADAS) and self-driving cars. In this work, we test whether these rather novel sensors can be applied to the popular task of traffic sign detection. To this end, we analyse different representations of the event data: event frame, event frequency, and the exponentially decaying time surface, and apply video frame reconstruction using a deep neural network called FireNet. We use the deep convolutional neural network YOLOv4 as a detector. For particular representations, we obtain a detection accuracy in the range of 86.9-88.9% mAP@0.5. The use of a fusion of the considered representations allows us to obtain a detector with higher accuracy of 89.9% mAP@0.5. In comparison, the detector for the frames reconstructed with FireNet is characterised by an accuracy of 72.67% mAP@0.5. The results obtained illustrate the potential of event cameras in automotive applications, either as standalone sensors or in close cooperation with typical frame-based cameras.

    @inproceedings{badap-agh:143871,
      title = {Traffic sign detection with event cameras and DCNN},
      author = {Wzorek, Piotr and Kryjak, Tomasz},
      year = {2022},
      series = {SPA~2022 [Dokument elektroniczny] : Signal Processing Algorithms, Architectures, Arrangements, and Applications : 25th IEEE SPA conference : Poznan, 21st - 22nd September 2022 : conference proceedings},
      pages = {86–91},
      doi = {10.23919/SPA53010.2022.9927864},
      collection = {SPA~2022 [Dokument elektroniczny] : Signal Processing Algorithms, Architectures, Arrangements, and Applications : 25th IEEE SPA conference : Poznan, 21st - 22nd September 2022 : conference proceedings},
    }

20. Automatyka

    Wbudowany system percepcji i sterowania dla modelu pojazdu autonomicznego

    Maciej Baczmanski, Balbina Molerus, Robert Synoczek, and Tomasz Kryjak

    In Automatyka przemysłowa i inżynieria w naukach technicznych – wybrane zagadnienia, Sep 2022

    Abstract Bib

    This paper presents the implementation of an autonomous driving algorithm for a Mecanum wheeled robot. Methods of roadway segmentation and detection of lane markings, pedestrians, obstacles and traffic lights using deep neural networks were applied. The developed algorithm was implemented on the NVIDIA Jetson Xavier eGPU computing platform. Video data was acquired from an Intel RealSense L515 sensor. The vehicle was controlled using video feedback. The result was an autonomous robot driving around a mock-up city. The vehicle detected upcoming turns, pedestrian crossings and objects on the road. On this basis, it executed turning manoeuvres in both directions and stopped in front of or evaded obstacles. The algorithm was verified in tests – both on images of the mock-up city and during test drives. The applicability and implementation of a deep network for simultaneous segmentation and detection for future system improvement and acceleration using the SoC FPGA platform and Vitis AI tool was also analysed.

    @inbook{badap-agh:142432,
      title = {Wbudowany system percepcji i sterowania dla modelu pojazdu autonomicznego},
      author = {Baczmanski, Maciej and Molerus, Balbina and Synoczek, Robert and Kryjak, Tomasz},
      year = {2022},
      series = {Automatyka przemys\l{}owa i~in\.{z}ynieria w~naukach technicznych – wybrane zagadnienia [Dokument elektroniczny]},
      pages = {22–37},
      url = {https://bc.wydawnictwo-tygiel.pl/public/assets/803/Automatyka%20przemys%C5%82owa%20i%20in%C5%BCynieria%20w%20naukach%20technicznych%20%E2%80%93%20wybrane%20zagadnienia.pdf},
      collection = {Automatyka przemys\l{}owa i~in\.{z}ynieria w~naukach technicznych – wybrane zagadnienia [Dokument elektroniczny]},
    }

21. TYGIEL

    Wbudowany system percepcji i sterowania dla modelu pojazdu autonomicznego

    Maciej Baczmanski, Balbina Molerus, Robert Synoczek, and Tomasz Kryjak

    In TYGIEL 2022 : ,,interdyscyplinarność kluczem do rozwoju” : XIV interdyscyplinarna konferencja naukowa : 24-27 marca 2022 r., [Lublin] : abstrakty, Sep 2022

    Bib

    @inproceedings{badap-agh:139866,
      title = {Wbudowany system percepcji i sterowania dla modelu pojazdu autonomicznego},
      author = {Baczmanski, Maciej and Molerus, Balbina and Synoczek, Robert and Kryjak, Tomasz},
      year = {2022},
      series = {TYGIEL~2022 : ,,interdyscyplinarno\'{s}\'{c} kluczem do rozwoju'' : XIV~interdyscyplinarna konferencja naukowa : 24-27 marca 2022 r., [Lublin] : abstrakty},
      pages = {193–193},
      url = {https://bc.wydawnictwo-tygiel.pl/publikacja/6ECBC4A2-3315-615F-FC8D-B0856D0CCC8F},
      collection = {TYGIEL~2022 : ,,interdyscyplinarno\'{s}\'{c} kluczem do rozwoju'' : XIV~interdyscyplinarna konferencja naukowa : 24-27 marca 2022 r., [Lublin] : abstrakty},
    }

2021

1. FPL Sonata
   Preludium
   

   A comparison of real-time 4K/UltraHD connected component labelling architectures

   Marcin Kowalczyk and Tomasz Kryjak

   In FPL 2021 : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings, Sep 2021

   Abstract DOI Bib

   This work presents a comparison of hardware architectures realising a connected component labelling (CCL) algorithm in reprogrammable logic. The architectures are capable of processing a video stream with 4K/UltraHD resolution at 60 frames per second in real-time. The modules were verified in simulation and in hardware on the Xilinx Zynq Ultrascale+ MPSoC chip on the ZCU104 evaluation board.

   @inproceedings{badap-agh:137405,
     title = {A comparison of real-time 4K/UltraHD connected component labelling architectures},
     projects = {Sonata, Preludium-Kowalczyk},
     author = {Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2021},
     series = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings},
     pages = {401–401},
     doi = {10.1109/FPL53798.2021.00086},
     collection = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings},
   }

2. DSD Preludium
   

   A connected component labelling algorithm for a multi-pixel per clock cycle video stream

   Marcin Kowalczyk and Tomasz Kryjak

   In 2021 24th Euromicro conference on Digital System Design : DSD 2021 : virtual conference, 1–3 September 2021 : proceedings, Sep 2021

   Abstract DOI arXiv Bib

   This work describes the hardware implementation of a connected component labelling (CCL) module in reprogammable logic. The main novelty of the design is the "full", i.e. without any simplifications, support of a 4 pixel per clock format (4 ppc) and real-time processing of a 4K/UltraHD video stream (3840 x 2160 pixels) at 60 frames per second. To achieve this, a special labelling method was designed and a functionality that stops the input data stream in order to process pixel groups which require writing more than one merger into the equivalence table. The proposed module was verified in simulation and in hardware on the Xilinx Zynq Ultrascale+ MPSoC chip on the ZCU104 evaluation board.

   @inproceedings{badap-agh:137005,
     title = {A connected component labelling algorithm for a multi-pixel per clock cycle video stream},
     author = {Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2021},
     projects = {Preludium-Kowalczyk},
     series = {2021~24th~Euromicro conference on Digital System Design [Dokument elektroniczny] : DSD~2021 : virtual conference, 1–3 September 2021 : proceedings},
     pages = {43–50},
     doi = {10.1109/DSD53832.2021.00016},
     collection = {2021~24th~Euromicro conference on Digital System Design [Dokument elektroniczny] : DSD~2021 : virtual conference, 1–3 September 2021 : proceedings},
   }

3. MMAR

   A simple vision-based navigation and control strategy for autonomous drone racing

   Artur Cyba, Hubert Szolc, and Tomasz Kryjak

   In MMAR 2021 : 2021 25th international conference on Methods and Models in Automation & Robotics : August 23–26, 2021, Międzyzdroje, Poland, Sep 2021

   Abstract DOI arXiv Bib

   In this paper, we present a control system that allows a drone to fly autonomously through a series of gates marked with ArUco tags. A simple and low-cost DJI Tello EDU quad-rotor platform was used. Based on the API provided by the manufacturer, we have created a Python application that enables the communication with the drone over WiFi, realises drone positioning based on visual feedback, and generates control. Two control strategies were proposed, compared and critically analysed. In addition, the accuracy of the positioning method used was measured. The application was evaluated on a laptop computer (about 40 fps) and a Nvidia Jetson TX2 embedded GPU platform (about 25 fps). We provide the developed code on GitHub.

   @inproceedings{badap-agh:135661,
     title = {A simple vision-based navigation and control strategy for autonomous drone racing},
     author = {Cyba, Artur and Szolc, Hubert and Kryjak, Tomasz},
     year = {2021},
     series = {MMAR~2021 [Dokument elektroniczny] : 2021~25th~international conference on Methods and Models in Automation \&~Robotics : August 23–26, 2021, Mi\k{e}dzyzdroje, Poland},
     pages = {185–190},
     doi = {10.1109/MMAR49549.2021.9528463},
     collection = {MMAR~2021 [Dokument elektroniczny] : 2021~25th~international conference on Methods and Models in Automation \&~Robotics : August 23–26, 2021, Mi\k{e}dzyzdroje, Poland},
   }

4. DASIP

   Automotive perception system evaluation with reference data obtained by a UAV

   Krzysztof Blachut, Michal Danilowicz, Hubert Szolc, Mateusz Wasala, Tomasz Kryjak, and 2 more authors

   In DASIP 2021 : workshop on Design and Architectures for Signal and Image Processing - 14th edition : January 18–20 2021, Budapest, Hungary : proceedings, Sep 2021

   Abstract DOI Bib

   Testing and evaluation of an automotive perception system is a complicated task which requires special equipment and infrastructure. To compute key performance indicators and compare the results with the real-world situation, some additional sensors and manual data labelling are often required. In this article, we propose a different approach, which is based on a UAV equipped with a 4K camera flying above the test track. Thanks to the synchronisation of the sensors between the tested vehicle and the UAV, it is possible to precisely determine the positions of the objects around the car and correlate them with the perception system readings. The performed experiments indicate that this approach could be an interesting alternative to the existing evaluation solutions.

   @inproceedings{badap-agh:132409,
     title = {Automotive perception system evaluation with reference data obtained by a UAV},
     author = {Blachut, Krzysztof and Danilowicz, Michal and Szolc, Hubert and Wasala, Mateusz and Kryjak, Tomasz and Pankiewicz, Nikodem and Komorkiewicz, Mateusz},
     year = {2021},
     series = {DASIP~2021 [Dokument elektroniczny] : workshop on Design and Architectures for Signal and Image Processing - 14th~edition : January 18–20 2021, Budapest, Hungary : proceedings},
     pages = {10–18},
     doi = {10.1145/3441110.3441151},
     collection = {DASIP~2021 [Dokument elektroniczny] : workshop on Design and Architectures for Signal and Image Processing - 14th~edition : January 18–20 2021, Budapest, Hungary : proceedings},
   }

5. FPL

   Hardware-software implementation of a DNN for 3D object detection using FINN - a demo

   Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In FPL 2021 : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings, Sep 2021

   Abstract DOI Bib

   In this demo, we present a hardware-software system for 3D object detection in LiDAR point clouds based on a deep neural network. The PointPillars architecture was used in this research, as it is a reasonable compromise between detection accuracy and computational complexity. The Brevitas tool was used for network quantisation and the FINN tool for hardware-software implementation in the reprogrammable Zynq UltraScale+ MPSoC device.

   @inproceedings{badap-agh:137413,
     title = {Hardware-software implementation of a DNN for 3D object detection using FINN - a demo},
     author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2021},
     series = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings},
     pages = {398–398},
     doi = {10.1109/FPL53798.2021.00083},
     collection = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings},
   }

6. DASIP

   Hardware-software implementation of the PointPillars network for 3D object detection in point clouds

   Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In DASIP 2021 : workshop on Design and Architectures for Signal and Image Processing - 14th edition : January 18–20 2021, Budapest, Hungary : proceedings, Sep 2021

   Abstract DOI Bib

   In this paper, we present a hardware-software implementation of a deep neural network for object detection based on a point cloud obtained by a LiDAR sensor. The Brevitas / PyTorch tools were used for network quantisation and the FINN tool for hardware implementation in the reprogrammable Zynq UltraScale+ MPSoC device. The PointPillars network was used in the research, as it is a reasonable compromise between detection accuracy and calculation complexity. The obtained results show that quite a significant computation precision limitation along with a few network architecture simplifications allows the solution to be implemented on an heterogeneous embedded platform with reasonable detection accuracy.

   @inproceedings{badap-agh:132404,
     title = {Hardware-software implementation of the PointPillars network for 3D object detection in point clouds},
     author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2021},
     series = {DASIP~2021 [Dokument elektroniczny] : workshop on Design and Architectures for Signal and Image Processing - 14th~edition : January 18–20 2021, Budapest, Hungary : proceedings},
     pages = {44–51},
     doi = {10.1145/3441110.3441150},
     collection = {DASIP~2021 [Dokument elektroniczny] : workshop on Design and Architectures for Signal and Image Processing - 14th~edition : January 18–20 2021, Budapest, Hungary : proceedings},
   }

7. FPL Sonata
   

   Quantised Siamese tracker for 4K/UltraHD video stream – a demo

   Dominika Przewlocka-Rus and Tomasz Kryjak

   In FPL 2021 : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings, Sep 2021

   Abstract DOI Bib

   This demo presents a hardware architecture for an object tracker based on a quantised Siamese neural network. The system is designed to work with a 4K/UltraHD input video stream and to meet the real time and energy efficiency constraints. The network is designed using Brevitas and implemented with FINN tool from Xilinx. The Siamese tracker runs on the ZCU 104 board, with the Zynq UltraScale+ MPSoC device from Xilinx.

   @inproceedings{badap-agh:137408,
     title = {Quantised Siamese tracker for 4K/UltraHD video stream – a demo},
     projects = {Sonata},
     author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz},
     year = {2021},
     series = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings},
     pages = {405–405},
     doi = {10.1109/FPL53798.2021.00089},
     collection = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings},
   }

8. JSPS Sonata
   Preludium
   

   Real-time FPGA implementation of parallel connected component labelling for a 4K video stream

   Marcin Kowalczyk, Piotr Ciarach, Dominika Przewlocka-Rus, Hubert Szolc, and Tomasz Kryjak

   Journal of Signal Processing Systems for Signal, Image, and Video Technology, Sep 2021

   Abstract DOI Bib

   In this paper, a hardware implementation in reconfigurable logic of a single-pass connected component labelling (CCL) and connected component analysis (CCA) module is presented. The main novelty of the design is the support of a video stream in 2 and 4 pixel per clock format (2 and 4 ppc) and real-time processing of 4K/UHD video stream (3840 x 2160 pixels) at 60 frames per second. We discuss several approaches to the issue and present in detail the selected ones. The proposed module was verified in an exemplary application – skin colour areas segmentation – on the ZCU 102 and ZCU 104 evaluation boards equipped with Xilinx Zynq UltraScale+ MPSoC devices.

   @article{badap-agh:134265,
     title = {Real-time FPGA implementation of parallel connected component labelling for a 4K video stream},
     projects = {Sonata, Preludium-Kowalczyk},
     author = {Kowalczyk, Marcin and Ciarach, Piotr and Przewlocka-Rus, Dominika and Szolc, Hubert and Kryjak, Tomasz},
     year = {2021},
     journal = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
     series = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
     volume = {93},
     number = {5},
     pages = {481},
     doi = {10.1007/s11265-021-01636-4},
     issn = {1939-8018},
     collection = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
   }

9. BAMS

   The bioinspired traffic sign classifier

   Dominika Przewlocka-Rus and Tomasz Kryjak

   Bio-Algorithms and Med-Systems, Sep 2021

   DOI Bib

   @article{badap-agh:135211,
     title = {The bioinspired traffic sign classifier},
     author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz},
     year = {2021},
     journal = {Bio-Algorithms and Med-Systems},
     series = {Bio-Algorithms and Med-Systems},
     volume = {17},
     pages = {eA22},
     doi = {10.1515/bams-2021-0050},
     issn = {1895-9091},
     collection = {Bio-Algorithms and Med-Systems},
   }

10. ZSTN

    Training dataset generation for automatic registration of a duplicate bridge game

    Piotr Wzorek and Tomasz Kryjak

    Zeszyty Studenckiego Towarzystwa Naukowego, Sep 2021

    Abstract Bib

    This paper presents a method for automatic generation of a training dataset for a deep convolutional neural network used for playing card detection. The solution allows to skip the time-consuming processes of manual image collecting and labelling recognised objects. The YOLOv4 network trained on the generated dataset achieved an efficiency of 99.8% in the cards detection task. The proposed method is a part of a project that aims to automate the process of broadcasting duplicate bridge competitions using a vision system and neural networks.

    @article{badap-agh:138692,
      title = {Training dataset generation for automatic registration of a duplicate bridge game},
      author = {Wzorek, Piotr and Kryjak, Tomasz},
      year = {2021},
      journal = {Zeszyty Studenckiego Towarzystwa Naukowego},
      series = {Zeszyty Studenckiego Towarzystwa Naukowego},
      pages = {217},
      issn = {1732-0925},
      url = {http://www.stn.agh.edu.pl/wydawnictwo/?pozycje-wydawnicze,4},
      collection = {Zeszyty Studenckiego Towarzystwa Naukowego},
    }

2020

1. ICCVG

   A vision based hardware-software real-time control system for the autonomous landing of an UAV

   Krzysztof Blachut, Hubert Szolc, Mateusz Wasala, Tomasz Kryjak, and Marek Gorgon

   In Computer Vision and Graphics : International Conference : ICCVG 2020 : Warsaw, Poland, September 14–16, 2020 : proceedings, Sep 2020

   Abstract DOI arXiv Bib

   In this paper we present a vision based hardware-software control system enabling autonomous landing of a mul-tirotor unmanned aerial vehicle (UAV). It allows the detection of a marked landing pad in real-time for a 1280 x 720 @ 60 fps video stream. In addition, a LiDAR sensor is used to measure the altitude above ground. A heterogeneous Zynq SoC device is used as the computing platform. The solution was tested on a number of sequences and the landing pad was detected with 96% accuracy. This research shows that a reprogrammable heterogeneous computing system is a good solution for UAVs because it enables real-time data stream processing with relatively low energy consumption.

   @inproceedings{badap-agh:130006,
     title = {A vision based hardware-software real-time control system for the autonomous landing of an UAV},
     author = {Blachut, Krzysztof and Szolc, Hubert and Wasala, Mateusz and Kryjak, Tomasz and Gorgon, Marek},
     year = {2020},
     series = {Computer Vision and Graphics : International Conference : ICCVG~2020 : Warsaw, Poland, September 14–16, 2020 : proceedings},
     pages = {13–24},
     doi = {10.1007/978-3-030-59006-2_2},
     collection = {Computer Vision and Graphics : International Conference : ICCVG~2020 : Warsaw, Poland, September 14–16, 2020 : proceedings},
   }

2. KKA

   Foreground object segmentation in RGB–D data implemented on GPU

   Piotr Janus, Tomasz Kryjak, and Marek Gorgon

   In Advanced, contemporary control : proceedings of KKA 2020 – the 20th Polish control conference : [14-16 October, 2020], Łódź, Poland, Sep 2020

   Abstract DOI arXiv Bib

   This paper presents a GPU implementation of two foreground object segmentation algorithms: Gaussian Mixture Model (GMM) and Pixel Based Adaptive Segmenter (PBAS) modified for RGB-D data support. The simultaneous use of colour (RGB) and depth (D) data allows to improve segmentation accuracy, especially in case of colour camouflage, illumination changes and occurrence of shadows. Three GPUs were used to accelerate calculations: embedded NVIDIA Jetson TX2 (Maxwell architecture), mobile NVIDIA GeForce GTX 1050m (Pascal architecture) and efficient NVIDIA RTX 2070 (Turing architecture).Segmentation accuracy comparable to previously published works was obtained. Moreover, the use of a GPU platform allowed to get real-time image processing. In addition, the system has been adapted to work with two RGB-D sensors: RealSense D415 and D435 from Intel.

   @inproceedings{badap-agh:129297,
     title = {Foreground object segmentation in RGB–D data implemented on GPU},
     author = {Janus, Piotr and Kryjak, Tomasz and Gorgon, Marek},
     year = {2020},
     series = {Advanced, contemporary control : proceedings of KKA 2020 – the 20th Polish control conference : [14-16 October, 2020], \L{}\'{o}d\'{z}, Poland},
     pages = {809–820},
     doi = {10.1007/978-3-030-50936-1_68},
     collection = {Advanced, contemporary control : proceedings of KKA 2020 – the 20th Polish control conference : [14-16 October, 2020], \L{}\'{o}d\'{z}, Poland},
   }

3. ICCVG

   Optimisation of a Siamese neural network for real-time energy efficient object tracking

   Dominika Przewlocka-Rus, Mateusz Wasala, Hubert Szolc, Krzysztof Blachut, and Tomasz Kryjak

   In Computer Vision and Graphics : International Conference : ICCVG 2020 : Warsaw, Poland, September 14–16, 2020 : proceedings, Sep 2020

   Abstract DOI arXiv Bib

   In this paper the research on optimisation of visual object tracking using a Siamese neural network for embedded vision systems is presented. It was assumed that the solution shall operate in real-time, preferably for a high resolution video stream, with the lowest possible energy consumption. To meet these requirements, techniques such as the reduction of computational precision and pruning were considered. Brevitas, a tool dedicated for optimisation and quantisation of neural networks for FPGA implementation, was used. A number of training scenarios were tested with varying levels of optimisations – from integer uniform quantisation with 16 bits to ternary and binary networks. Next, the influence of these optimisations on the tracking performance was evaluated. It was possible to reduce the size of the convolutional filters up to 10 times in relation to the original network. The obtained results indicate that using quantisation can significantly reduce the memory and computational complexity of the proposed network while still enabling precise tracking, thus allow to use it in embedded vision systems. Moreover, quantisation of weights positively affects the network training by decreasing overfitting.

   @inproceedings{badap-agh:130002,
     title = {Optimisation of a Siamese neural network for real-time energy efficient object tracking},
     author = {Przewlocka-Rus, Dominika and Wasala, Mateusz and Szolc, Hubert and Blachut, Krzysztof and Kryjak, Tomasz},
     year = {2020},
     series = {Computer Vision and Graphics : International Conference : ICCVG~2020 : Warsaw, Poland, September 14–16, 2020 : proceedings},
     pages = {151–163},
     doi = {10.1007/978-3-030-59006-2_14},
     collection = {Computer Vision and Graphics : International Conference : ICCVG~2020 : Warsaw, Poland, September 14–16, 2020 : proceedings},
   }

4. SPA

   Optimisation of the PointPillars network for 3D object detection in point clouds

   Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In SPA 2020 : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings, Sep 2020

   Abstract DOI arXiv Bib

   In this paper we present our research on the optimisation of a deep neural network for 3D object detection in a point cloud. Techniques like quantisation and pruning available in the Brevitas and PyTorch tools were used. We performed the experiments for the PointPillars network, which offers a reasonable compromise between detection accuracy and calculation complexity. The aim of this work was to propose a variant of the network which we will ultimately implement in an FPGA device. This will allow for real-time LiDAR data processing with low energy consumption. The obtained results indicate that even a significant quantisation from 32-bit floating point to 2-bit integer in the main part of the algorithm, results in 5%-9% decrease of the detection accuracy, while allowing for almost a 16-fold reduction in size of the model.

   @inproceedings{badap-agh:130350,
     title = {Optimisation of the PointPillars network for 3D object detection in point clouds},
     author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2020},
     series = {SPA~2020 [Dokument elektroniczny] : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings},
     pages = {122–127},
     doi = {10.23919/SPA50552.2020.9241265},
     collection = {SPA~2020 [Dokument elektroniczny] : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings},
   }

2019

1. DASIP Sonata
   

   Hardware implementation of the SURF feature detector for 4K 4PPC video stream – a demo

   Karol Radwan and Tomasz Kryjak

   In dasip : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montréal, Canada, Sep 2019

   Abstract Bib

   This paper describes a hardware implementation of the Speeded Up Robust Features (SURF) algorithm. The feature detector has been fully implemented in programmable logic resources (FPGA) of a Xilinx Zynq UltraScale+ MPSoC device on the ZCU 104 development board. The system has been adapted to work with a 4K (3840x2160 @ 60 fps) video stream in 4 ppc (pixel per clock) format.

   @inproceedings{badap-agh:126255,
     title = {Hardware implementation of the SURF feature detector for 4K 4PPC video stream – a demo},
     projects = {Sonata},
     author = {Radwan, Karol and Kryjak, Tomasz},
     year = {2019},
     series = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
     pages = {1–1},
     url = {https://dasip-conference.org/wp-content/uploads/sites/2/2019/10/DASIP2019_paper_35.pdf},
     collection = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
   }

2. DASIP

   Hardware-software implementation of car detection system based on LiDAR sensor data - a demo

   Joanna Stanisz, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In dasip : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montréal, Canada, Sep 2019

   Abstract Bib

   The article presents an object detection system based on LiDAR sensor data. The solution allows real-time detection of cars using information about the distance and intensity of the reflection obtained from LiDAR. The module has been implemented in programmable logic resources of the Xilinx Zynq SoC (System on Chip) device. The detected objects were projected onto the camera image for visualization. The system is dedicated to Velodyne HDL-64E LiDAR.

   @inproceedings{badap-agh:126248,
     title = {Hardware-software implementation of car detection system based on LiDAR sensor data - a demo},
     author = {Stanisz, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2019},
     series = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
     pages = {1–2},
     url = {https://dasip-conference.org/wp-content/uploads/sites/2/2019/10/DASIP2019_paper_30.pdf},
     collection = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
   }

3. ARC Sonata
   

   Real-time FPGA implementation of connected component labelling for a 4K video stream

   Piotr Ciarach, Marcin Kowalczyk, Dominika Przewlocka-Rus, and Tomasz Kryjak

   In Applied Reconfigurable Computing : 15th international symposium, ARC 2019 : Darmstadt, Germany, April 9–11, 2019 : proceedings, Sep 2019

   Abstract DOI Bib

   In this paper, a hardware implementation in reconfigurable logic of a single-pass connected component labelling (CCL) and connected component analysis (CCA) module is presented. The main novelty of the design is the support of a video stream in 2 and 4 pixel per clock format (2 and 4 ppc) and real-time processing of 4K/UHD video stream (3840 x 2160 pixels) at 60 frames per second. We discuss several approaches to the issue and present in detail the selected ones. The proposed module was verified in an exemplary application – skin colour areas segmentation – on the ZCU 102 and ZCU 104 evaluation boards equipped with Xilinx Zynq UltraScale+ MPSoC devices.

   @inproceedings{badap-agh:121175,
     title = {Real-time FPGA implementation of connected component labelling for a 4K video stream},
     projects = {Sonata},
     author = {Ciarach, Piotr and Kowalczyk, Marcin and Przewlocka-Rus, Dominika and Kryjak, Tomasz},
     year = {2019},
     series = {Applied Reconfigurable Computing : 15th~international symposium, ARC~2019 : Darmstadt, Germany, April 9–11, 2019 : proceedings},
     pages = {165–180},
     doi = {10.1007/978-3-030-17227-5_13},
     collection = {Applied Reconfigurable Computing : 15th~international symposium, ARC~2019 : Darmstadt, Germany, April 9–11, 2019 : proceedings},
   }

4. DASIP Sonata
   

   Real-time implementation of adaptive correlation filter tracking for 4K video stream in Zynq UltraScale+ MPSoC

   Marcin Kowalczyk, Dominika Przewlocka-Rus, and Tomasz Kryjak

   In dasip : conference on Design and Architectures for Signal and Image Processing : 16–18 October, 2019, Montréal, Canada, Sep 2019

   Abstract DOI Bib

   In this paper a hardware-software implementation of adaptive correlation filter tracking for a 3840 × 2160 @ 60 fps video stream in a Zynq UltraScale+ MPSoC is discussed. Correlation filters gained popularity in recent years because of their efficiency and good results in the VOT (Visual Object Tracking) challenge. An implementation of the MOSSE (Minimum Output Sum of Squared Error) algorithm is presented. It utilizes 2-dimensional FFT for computing correlation and updates filter coefficients in every frame. The initial filter coefficients are computed on the ARM processor in the PS (Processing System), while all other operations are preformed in PL (Programmable Logic). The presented architecture was described with the use of Verilog hardware description language.

   @inproceedings{badap-agh:126251,
     title = {Real-time implementation of adaptive correlation filter tracking for 4K video stream in Zynq UltraScale+ MPSoC},
     projects = {Sonata},
     author = {Kowalczyk, Marcin and Przewlocka-Rus, Dominika and Kryjak, Tomasz},
     year = {2019},
     series = {dasip [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : 16–18 October, 2019, Montr\'{e}al, Canada},
     pages = {53–58},
     doi = {10.1109/DASIP48288.2019.9049203},
     collection = {dasip [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : 16–18 October, 2019, Montr\'{e}al, Canada},
   }

5. DASIP Sonata
   

   XNOR CNNs in FPGA: real-time detection and classification of traffic signs in 4K – a demo

   Dominika Przewlocka-Rus, Marcin Kowalczyk, and Tomasz Kryjak

   In dasip : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montréal, Canada, Sep 2019

   Bib

   @inproceedings{badap-agh:126253,
     title = {XNOR CNNs in FPGA: real-time detection and classification of traffic signs in 4K – a demo},
     projects = {Sonata},
     author = {Przewlocka-Rus, Dominika and Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2019},
     series = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
     pages = {1–1},
     url = {https://dasip-conference.org/wp-content/uploads/sites/2/2019/10/DASIP2019_paper_34.pdf},
     collection = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
   }

2018

1. ICCVG

   Embedded vision system for automated drone landing site detection

   Patryk Fraczek, Andre Mora, and Tomasz Kryjak

   In Computer vision and graphics : international conference : ICCVG 2018 : Warsaw, Poland, September 17–19, 2018 : proceedings, Sep 2018

   Abstract Collaboration DOI Bib

   • Department of Electrical and Computer Engineering
     NOVA School of Science and Technology, NOVA University Lisbon, Portugal

   All our collaborations

   This paper presents an embedded video subsystem used to classify the terrain, based on an image from a camera located under the drone, for the purpose of an automatic landing system. Colour and texture features, as well as decision trees and support vector machine classifiers were analysed and evaluated. The algorithm was supported with a shadow detection module. It was evaluated on 100 test cases and achieved over 80% performance. The designed video system was implemented on two embedded platforms – a Zynq SoC (System on Chip – Field Programmable Gate Array + ARM processor system) and a Jetson GPU (Graphic Processing Unit + ARM processor system). The performance achieved on both architectures is compared and discussed.

   @inproceedings{badap-agh:116985,
     title = {Embedded vision system for automated drone landing site detection},
     author = {Fraczek, Patryk and Mora, Andre and Kryjak, Tomasz},
     year = {2018},
     series = {Computer vision and graphics : international conference : ICCVG~2018 : Warsaw, Poland, September 17–19, 2018 : proceedings},
     pages = {397–409},
     doi = {10.1007/978-3-030-00692-1_35},
     collection = {Computer vision and graphics : international conference : ICCVG~2018 : Warsaw, Poland, September 17–19, 2018 : proceedings},
   }

2. DASIP

   Embedded vision system for automated drone landing site detection – a demo

   Patryk Fraczek, Andre Mora, and Tomasz Kryjak

   In dasip 2018 : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal, Sep 2018

   Collaboration Bib

   • Department of Electrical and Computer Engineering
     NOVA School of Science and Technology, NOVA University Lisbon, Portugal

   All our collaborations

   @inproceedings{badap-agh:118434,
     title = {Embedded vision system for automated drone landing site detection – a demo},
     author = {Fraczek, Patryk and Mora, Andre and Kryjak, Tomasz},
     year = {2018},
     series = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
     pages = {1–2},
     url = {https://web.fe.up.pt/~specs/events/dasip2018/files/paper_47.pdf},
     collection = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
   }

3. DASIP

   Hardware acceleration of face detection using a deep convolutional neural network – a demo

   Dominika Przewlocka-Rus and Tomasz Kryjak

   In dasip 2018 : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal, Sep 2018

   Bib

   @inproceedings{badap-agh:118431,
     title = {Hardware acceleration of face detection using a deep convolutional neural network – a demo},
     author = {Przewlocka-Rus, Dominika and Kryjak, Tomasz},
     year = {2018},
     series = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
     pages = {1–2},
     url = {https://web.fe.up.pt/~specs/events/dasip2018/files/paper_45.pdf},
     collection = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
   }

4. DASIP

   Hardware implementation of multi-scale Lucas-Kanade optical flow computation algorithm – a demo

   Krzysztof Blachut, Tomasz Kryjak, and Marek Gorgon

   In dasip 2018 : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal, Sep 2018

   Bib

   @inproceedings{badap-agh:118435,
     title = {Hardware implementation of multi-scale Lucas-Kanade optical flow computation algorithm – a~demo},
     author = {Blachut, Krzysztof and Kryjak, Tomasz and Gorgon, Marek},
     year = {2018},
     series = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
     pages = {1–2},
     url = {https://web.fe.up.pt/~specs/events/dasip2018/files/paper_49.pdf},
     collection = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
   }

5. SPA

   Hardware implementation of the Gaussian Mixture Model foreground object segmentation algorithm working with ultra-high resolution video stream in real-time

   Piotr Janus and Tomasz Kryjak

   In SPA 2018 : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 19th-21st September 2018 : conference proceedings, Sep 2018

   Abstract DOI Bib

   In this paper a hardware implementation of the Gaussian Mixture Model algorithm for background modelling and foreground object segmentation is presented. The proposed vision system is able to handle video stream with resolution up to 4K (3840 x 2160 pixels) and 60 frames per second. Moreover, the constraints caused by memory bandwidth limit are also discussed and a few different solutions to tackle this issue have been considered. The designed modules have been verified on the ZCU 102 development board with Xilinx Zynq UltraScale+ MPSoC device. Additionally, the computing performance and power consumption have been estimated.

   @inproceedings{badap-agh:116771,
     title = {Hardware implementation of the Gaussian Mixture Model foreground object segmentation algorithm working with ultra-high resolution video stream in real-time},
     author = {Janus, Piotr and Kryjak, Tomasz},
     year = {2018},
     series = {SPA~2018 [Dokument elektroniczny] : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 19th-21st September 2018 : conference proceedings},
     pages = {140–145},
     doi = {10.23919/SPA.2018.8563404},
     collection = {SPA~2018 [Dokument elektroniczny] : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 19th-21st September 2018 : conference proceedings},
   }

6. DASIP

   Hardware – software implementation of a SFM module for navigation an unmanned aerial vehicles – a demo

   Karol Radwan, Tomasz Kryjak, and Marek Gorgon

   In dasip : the 2018 conference on Design & Architectures for Signal & Image Processing : Porto, Portugal, October 10–12, 2018, Sep 2018

   Abstract DOI Bib

   In this paper the design of an embedded vision system for structure from motion (SFM) computation is presented. The solution allows for scene depth reconstruction based on two consecutive video frames registered by a moving camera. The module is developed for a Xilinx Zynq SoC (System on Chip) device and is divided into a hardware and software part. The programmable logic (PL) is used mainly for detect and match features. Fundamental matrix estimation and triangulation are executed in the processing system (PS-ARM based). The module is dedicated to navigation of unmanned aerial vehicles (UAV)-autonomous flight and landing. The system is adapted to process a 1280×720 pixel video stream @ 60 frames per second in realtime.

   @inproceedings{badap-agh:119436,
     title = {Hardware – software implementation of a SFM module for navigation an unmanned aerial vehicles – a~demo},
     author = {Radwan, Karol and Kryjak, Tomasz and Gorgon, Marek},
     year = {2018},
     series = {dasip [Dokument elektroniczny] : the 2018~conference on Design \& Architectures for Signal \& Image Processing : Porto, Portugal, October 10–12, 2018},
     pages = {60–61},
     doi = {10.1109/DASIP.2018.8596937},
     collection = {dasip [Dokument elektroniczny] : the 2018~conference on Design \& Architectures for Signal \& Image Processing : Porto, Portugal, October 10–12, 2018},
   }

7. JRTIP

   Real-time hardware-software embedded vision system for ITS smart camera implemented in Zynq SoC

   Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

   Journal of Real-Time Image Processing, Sep 2018

   Abstract DOI Bib

   The article demonstrates the usefulness of heterogeneous System on Chip (SoC) devices in smart cameras used in intelligent transportation systems (ITS). In a compact, energy efficient system the following exemplary algorithms were implemented: vehicle queue length estimation, vehicle detection, vehicle counting and speed estimation (using multiple virtual detection lines), as well as vehicle type (local binary features and SVM classifier) and colour (k-means classifier and YCbCr colourspace analysis) recognition. The solution exploits the hardware–software architecture, i.e. the combination of reconfigurable resources and the efficient ARM processor. Most of the modules were implemented in hardware, using Verilog HDL, taking full advantage of the possible parallelization and pipeline, which allowed to obtain real-time image processing. The ARM processor is responsible for executing some parts of the algorithm, i.e. high-level image processing and analysis, as well as for communication with the external systems (e.g. traffic lights controllers). The demonstrated results indicate that modern SoC systems are a very interesting platform for advanced ITS systems and other advanced embedded image processing, analysis and recognition applications.

   @article{badap-agh:104286,
     title = {Real-time hardware-software embedded vision system for ITS smart camera implemented in Zynq SoC},
     author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
     year = {2018},
     journal = {Journal of Real-Time Image Processing},
     series = {Journal of Real-Time Image Processing},
     volume = {15},
     number = {1},
     pages = {123},
     doi = {10.1007/s11554-016-0588-9},
     issn = {1861-8200},
     collection = {Journal of Real-Time Image Processing},
   }

8. DASIP Sonata
   

   Real-time implementation of contextual image processing operations for 4K video stream in Zynq UltraScale+ MPSoC

   Marcin Kowalczyk, Dominika Przewlocka-Rus, and Tomasz Kryjak

   In dasip : the 2018 conference on Design & Architectures for Signal & Image Processing : Porto, Portugal, October 10–12, 2018, Sep 2018

   Abstract DOI Bib

   In this paper hardware implementation of selected contextual based image pre-processing modules for a 3840×2160 @60 FPS video stream in a Zynq UltraScale+ MPSoC is discussed. The following operations are considered: simple averaging (box filter), Gaussian filter, edge detection using the Sobel and Canny methods, median filter and morphological erosion and dilation operations. The scheme for implementing contextual based operations for a video stream in the format of 2 and 4 pixels per clock and challenges related to the pipelined implementation of processing such data are described. Also the use of logic resources and energy efficiency of modules described in the Verilog hardware description language and using the High Level Synthesis tools (Vivado HLS, SDSoC and xfOpenCV library) are compared. All designed modules support real-time processing of a 4K@60 FPS video stream.

   @inproceedings{badap-agh:119438,
     title = {Real-time implementation of contextual image processing operations for 4K video stream in Zynq UltraScale+ MPSoC},
     projects = {Sonata},
     author = {Kowalczyk, Marcin and Przewlocka-Rus, Dominika and Kryjak, Tomasz},
     year = {2018},
     series = {dasip [Dokument elektroniczny] : the 2018~conference on Design \& Architectures for Signal \& Image Processing : Porto, Portugal, October 10–12, 2018},
     pages = {37–42},
     doi = {10.1109/DASIP.2018.8597105},
     collection = {dasip [Dokument elektroniczny] : the 2018~conference on Design \& Architectures for Signal \& Image Processing : Porto, Portugal, October 10–12, 2018},
   }

9. DASIP Sonata
   

   Real-time implementation of contextual image processing operations for 4K video stream in Zynq UltraScale+ MPSoC – a demo

   Marcin Kowalczyk, Dominika Przewlocka-Rus, and Tomasz Kryjak

   In dasip 2018 : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal, Sep 2018

   Bib

   @inproceedings{badap-agh:118432,
     title = {Real-time implementation of contextual image processing operations for 4K video stream in Zynq UltraScale+ MPSoC – a~demo},
     projects = {Sonata},
     author = {Kowalczyk, Marcin and Przewlocka-Rus, Dominika and Kryjak, Tomasz},
     year = {2018},
     series = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
     pages = {1–2},
     url = {https://web.fe.up.pt/~specs/events/dasip2018/files/paper_46.pdf},
     collection = {dasip~2018 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : October 10–12 2018, Porto, Portugal},
   }

10. ARC

    The use of HACP+SBT lossless compression in optimizing memory bandwidth requirement for hardware implementation of background modelling algorithms

    Kamil Piszczek, Piotr Janus, and Tomasz Kryjak

    In Applied Reconfigurable Computing : architectures, tools, and applications : 14th international symposium, ARC 2018 : Santorini, Greece, May 2–4, 2018 : proceedings, Sep 2018

    Abstract DOI Bib

    In this paper the issue of optimizing memory bandwidth to external RAM in FPGA hardware implementation of foreground object segmentation methods is discussed. Three representative background modelling algorithms: Running Average (RA), Gaussian Mixture Model (GMM) and Pixel Based Adaptive Segmenter (PBAS) and three lossless compression methods: Run Length Encoding (RLE), Huffman coding and Hierarchical Average and Copy Prediction (HACP) with Significant Bit Truncation (SBT) coding were considered. After initial simulations in a software model, it was decided to implement the HACP+SBT approach in hardware. In addition, the possibility of using the proposed solution for ultra high-definition video stream processing was evaluated.

    @inproceedings{badap-agh:113818,
      title = {The use of HACP+SBT lossless compression in optimizing memory bandwidth requirement for hardware implementation of background modelling algorithms},
      author = {Piszczek, Kamil and Janus, Piotr and Kryjak, Tomasz},
      year = {2018},
      series = {Applied Reconfigurable Computing : architectures, tools, and applications : 14th~international symposium, ARC~2018 : Santorini, Greece, May 2–4, 2018 : proceedings},
      pages = {379–391},
      doi = {10.1007/978-3-319-78890-6_31},
      collection = {Applied Reconfigurable Computing : architectures, tools, and applications : 14th~international symposium, ARC~2018 : Santorini, Greece, May 2–4, 2018 : proceedings},
    }

2017

1. DASIP

   Demo: embedded vision system for pedestrian detection based on HoG+SVM and use of motion information implemented in Zynq heterogeneous device

   Bartosz Meus, Tomasz Kryjak, and Marek Gorgon

   In dasip 2017 : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden, Sep 2017

   Bib

   @inproceedings{badap-agh:109751,
     title = {Demo: embedded vision system for pedestrian detection based on HoG+SVM and use of motion information implemented in Zynq heterogeneous device},
     author = {Meus, Bartosz and Kryjak, Tomasz and Gorgon, Marek},
     year = {2017},
     series = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden},
     pages = {1–2},
     collection = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden},
   }

2. SPA

   Embedded vision system for pedestrian detection based on HOG+SVM and use of motion information implemented in Zynq heterogeneous device

   Bartosz Meus, Tomasz Kryjak, and Marek Gorgon

   In SPA 2017 : Signal Processing : Algorithms, Architectures, Arrangements, and Applications : Poznan, 20–22nd September 2017 : conference proceedings, Sep 2017

   Abstract DOI Bib

   Pedestrian detection is a very important application of embedded real-time vision systems. It is essential in Advanced Driver Assistance Systems (ADAS) and Advanced Video Surveillance Systems (AVSS). The most widely used method involves a combination of Histogram of Oriented Gradients (HOG) features and Support Vector Machine (SVM) classifier. It offers quite high detection accuracy at the cost of high computational complexity. Therefore, it is impossible to use GPP-based (General Purpose Processor) embedded systems for this application. In this paper a hardware-software HOG+SVM pedestrian detection system implemented in a heterogeneous Xilinx Zynq device is presented. The base algorithm is supported with detection grouping and tracking procedures. They increase the performance of the solution, especially in case of temporal partially occluded or vague visible objects. The system is able to process a 1280 × 720 @ 60 fps video stream in real-time.

   @inproceedings{badap-agh:108956,
     title = {Embedded vision system for pedestrian detection based on HOG+SVM and use of motion information implemented in Zynq heterogeneous device},
     author = {Meus, Bartosz and Kryjak, Tomasz and Gorgon, Marek},
     year = {2017},
     series = {SPA~2017 : Signal Processing : Algorithms, Architectures, Arrangements, and Applications : Poznan, 20–22nd September 2017 : conference proceedings},
     pages = {406–411},
     doi = {10.23919/SPA.2017.8166901},
     collection = {SPA~2017 : Signal Processing : Algorithms, Architectures, Arrangements, and Applications : Poznan, 20–22nd September 2017 : conference proceedings},
   }

3. DASIP

   Hardware-software abandoned object detection vision system in heterogeneous Zynq device

   Tomasz Kryjak, Artur Skirzynski, and Marek Gorgon

   In dasip 2017 : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden, Sep 2017

   Abstract DOI Bib

   In this paper a hardware-software abandoned object detection vision system implemented in the Zynq SoC (System on Chip) device is presented. First, the solution was implemented in C++ and run as a bare metal application on the ARM processor core of the Zynq (using floating and fixed-point computations). For the target video stream 1280 χ 720 @ 50 fps (74.25 MHz pixel clock) it reached only 2 fps. Therefore, to speed-up the application, it was decided to move some of the image processing and analysis operations to the programmable logic. This allowed to obtain real-time image processing i.e. 50 fps, with power consumption of less than 4W.

   @inproceedings{badap-agh:109748,
     title = {Hardware-software abandoned object detection vision system in heterogeneous Zynq device},
     author = {Kryjak, Tomasz and Skirzynski, Artur and Gorgon, Marek},
     year = {2017},
     series = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden},
     pages = {1–2},
     doi = {10.1109/DASIP.2017.8122122},
     collection = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden},
   }

4. ZSTN

   Object tracking using a moving camera based on object detection and connected component labelling implemented in Zynq SoC

   Marcin Kowalczyk, Piotr Ciarach, and Tomasz Kryjak

   Zeszyty Studenckiego Towarzystwa Naukowego, Sep 2017

   Bib

   @article{badap-agh:109620,
     title = {Object tracking using a moving camera based on object detection and connected component labelling implemented in Zynq SoC},
     author = {Kowalczyk, Marcin and Ciarach, Piotr and Kryjak, Tomasz},
     year = {2017},
     journal = {Zeszyty Studenckiego Towarzystwa Naukowego},
     series = {Zeszyty Studenckiego Towarzystwa Naukowego},
     pages = {141},
     issn = {1732-0925},
     collection = {Zeszyty Studenckiego Towarzystwa Naukowego},
   }

5. DASIP

   Object tracking with the use of a moving camera implemented in heterogeneous Zynq SoC – a demo

   Marcin Kowalczyk, Tomasz Kryjak, and Marek Gorgon

   In dasip 2017 : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden, Sep 2017

   Abstract Bib

   In this paper a hardware-software design of an object tracking system, which uses a moving camera is presented. The solution is implemented on the Zybo development board with the Zynq SoC (System on Chip) device from Xilinx. The object’s position is used to control two servomotors, which constitute a pan-tilt mounting of the camera. The proposed system is able to process a 1280 x 720 @ 60 fps video stream in real-time and track moving objects.

   @inproceedings{badap-agh:109750,
     title = {Object tracking with the use of a moving camera implemented in heterogeneous Zynq SoC – a~demo},
     author = {Kowalczyk, Marcin and Kryjak, Tomasz and Gorgon, Marek},
     year = {2017},
     series = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden},
     pages = {1–2},
     url = {https://ieeexplore.ieee.org/document/8122123},
     collection = {dasip~2017 [Dokument elektroniczny] : conference on Design and Architectures for Signal and Image Processing : September 27–29, 2017, Dresden},
   }

6. KKA

   Object tracking with the use of a moving camera implemented in heterogeneous Zynq System on Chip

   Marcin Kowalczyk and Tomasz Kryjak

   In Trends in Advanced Intelligent Control, Optimization and Automation : proceedings of KKA 2017 – the 19th Polish Control Conference, Kraków, Poland, June 18–21, 2017, Sep 2017

   Abstract DOI Bib

   In this paper a hardware-software design of an object tracking system, which uses a moving camera is presented. The solution is implemented on the Zybo development board with the Zynq SoC (System on Chip) device from Xilinx. The object’s position is used to control two servomotors, which constitute a pan-tilt mounting of the camera. The proposed system is able to process a 1280 × 720 @ 60 fps video stream in real-time and track moving objects.

   @inproceedings{badap-agh:106487,
     title = {Object tracking with the use of a moving camera implemented in heterogeneous Zynq System on Chip},
     author = {Kowalczyk, Marcin and Kryjak, Tomasz},
     year = {2017},
     series = {Trends in Advanced Intelligent Control, Optimization and Automation : proceedings of KKA 2017 -- the 19th Polish Control Conference, Krak\'{o}w, Poland, June 18–21, 2017},
     pages = {354–363},
     doi = {10.1007/978-3-319-60699-6_34},
     collection = {Trends in Advanced Intelligent Control, Optimization and Automation : proceedings of KKA 2017 -- the 19th Polish Control Conference, Krak\'{o}w, Poland, June 18–21, 2017},
   }

2016

1. FedCSIS

   A compact deep convolutional neural network architecture for video based age and gender estimation

   Bartlomiej Hebda and Tomasz Kryjak

   In FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : 11–14 September, 2016, Gdansk, Poland : book of abstracts, Sep 2016

   Abstract DOI Bib

   In this paper research on a compact deep convolutional neural network (DCNN) architecture for age and gender estimation from facial images has been presented.The proposed solution was tested on the FERET and the Adience Benchmark databases.In the first case a 98.6% accuracy for gender and 86.4% for age estimation was obtained.For the Adience database, which contains images recorded in unconstrained conditions and is much more demanding, a 62.0% for gender and 42.0% for age accuracy was obtained.When compared to the reference results on a much larger network, the performance should be considered as satisfactory.The research shows that a compact DCNN with small input images can provide quite good classification results.

   @inproceedings{badap-agh:101176,
     title = {A compact deep convolutional neural network architecture for video based age and gender estimation},
     author = {Hebda, Bartlomiej and Kryjak, Tomasz},
     year = {2016},
     series = {FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : 11–14 September, 2016, Gdansk, Poland : book of abstracts},
     pages = {787–790},
     doi = {10.15439/2016F472},
     collection = {FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : 11–14 September, 2016, Gdansk, Poland : book of abstracts},
   }

2. ARC

   An efficient hardware architecture for block based image processing algorithms

   Tomasz Kryjak, Marek Gorgon, and Mateusz Komorkiewicz

   In Applied Reconfigurable Computing : 12th international symposium, ARC 2016 : Mangaratiba, RJ, Brazil, March 22–24, 2016 : proceedings, Sep 2016

   Abstract DOI Bib

   In this paper a novel method for data reordering in a video stream is presented. It can be used in high performance vision algorithms that require block based image processing. For case study a block based optical flow histogram computation application was selected. The proposed solution allows for a 2.3x and 6.3x speed-up for fixed-point and floating-point calculations respectively. This enables real-time operations for 1920 × 1080 pixels or higher resolutions

   @inproceedings{badap-agh:97049,
     title = {An efficient hardware architecture for block based image processing algorithms},
     author = {Kryjak, Tomasz and Gorgon, Marek and Komorkiewicz, Mateusz},
     year = {2016},
     series = {Applied Reconfigurable Computing : 12th international symposium, ARC~2016 : Mangaratiba, RJ, Brazil, March 22–24, 2016 : proceedings},
     pages = {54–65},
     doi = {10.1007/978-3-319-30481-6_5},
     collection = {Applied Reconfigurable Computing : 12th international symposium, ARC~2016 : Mangaratiba, RJ, Brazil, March 22–24, 2016 : proceedings},
   }

3. MAM

   An embedded vision-based tracking system for autonomous robot navigation

   Krzysztof Mazur and Tomasz Kryjak

   Measurement Automation and Monitoring, Sep 2016

   Abstract Bib

   In this paper, a prototype of an autonomous vehicle with an FPGA based computing system is described. It uses an ultrasonic sensor and a camera as sources of information about the surrounding environment. The vehicle is able to track an object selected by the operator (it does not need to be defined a-priori). In the proposed solution, the mean-shift algorithm is used. It has been adapted to operate in a pipeline, real-time vision system. The control unit is built of two PD controllers which determine the PWM ratio for the left and right DC drive. The whole system has been implemented on the Atlys development board with Spartan 6 FPGA device from Xilinx.

   @article{badap-agh:103408,
     title = {An embedded vision-based tracking system for autonomous robot navigation},
     author = {Mazur, Krzysztof and Kryjak, Tomasz},
     year = {2016},
     journal = {Measurement Automation and Monitoring},
     series = {Measurement Automation and Monitoring},
     volume = {62},
     pages = {172},
     issn = {2450-2855},
     collection = {Measurement Automation and Monitoring},
   }

4. DASIP

   Demo: FPGA implementation of the flux tensor moving object detection method

   Piotr Janus, Kamil Piszczek, Tomasz Kryjak, and Marek Gorgon

   In dasip : the 2016 conference on Design & Architectures for Signal & Image Processing : Rennes, France, October 12–14, 2016 : proceedings, Sep 2016

   Bib

   @inproceedings{badap-agh:101813,
     title = {Demo: FPGA implementation of the flux tensor moving object detection method},
     author = {Janus, Piotr and Piszczek, Kamil and Kryjak, Tomasz and Gorgon, Marek},
     year = {2016},
     series = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
     pages = {286–287},
     collection = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
   }

5. DASIP

   Demo: an embedded vision system supporting the home care for convalescent or elderly people

   Piotr Konopka, Tomasz Jonak, Tomasz Kryjak, and Marek Gorgon

   In dasip : the 2016 conference on Design & Architectures for Signal & Image Processing : Rennes, France, October 12–14, 2016 : proceedings, Sep 2016

   Bib

   @inproceedings{badap-agh:101812,
     title = {Demo: an embedded vision system supporting the home care for convalescent or elderly people},
     author = {Konopka, Piotr and Jonak, Tomasz and Kryjak, Tomasz and Gorgon, Marek},
     year = {2016},
     series = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
     pages = {288–289},
     collection = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
   }

6. DASIP

   Demo: autonomous vehicle model with vision-based tracking system

   Krzysztof Mazur, Tomasz Kryjak, and Marek Gorgon

   In dasip : the 2016 conference on Design & Architectures for Signal & Image Processing : Rennes, France, October 12–14, 2016 : proceedings, Sep 2016

   Bib

   @inproceedings{badap-agh:101815,
     title = {Demo: autonomous vehicle model with vision-based tracking system},
     author = {Mazur, Krzysztof and Kryjak, Tomasz and Gorgon, Marek},
     year = {2016},
     series = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
     pages = {293–294},
     collection = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
   }

7. IPC

   FPGA implementation of multi-scale face detection using HOG features and SVM classifier

   Michal Drożdż and Tomasz Kryjak

   Image Processing & Communications, Sep 2016

   Abstract Bib

   In this paper an FPGA based embedded vision system for face detection is presented. The sliding detection window, HOG+SVM algorithm and multi-scale image processing were used and extensively described. The applied computation parallelizations allowed to obtain real-time processing of a 1280 [×] 720 @ 50Hz video stream. The presented module has been verified on the Zybo development board with Zynq SoC device from Xilinx. It can be used in a vast number of vision systems, including diver fatigue monitoring.

   @article{badap-agh:112590,
     title = {FPGA implementation of multi-scale face detection using HOG features and SVM classifier},
     author = {Dro\.{z}d\.{z}, Michal and Kryjak, Tomasz},
     year = {2016},
     journal = {Image Processing \& Communications},
     series = {Image Processing \& Communications},
     volume = {21},
     number = {3},
     pages = {27},
     issn = {1425-140X},
     url = {https://goo.gl/85yxdp},
     collection = {Image Processing \& Communications},
   }

8. IPC

   FPGA implementation of multi-scale pedestrian detection in thermal images

   Tomasz Kanka, Tomasz Kryjak, and Marek Gorgon

   Image Processing & Communications, Sep 2016

   Abstract Bib

   In this paper an embedded vision system for human silhouette detection in thermal images is presented. As the computing platform a reprogrammable device (FPGA - Field Programmable Gate Array) is used. The detection algorithm is based on a sliding window approach, which content is compared with a probabilistic template. Moreover, detection is four scales in supported. On the used test database, the proposed method obtained 97% accuracy, with average one false detection per frame. Due to the used parallelization and pipelining real-time processing for 720 × 480 @ 50 fps and 1280 × 720 @ 50 fps video streams was achieved. The system has been practically verified in a test setup with a thermal camera.

   @article{badap-agh:112592,
     title = {FPGA implementation of multi-scale pedestrian detection in thermal images},
     author = {Kanka, Tomasz and Kryjak, Tomasz and Gorgon, Marek},
     year = {2016},
     journal = {Image Processing \& Communications},
     series = {Image Processing \& Communications},
     volume = {21},
     pages = {55},
     issn = {1425-140X},
     url = {https://goo.gl/jj6BRX},
     collection = {Image Processing \& Communications},
   }

9. ICCVG

   FPGA implementation of the flux tensor moving object detection method

   Piotr Janus, Kamil Piszczek, and Tomasz Kryjak

   In Computer vision and graphics : international conference : ICCVG 2016 : Warsaw, Poland, September 19–21, 2016 : proceedings, Sep 2016

   Abstract DOI Bib

   In this paper a hardware implementation in a field programmable gate array (FPGA) device of moving object segmentation using the flux tensor (FT) method is presented. The used algorithm and its parallelized version are described in details. The designed module has been verified on the VC 707 development board with Virtex 7 FPGA device for the following video stream parameters: 720 × 576 @ 50 fps (25 MHz pixel clock), 1280 × 720 @ 50 fps (74.25 MHz pixel clock) and 1920 × 1080 @ 50 fps (148.5MHz pixel clock). Additionally, the computing performance and power consumption have been estimated. The proposed module outperforms the previous FT implementations both in terms of real-time processing capabilities for high-definition stream, as well as energy efficiency.

   @inproceedings{badap-agh:101467,
     title = {FPGA implementation of the flux tensor moving object detection method},
     author = {Janus, Piotr and Piszczek, Kamil and Kryjak, Tomasz},
     year = {2016},
     series = {Computer vision and graphics [Dokument elektroniczny] : international conference : ICCVG~2016 : Warsaw, Poland, September 19–21, 2016 : proceedings},
     pages = {486–497},
     doi = {10.1007/978-3-319-46418-3_43},
     collection = {Computer vision and graphics [Dokument elektroniczny] : international conference : ICCVG~2016 : Warsaw, Poland, September 19–21, 2016 : proceedings},
   }

10. DASIP

    FPGA-based hardware-in-the-loop environment using video injection concept for camera-based systems in automotive applications

    Mateusz Komorkiewicz, Pawel Skruch, Tomasz Kryjak, Krzysztof Turek, and Marek Gorgon

    In dasip : the 2016 conference on Design & Architectures for Signal & Image Processing : Rennes, France, October 12–14, 2016 : proceedings, Sep 2016

    Abstract Bib

    In this paper an FPGA-based video HiL (Hardwarein-the-Loop) solution is presented. It was designed to evaluate an of-the-shelf ADAS (Advanced Driver Assistance System) smart camera consisting of a video sensor (so-called forward looking camera (FLC)) and video processing unit. It allows to directly capture and store on a PC the video stream from the sensor. Then, the pre-recorded sequences can be directly injected into the video processing unit. Therefore, it is possible to evaluate the vision system in the laboratory under conditions almost identical to those present during real test drives. Moreover, all experiments are fully repeatable. The proposed system supports video streams with resolution 1280×960 @ 30 fps in the RCCC (Red/Clear)1 colour system. It is used in research and development projects in Delphi Technical Center Krakow.

    @inproceedings{badap-agh:101814,
      title = {FPGA-based hardware-in-the-loop environment using video injection concept for camera-based systems in automotive applications},
      author = {Komorkiewicz, Mateusz and Skruch, Pawel and Kryjak, Tomasz and Turek, Krzysztof and Gorgon, Marek},
      year = {2016},
      series = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
      pages = {179–185},
      collection = {dasip [Dokument elektroniczny] : the 2016~conference on Design \& Architectures for Signal \& Image Processing : Rennes, France, October 12–14, 2016 : proceedings},
    }

11. AfCAI

    Using scanpath analysis for affects detection in computer users

    Piotr Augustyniak, Miroslaw Jablonski, Eliasz Kantoch, Tomasz Kryjak, Zbigniew Mikrut, and 2 more authors

    In AfCAI 2016 : proceedings of the workshop on Affective Computing and Context Awareness in Ambient Intelligence (AfCAI 2016) : Murcia, Spain, November 24–25, 2016, Sep 2016

    Bib

    @inproceedings{badap-agh:104194,
      title = {Using scanpath analysis for affects detection in computer users},
      author = {Augustyniak, Piotr and Jablonski, Miroslaw and Kantoch, Eliasz and Kryjak, Tomasz and Mikrut, Zbigniew and Pociask, El\.{z}bieta and Przybylo, Jaromir},
      year = {2016},
      series = {AfCAI~2016 [Dokument elektroniczny] : proceedings of the workshop on Affective Computing and Context Awareness in Ambient Intelligence (AfCAI 2016) : Murcia, Spain, November 24–25, 2016},
      pages = {1–6},
      url = {http://ceur-ws.org/Vol-1794/afcai16-paper9.pdf},
      collection = {AfCAI~2016 [Dokument elektroniczny] : proceedings of the workshop on Affective Computing and Context Awareness in Ambient Intelligence (AfCAI 2016) : Murcia, Spain, November 24–25, 2016},
    }

2015

1. ZSTN

   Age, race and gender estimation based on facial images

   Bartlomiej Hebda and Tomasz Kryjak

   Zeszyty Studenckiego Towarzystwa Naukowego, Sep 2015

   Bib

   @article{badap-agh:93338,
     title = {Age, race and gender estimation based on facial images},
     author = {Hebda, Bartlomiej and Kryjak, Tomasz},
     year = {2015},
     journal = {Zeszyty Studenckiego Towarzystwa Naukowego},
     series = {Zeszyty Studenckiego Towarzystwa Naukowego},
     pages = {137},
     issn = {1732-0925},
     collection = {Zeszyty Studenckiego Towarzystwa Naukowego},
   }

2. DASIP

   FPGA based system for real-time structure from motion computation

   Mateusz Komorkiewicz, Tomasz Kryjak, Katarzyna Chuchacz-Kowalczyk, Pawel Skruch, and Marek Gorgon

   In dasip : proceedings of the 2015 conference on Design & Architectures for Signal & Image Processing : Cracow, Poland, September 23–25, 2015, Sep 2015

   Abstract Bib

   In this paper a system for real-time structure from motion computation is presented. The module is dedicated for a video based Advanced Driver Assistance System, where information about the surrounding environment and scene depth is often required to facilitate advanced applications. A heterogeneous System on a Chip solution, based on a FPGA device and an ARM processor is presented. The described hardware-software co-design is able to perform all steps required for structure from motion computation: feature extraction for every video frame, feature matching between two consecutive frames, finding the fundamental matrix, recovering the projection matrix and points triangulation. The design space exploration for the hardware part of the system is presented, which clearly points out the tradeoffs between resource usage, maximum clock and results accuracy. The system can process in real-time the images of 720x576 pixel resolution @ 50 fps. While the triangulation module allows for even high definition images processing and is able to compute results for up to 388 million points per second.

   @inproceedings{badap-agh:93233,
     title = {FPGA based system for real-time structure from motion computation},
     author = {Komorkiewicz, Mateusz and Kryjak, Tomasz and Chuchacz-Kowalczyk, Katarzyna and Skruch, Pawel and Gorgon, Marek},
     year = {2015},
     series = {dasip [Dokument elektroniczny] : proceedings of the 2015~conference on Design \& Architectures for Signal \& Image Processing : Cracow, Poland, September 23–25, 2015},
     pages = {28–34},
     collection = {dasip [Dokument elektroniczny] : proceedings of the 2015~conference on Design \& Architectures for Signal \& Image Processing : Cracow, Poland, September 23–25, 2015},
   }

3. ICIAP

   Real-time implementation of background modelling algorithms in FPGA devices

   Tomasz Kryjak and Marek Gorgon

   In ICIAP 2015 : international workshops BioFor, CTMR, RHEUMA, ISCA, MADiMa, SBMI, and QoEM : new trends in image analysis and processing : Genoa, Italy, September 7–8, 2015 : proceedings, Sep 2015

   Abstract DOI Bib

   The article discusses the possibilities of hardware implementation of foreground object segmentation and background modelling algorithms in FPGA. The potential benefits, as well as challenges and problems associated with porting algorithms from general-purpose processors (CPU) to reconfigurable logic (FPGA) are presented. Also several hardware implementation of well known method are reviewed: GMM, Codebook, Clustering, ViBE and PBAS. The last algorithm was also evaluated on the SBI dataset.

   @inproceedings{badap-agh:92036,
     title = {Real-time implementation of background modelling algorithms in FPGA devices},
     author = {Kryjak, Tomasz and Gorgon, Marek},
     year = {2015},
     series = {ICIAP~2015 [Dokument elektroniczny] : international workshops BioFor, CTMR, RHEUMA, ISCA, MADiMa, SBMI, and QoEM : new trends in image analysis and processing : Genoa, Italy, September 7–8, 2015 : proceedings},
     pages = {519–526},
     doi = {10.1007/978-3-319-23222-5_63},
     collection = {ICIAP~2015 [Dokument elektroniczny] : international workshops BioFor, CTMR, RHEUMA, ISCA, MADiMa, SBMI, and QoEM : new trends in image analysis and processing : Genoa, Italy, September 7–8, 2015 : proceedings},
   }

4. IPC

   Segmentation of dishes for customer service automation in a self-service canteen

   Tomasz Kryjak, Damian Król, and Marek Gorgon

   Image Processing & Communications, Sep 2015

   Abstract DOI Bib

   The article describes research on dishes segmentation for the purposes of customer service process automation in a self-service canteen. The project assumptions and a prototype test stand are presented. The developed empty workspace detection and tray position determination algorithms are discussed. Finally, the chosen dishes segmentation algorithm is described and justified.

   @article{badap-agh:96348,
     title = {Segmentation of dishes for customer service automation in a self-service canteen},
     author = {Kryjak, Tomasz and Kr\'{o}l, Damian and Gorgon, Marek},
     year = {2015},
     journal = {Image Processing \& Communications},
     series = {Image Processing \& Communications},
     volume = {20},
     pages = {5},
     doi = {10.1515/ipc-2015-0030},
     issn = {1425-140X},
     collection = {Image Processing \& Communications},
   }

5. Conference

   Segmentation of dishes for the purposes of customer service process automation in a self-service canteen

   Tomasz Kryjak, Damian Król, and Marek Gorgon

   In Przetwarzanie i analiza sygnałów w systemach wizji i sterowania : Słok, 17–18 czerwca 2015 : [streszczenia artykułów], Sep 2015

   Bib

   @inproceedings{badap-agh:97542,
     title = {Segmentation of dishes for the purposes of customer service process automation in a self-service canteen},
     author = {Kryjak, Tomasz and Kr\'{o}l, Damian and Gorgon, Marek},
     year = {2015},
     series = {Przetwarzanie i~analiza sygna\l{}\'{o}w w~systemach wizji i~sterowania [Dokument elektroniczny] : S\l{}ok, 17–18 czerwca 2015 : [streszczenia artyku\l{}\'{o}w]},
     pages = {2–2},
     url = {http://www.kis.p.lodz.pl/g/content/file/slok/Streszczenia_AGH.pdf},
     collection = {Przetwarzanie i~analiza sygna\l{}\'{o}w w~systemach wizji i~sterowania [Dokument elektroniczny] : S\l{}ok, 17–18 czerwca 2015 : [streszczenia artyku\l{}\'{o}w]},
   }

6. FedCSIS

   Shape and colour recognition of dishes for the purpose of customer service process automation in a self-service canteen

   Tomasz Kryjak and Damian Król

   In FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : September 13-16, 2015, Łódź, Poland, Sep 2015

   Abstract DOI Bib

   In the article a vision system for shape and colour recognition of dishes (plates, bowls, mugs), which can be used to automate the process of customer service in a self-service canteen is described.In consists of three basic components: object segmentation using so-called background model subtraction, shape recognition using geometric invariant moments and SVM classifier, as well as colour recognition using a Gaussian model.In addition, recognition in case of close or abut objects using a distance transform like approach is presented.The solution was evaluated on a dedicated test stand with controlled LED lightning.A 98% accuracy was obtained on over 100 test images, which indicates that the solution could be used in business practise.

   @inproceedings{badap-agh:92094,
     title = {Shape and colour recognition of dishes for the purpose of customer service process automation in a~self-service canteen},
     author = {Kryjak, Tomasz and Kr\'{o}l, Damian},
     year = {2015},
     series = {FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : September 13-16, 2015, \L{}\'{o}d\'{z}, Poland},
     pages = {799–808},
     doi = {10.15439/2015F184},
     collection = {FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : September 13-16, 2015, \L{}\'{o}d\'{z}, Poland},
   }

2014

1. Sensors

   Efficient hardware implementation of the Horn-Schunck algorithm for high-resolution real-time dense optical flow sensor

   Mateusz Komorkiewicz, Tomasz Kryjak, and Marek Gorgon

   Sensors, Sep 2014

   Abstract DOI Bib

   This article presents an efficient hardware implementation of the Horn-Schunck algorithm that can be used in an embedded optical flow sensor. An architecture is proposed, that realises the iterative Horn-Schunck algorithm in a pipelined manner. This modification allows to achieve data throughput of 175 MPixels/s and makes processing of Full HD video stream (1; 920 × 1; 080 @ 60 fps) possible. The structure of the optical flow module as well as pre- and post-filtering blocks and a flow reliability computation unit is described in details. Three versions of optical flow modules, with different numerical precision, working frequency and obtained results accuracy are proposed. The errors caused by switching from floating- to fixed-point computations are also evaluated. The described architecture was tested on popular sequences from an optical flow dataset of the Middlebury University. It achieves state-of-the-art results among hardware implementations of single scale methods. The designed fixed-point architecture achieves performance of 418 GOPS with power efficiency of 34 GOPS/W. The proposed floating-point module achieves 103 GFLOPS, with power efficiency of 24 GFLOPS/W. Moreover, a 100 times speedup compared to a modern CPU with SIMD support is reported. A complete, working vision system realized on Xilinx VC707 evaluation board is also presented. It is able to compute optical flow for Full HD video stream received from an HDMI camera in real-time. The obtained results prove that FPGA devices are an ideal platform for embedded vision systems.

   @article{badap-agh:80299,
     title = {Efficient hardware implementation of the Horn-Schunck algorithm for high-resolution real-time dense optical flow sensor},
     author = {Komorkiewicz, Mateusz and Kryjak, Tomasz and Gorgon, Marek},
     year = {2014},
     journal = {Sensors},
     series = {Sensors},
     volume = {14},
     number = {2},
     pages = {2860},
     doi = {10.3390/s140202860},
     issn = {1424-8220},
     collection = {Sensors},
   }

2. IPC

   Foreground object segmentation in dynamic background scenarios

   Tomasz Kryjak

   Image Processing & Communications, Sep 2014

   Abstract DOI Bib

   In the paper research on foreground object segmentation in dynamic background scenarios (i.e. flowing water, moving leaves or shrubs) is described. The effectiveness of different algorithms: based on FIFO sample buffer, singlevariant, multi-variant (MOG, Clustering) and recently proposed ViBE and PBAS is evaluated. A post-processing method, that allows false detections reduction is also proposed. The solution was tested on sequences from the changedetection.net dataset. The obtained results indicate usefulness of the proposed approach.

   @article{badap-agh:88627,
     title = {Foreground object segmentation in dynamic background scenarios},
     author = {Kryjak, Tomasz},
     year = {2014},
     journal = {Image Processing \& Communications},
     series = {Image Processing \& Communications},
     volume = {19},
     pages = {25},
     doi = {10.1515/ipc-2015-0007},
     issn = {1425-140X},
     collection = {Image Processing \& Communications},
   }

3. DASIP

   Hardware-software implemantation of vehicle detection and counting using virtual detection lines

   Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

   In dasip : proceedings of the 2014 conference on Design & Architectures for Signal & Image Processing : Madrid, Spain, October 8–10, 2014, Sep 2014

   Abstract DOI Bib

   In this paper a hardware-software system for vehicle detection and counting at intersections is presented. It has been implemented and evaluated on the heterogeneous Zynq platform. The vehicle detection is based on the concept of virtual detection lines (VDL). Differences in colour, horizontal edges and Census transform results between areas around the VDL for two consecutive frames are used to disclosure particular vehicles. This part of the system is designed in hardware description language and implemented in the reconfigurable resources of the Zynq platform. Information about the vehicle presence, along with the time-spatial image obtained at the VDL are transmitted and processed on the ARM core integrated in the Zynq device. This allows to perform further analysis and eliminate false or multiple detections. The solution has been evaluated on several video sequences recorded in various conditions: sunny day (the occurrence of deep shadows), cloudy day, rain and night-time. The system could be an important element of an intelligent transportation system (ITS) i.e. applied in a smart camera.

   @inproceedings{badap-agh:85563,
     title = {Hardware-software implemantation of vehicle detection and counting using virtual detection lines},
     author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
     year = {2014},
     series = {dasip [Dokument elektroniczny] : proceedings of the 2014 conference on Design \& Architectures for Signal \& Image Processing : Madrid, Spain, October 8–10, 2014},
     pages = {1–8},
     doi = {10.1109/DASIP.2014.7115618},
     collection = {dasip [Dokument elektroniczny] : proceedings of the 2014 conference on Design \& Architectures for Signal \& Image Processing : Madrid, Spain, October 8–10, 2014},
   }

4. Book chapter

   Implementacja zaawansowanych algorytmów przetwarzania i analizy obrazów oraz szyfrowania w układach reprogramowalnych

   Tomasz Kryjak

   In Innowacyjne rozwiązania w obszarze automatyki, robotyki i pomiarów : młodzi innowacyjni 2014 : praca zbiorowa, Sep 2014

   Bib

   @inbook{badap-agh:88457,
     title = {Implementacja zaawansowanych algorytm\'{o}w przetwarzania i analizy obraz\'{o}w oraz szyfrowania w~uk\l{}adach reprogramowalnych},
     author = {Kryjak, Tomasz},
     year = {2014},
     series = {Innowacyjne rozwi\k{a}zania w~obszarze automatyki, robotyki i~pomiar\'{o}w : m\l{}odzi innowacyjni~2014 : praca zbiorowa},
     pages = {176–187},
     collection = {Innowacyjne rozwi\k{a}zania w~obszarze automatyki, robotyki i~pomiar\'{o}w : m\l{}odzi innowacyjni~2014 : praca zbiorowa},
   }

5. Book

   Implementacja zaawansowanych algorytmów przetwarzania i analizy obrazów oraz szyfrowania w układach reprogramowalnych [Dokument elektroniczny]

   Tomasz Kryjak

   Sep 2014

   Bib

   @book{badap-agh:86521,
     title = {Implementacja zaawansowanych algorytm\'{o}w przetwarzania i~analizy obraz\'{o}w oraz szyfrowania w~uk\l{}adach reprogramowalnych [Dokument elektroniczny]},
     author = {Kryjak, Tomasz},
     year = {2014},
     publisher = {Wydawnictwa AGH},
     isbn = {978-83-7464-747-2},
     url = {http://www.wydawnictwoagh.pl/IMPLEMENTACJA-ZAAWANSOWANYCH-ALGORYTMOW-%3Cbr-/%3EPRZETWARZANIA-I-ANALIZY-OBRAZOW%3Cbr-/%3EORAZ-SZYFROWA;s,karta,id,1014},
     place = {Krak\'{o}w},
   }

6. ICCVG

   Implementation of advanced foreground segmentation algorithms GMM, ViBE and PBAS in FPGA and GPU – a comparison

   Bartlomiej Bulat, Tomasz Kryjak, and Marek Gorgon

   In Computer Vision and Graphics : International Conference : ICCVG 2014 : Warsaw, Poland, September 15–17, 2014 : proceedings, Sep 2014

   Abstract DOI Bib

   The article presents the results of implementing advanced foreground object segmentation algorithms: GMM (Gaussian Mixture Model), ViBE (Visual Background Extractor) and PBAS (Pixel-Based Adaptive Segmenter) on different hardware platforms: CPU, GPU and FPGA. The influence of the architecture on the segmentation accuracy and feasibility to perform real-time video stream processing was analysed. Also the limitations resulting from the specific features of GPU and FPGA were pointed out. Furthermore, the possible use of different platforms in advanced vision systems was discussed.

   @inproceedings{badap-agh:84669,
     title = {Implementation of advanced foreground segmentation algorithms GMM, ViBE and PBAS in FPGA and GPU – a~comparison},
     author = {Bulat, Bartlomiej and Kryjak, Tomasz and Gorgon, Marek},
     year = {2014},
     series = {Computer Vision and Graphics : International Conference : ICCVG 2014 : Warsaw, Poland, September 15–17, 2014 : proceedings},
     pages = {124–131},
     doi = {10.1007/978-3-319-11331-9_16},
     collection = {Computer Vision and Graphics : International Conference : ICCVG 2014 : Warsaw, Poland, September 15–17, 2014 : proceedings},
   }

7. JRTIP Preludium
   

   Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device

   Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

   Journal of Real-Time Image Processing, Sep 2014

   Abstract DOI Bib

   The processing of a high-definition video stream in real-time is a challenging task for embedded systems. However, modern FPGA devices have both a high operating frequency and sufficient logic resources to be successfully used in these tasks. In this article, an advanced system that is able to generate and maintain a complex background model for a scene as well as segment the foreground for an HD colour video stream (1,920 × 1,080 @ 60 fps) in real-time is presented. The possible application ranges from video surveillance to machine vision systems. That is, in all cases, when information is needed about which objects are new or moving in the scene. Excellent results are obtained by using the CIE Lab colour space, advanced background representation as well as integrating information about lightness, colour and texture in the segmentation step. Finally, the complete system is implemented in a single high-end FPGA device.

   @article{badap-agh:80341,
     title = {Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device},
     author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
     year = {2014},
     journal = {Journal of Real-Time Image Processing},
     series = {Journal of Real-Time Image Processing},
     volume = {9},
     number = {1},
     pages = {61},
     doi = {10.1007/s11554-012-0290-5},
     issn = {1861-8200},
     collection = {Journal of Real-Time Image Processing},
     projects = {Preludium-Kryjak},
   }

8. IJET

   Real-time foreground object detection combining the PBAS background modelling algorithm and feedback from scene analysis module

   Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

   International Journal of Electronics and Telecommunications, Sep 2014

   Abstract DOI Bib

   The article presents a hardware implementation of the foreground object detection algorithm PBAS (Pixel-Based Adaptive Segmenter) with a scene analysis module. A mechanism for static object detection is proposed, which is based on consecutive frame differencing. The method allows to distinguish stopped foreground objects (e.g. a car at the intersection, abandoned luggage) from false detections (so-called ghosts) using edge similarity. The improved algorithm was compared with the original version on popular test sequences from the changedetection.net dataset. The obtained results indicate that the proposed approach allows to improve the performance of the method for sequences with the stopped objects. The algorithm has been implemented and successfully verified on a hardware platform with Virtex 7 FPGA device. The PBAS segmentation, consecutive frame differencing, Sobel edge detection and advanced one-pass connected component analysis modules were designed. The system is capable of processing 50 frames with a resolution of 720 × 576 pixels per second

   @article{badap-agh:81415,
     title = {Real-time foreground object detection combining the PBAS background modelling algorithm and feedback from scene analysis module},
     author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
     year = {2014},
     journal = {International Journal of Electronics and Telecommunications},
     series = {International Journal of Electronics and Telecommunications},
     volume = {60},
     pages = {61},
     doi = {10.2478/eletel-2014-0006},
     issn = {2081-8491},
     collection = {International Journal of Electronics and Telecommunications},
   }

9. DASIP

   Video surveillance algorithms implemented on the heterogeneous Zynq platform : [poster]

   Tomasz Kryjak, Mateusz Zun, and Marek Gorgon

   In dasip : proceedings of the 2014 conference on Design & Architectures for Signal & Image Processing : Madrid, Spain, October 8–10, 2014, Sep 2014

   Bib

   @inproceedings{badap-agh:85623,
     title = {Video surveillance algorithms implemented on the heterogeneous Zynq platform : [poster]},
     author = {Kryjak, Tomasz and Zun, Mateusz and Gorgon, Marek},
     year = {2014},
     booktitle = {dasip [Dokument elektroniczny] : proceedings of the 2014 conference on Design \& Architectures for Signal \& Image Processing : Madrid, Spain, October 8–10, 2014},
     pages = {1},
   }

10. Journal

    Real-time implementation of foreground object detection from a moving camera using ViBE algorithm

    Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

    Computer Science and Information Systems, Sep 2014

    Abstract DOI Bib

    The article presents a real-time hardware implementation of a foreground object detection for a non-static camera setup. The system consists of two parts: the calculation of the displacement between two consecutive frames using a correlation based corner tracker and background generation method ViBE (Visual Background Extractor). The paper discusses details of the used hardware modules, resource utilization, computing performance and power dissipation. The solution was evaluated on sequences recorded with a static and moving camera. The system was successfully tested on a hardware platform with an FPGA device. It allows to process a 720x576 pixels and 50 frames per second video stream in real-time.

    @article{badap-agh:86089,
      title = {Real-time implementation of foreground object detection from a moving camera using ViBE algorithm},
      author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
      year = {2014},
      journal = {Computer Science and Information Systems},
      volume = {11},
      number = {4},
      pages = {1617–1637},
      doi = {10.2298/CSIS131218055K},
      issn = {1820-0214},
    }

2013

1. PAR

   Rekonfigurowalny akcelerator kryptograficzny

   Tomasz Kryjak

   PAR Pomiary Automatyka Robotyka, Sep 2013

   Abstract Bib

   This paper discusses the use of FPGA devices as cryptographic accelerators, which are able to perform the encryption or decryption operation faster and using less power than general-purpose processors while offering great flexibility and the ability to further develop and modify the design. In the first part, the structure and resources available in modern FPGAs are presented and in the second the implementation of the cryptographic algorithm CLEFIA is discussed.

   @article{badap-agh:74342,
     title = {Rekonfigurowalny akcelerator kryptograficzny},
     author = {Kryjak, Tomasz},
     year = {2013},
     journal = {PAR Pomiary Automatyka Robotyka},
     number = {5},
     pages = {78–85},
     issn = {1427-9126},
   }

2. Conference

   Hardware implementation of the PBAS foreground detection method in FPGA

   Tomasz Kryjak, Mateusz Komorkiewicz, and Marek Gorgon

   In MIXDES 2013 : mixed design of integrated circuits and systems : Gdynia, June 20–22, 2013 : book of abstracts of the 20th international conference, Sep 2013

   Abstract Bib

   In this paper a hardware implementation of the state-of-the-art PBAS (Pixel-Based Adaptive Segmenter) foreground object detection algorithm in FPGA is presented. This background modelling and subtraction technique is using novel ideas which improve the detection results. In this work a research is presented that determines which parts of the original algorithm need to be adjusted and modified to allow hardware implementation. A comparison is presented how this modifications influence the results accuracy compared to the original software implementation and other algorithms. An example working system based on the proposed module using the evaluation board with Virtex 7 device is also presented. This is the first reported FPGA implementation of this algorithm.

   @inproceedings{badap-agh:74690,
     title = {Hardware implementation of the PBAS foreground detection method in FPGA},
     author = {Kryjak, Tomasz and Komorkiewicz, Mateusz and Gorgon, Marek},
     year = {2013},
     booktitle = {MIXDES 2013 : mixed design of integrated circuits and systems : Gdynia, June 20–22, 2013 : book of abstracts of the 20th international conference},
     pages = {129},
   }

3. FedCSIS

   Real-time implementation of the ViBe foreground object segmentation algorithm

   Tomasz Kryjak and Marek Gorgon

   In FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : September 8–11, 2013, Kraków, Poland, Sep 2013

   Abstract Bib

   This paper presents a novel real-time hardware implementation of the ViBe (VIsual Background Extractor) background generation algorithm in reconfigurable FPGA device. This novel method combines the advantages of typical recursive and non-recursive approaches and achieves very good foreground object segmentation results. In this work the issue of porting ViBe to a FPGA hardware platform is discussed, two modification to the original approach are proposed and a detailed description of the implemented system is presented. This is the first, known to the authors, FPGA implementation of this algorithm

   @inproceedings{badap-agh:76794,
     title = {Real-time implementation of the ViBe foreground object segmentation algorithm},
     author = {Kryjak, Tomasz and Gorgon, Marek},
     year = {2013},
     booktitle = {FedCSIS : abstracts of the Federated Conference on Computer Science and Information Systems : September 8–11, 2013, Kraków, Poland},
     pages = {56},
   }

4. PAK Preludium
   

   Implementacja obliczania map dysparycji w czasie rzeczywistym dla strumienia wizyjnego 3D zrealizowana w układzie FPGA

   Tomasz Kryjak and Mateusz Komorkiewicz

   Pomiary, Automatyka, Kontrola, Sep 2013

   Abstract Bib

   In the paper a system for acquisition and processing of a 3D video stream is presented. It can work with 3D HDMI cameras available on the market. In Section 2 the basic concepts of stereovision systems are described [1]. In Section 3 three distance metrics, SAD [4], ZSAD and Census [5], used for correspondence matching are discussed. Evaluation of the matching process on the Middlebury dataset [2] is also presented. The best results were obtained for the SAD and ZSAD methods and greyscale images. In Table 1 there are shown three best configurations. Figure 1 illustrates the obtained disparity maps. A description of the hardware implementation is given in Section 4. The block diagram of the system is presented in Figure 2. The proposed solution is able to process images transmitted in side by side mode, to compute two disparity maps (left to right and right to left, method from [4]), to use SAD or ZSAD cost function, to check maps consistency and execute median filtering for final image processing. The described module is highly parameterizable: different cost functions, window sizes and disparity range can be used, image size and median filtering size can be adjusted. FPGA resource utilization is presented in Table 2. A picture of the working system is shown in Figure 3 (1280 x 720 @60 fps, real-time video-stream processing). The proposed module can be used for video surveillance, pedestrian collision avoidance systems or in autonomous vehicles.

   @article{badap-agh:76884,
     title = {Implementacja obliczania map dysparycji w czasie rzeczywistym dla strumienia wizyjnego 3D zrealizowana w układzie FPGA},
     author = {Kryjak, Tomasz and Komorkiewicz, Mateusz},
     year = {2013},
     journal = {Pomiary, Automatyka, Kontrola},
     volume = {59},
     number = {8},
     pages = {748–750},
     issn = {0032-4140},
     projects = {Preludium-Kryjak},
   }

5. PAK

   Implementacja sprzętowa odejmowania tła metodą ViBe w układzie FPGA

   Tomasz Kryjak and Marek Gorgon

   Pomiary, Automatyka, Kontrola, Sep 2013

   Abstract Bib

   This paper presents a hardware implementation in the FPGA reconfigurable device of ViBe - a novel background subtraction algorithm. The method is based on a different, from those previously described and implemented, background modelling concept. It partly uses random numbers, which allowed us to significantly reduce the buffer size in relation to the standard methods like mean or median form a buffer. A detailed description of ViBe can be found in papers [6, 7, 8]. In this paper the role of background generation algorithms in image processing and analysis systems, with particular emphasis on hardware implementations is discussed (Section 1). The ViBe algorithm is described in Section 2. Then an analysis of the possibility of implementing ViBe in FPGA is presented (Section 3). Section 4 describes two proposed modifications: the use of the CIE Lab colour space and the enhanced flashing pixels detection method. Their desirability has been confirmed quantitatively using the "ChangeDetection" database [9]. A detailed description of the designed ViBe hardware module and image processing system is presented in Section 5. The scheme of the ViBe module is shown in Figure 5 and the whole system in Figure 4. Table 3 summarizes the hardware resource utilization. The proposed solution enables the detection of objects using the method ViBe and enables realtime processing of a colour 640 x 480 video stream at 60 frames per second. The obtained results confirm the high usefulness of FPGA in the implementation of advanced image processing and analysis algorithms.

   @article{badap-agh:78883,
     title = {Implementacja sprzętowa odejmowania tła metodą ViBe w układzie FPGA},
     author = {Kryjak, Tomasz and Gorgon, Marek},
     year = {2013},
     journal = {Pomiary, Automatyka, Kontrola},
     volume = {59},
     number = {11},
     pages = {1191–1194},
     issn = {0032-4140},
   }

6. IPC Preludium
   

   Human action recognition using simple geometric features and a finite state machine

   Damian Dudzinski, Tomasz Kryjak, and Zbigniew Mikrut

   Image Processing & Communications, Sep 2013

   Abstract Bib

   In this paper a human action recognition algorithm, which uses background generation with shadow elimination, silhouette description based on simple geometrical features and a finite state machine for recognizing particular actions is described. The performed tests indicate that this approach obtains a 81 % correct recognition rate allowing real-time image processing of a 360 X 288 video stream.

   @article{badap-agh:80270,
     title = {Human action recognition using simple geometric features and a finite state machine},
     author = {Dudzinski, Damian and Kryjak, Tomasz and Mikrut, Zbigniew},
     year = {2013},
     journal = {Image Processing \& Communications},
     volume = {18},
     number = {2–3},
     pages = {49–60},
     issn = {1425-140X},
     projects = {Preludium-Kryjak},
   }

7. IPC Preludium
   

   Hardware-software vision system for moving people counting based on 3D information

   Tomasz Kryjak and Mateusz Komorkiewicz

   Image Processing & Communications, Sep 2013

   Abstract Bib

   The article presents a hardware-software system for people counting based on analysis of 3D information. It consists of a 3D video source (3D camera), an FPGA evaluation board and a PC computer. The computationaly demanding disparity map calculation and median filtering are realized in hardware. The software part is responsible for people segmentation, tracking and counting. The data between system nodes is transferred via Ethernet. In the paper the system architecture, proposed algorithms and obtained results are presented.

   @article{badap-agh:80272,
     title = {Hardware-software vision system for moving people counting based on 3D information},
     author = {Kryjak, Tomasz and Komorkiewicz, Mateusz},
     year = {2013},
     journal = {Image Processing \& Communications},
     volume = {18},
     number = {2–3},
     pages = {71–81},
     issn = {1425-140X},
     projects = {Preludium-Kryjak},
   }