Konrad Lis

2025

1. DASIP

   LiFT: lightweight, FPGA-tailored 3D object detection based on LiDAR data

   Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP 2025 : Barcelona, Spain, January 20–22, 2025 : proceedings, Sep 2025

   Abstract DOI arXiv Bib

   This paper presents LiFT, a lightweight, fully quantized 3D object detection algorithm for LiDAR data, optimized for real-time inference on FPGA platforms. Through an in-depth analysis of FPGA-specific limitations, we identify a set of FPGA-induced constraints that shape the algorithm’s design. These include a computational complexity limit of 30 GMACs (billion multiply-accumulate operations), INT8 quantization for weights and activations, 2D cell-based processing instead of 3D voxels, and minimal use of skip connections. To meet these constraints while maximizing performance, LiFT combines novel mechanisms with state-of-the-art techniques such as reparameterizable convolutions and fully sparse architecture. Key innovations include the Dual-bound Pillar Feature Net, which boosts performance without increasing complexity, and an efficient scheme for INT8 quantization of input features. With a computational cost of just 20.73 GMACs, LiFT stands out as one of the few algorithms targeting minimal-complexity 3D object detection. Among comparable methods, LiFT ranks first, achieving an mAP of 51.84% and an NDS of 61.01% on the challenging NuScenes validation dataset. The code will be available at https://github.com/vision-agh/lift.

   @inproceedings{badap-agh:160941,
     title = {LiFT: lightweight, FPGA-tailored 3D object detection based on LiDAR data},
     author = {Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2025},
     series = {Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP~2025 : Barcelona, Spain, January~20–22,~2025 : proceedings},
     pages = {28–40},
     doi = {10.1007/978-3-031-87897-8_3},
     collection = {Design and Architecture for Signal and Image Processing : 18th international workshop, DASIP~2025 : Barcelona, Spain, January~20–22,~2025 : proceedings},
   }

2023

1. MMAR

   Comparative study of subset selection methods for rapid prototyping of 3D object detection algorithms

   Konrad Lis and Tomasz Kryjak

   In MMAR 2023 : 27th international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Międzyzdroje, Poland : on line proceedings : technical papers, Sep 2023

   Abstract DOI arXiv Bib

   Object detection in 3D is a crucial aspect in the context of autonomous vehicles and drones. However, prototyping detection algorithms is time-consuming and costly in terms of energy and environmental impact. To address these challenges, one can check the effectiveness of different models by training on a subset of the original training set. In this paper, we present a comparison of three algorithms for selecting such a subset – random sampling, random per class sampling, and our proposed MONSPeC (Maximum Object Number Sampling per Class). We provide empirical evidence for the superior effectiveness of random per class sampling and MONSPeC over basic random sampling. By replacing random sampling with one of the more efficient algorithms, the results obtained on the subset are more likely to transfer to the results on the entire dataset. The code is available at: https://github.com/vision-agh/monspec.

   @inproceedings{badap-agh:148559,
     title = {Comparative study of subset selection methods for rapid prototyping of 3D object detection algorithms},
     author = {Lis, Konrad and Kryjak, Tomasz},
     year = {2023},
     series = {MMAR~2023 [Dokument elektroniczny] : 27th~international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Mi\k{e}dzyzdroje, Poland : on line proceedings : technical papers},
     pages = {344–349},
     doi = {10.1109/MMAR58394.2023.10242454},
     collection = {MMAR~2023 [Dokument elektroniczny] : 27th~international conference on Methods and Models in Automation and Robotics : 22–25 August 2023, Mi\k{e}dzyzdroje, Poland : on line proceedings : technical papers},
   }

2. ICCVG

   PointPillars backbone type selection for fast and accurate LiDAR object detection

   Konrad Lis and Tomasz Kryjak

   In Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw], Sep 2023

   Abstract DOI arXiv Bib

   3D object detection from LiDAR sensor data is an important topic in the context of autonomous cars and drones. In this paper, we present the results of experiments on the impact of backbone selection of a deep convolutional neural network on detection accuracy and computation speed. We chose the PointPillars network, which is characterised by a simple architecture, high speed, and modularity that allows for easy expansion. During the experiments, we paid particular attention to the change in detection efficiency (measured by the mAP metric) and the total number of multiply-addition operations needed to process one point cloud. We tested 10 different convolutional neural network architectures that are widely used in image-based detection problems. For a backbone like MobilenetV1, we obtained an almost 4x speedup at the cost of a 1.13% decrease in mAP. On the other hand, for CSPDarknet we got an acceleration of more than 1.5x at an increase in mAP of 0.33%. We have thus demonstrated that it is possible to significantly speed up a 3D object detector in LiDAR point clouds with a small decrease in detection efficiency. This result can be used when PointPillars or similar algorithms are implemented in embedded systems, including SoC FPGAs. The code is available at https://github.com/vision-agh/pointpillars_backbone.

   @inproceedings{badap-agh:145406,
     title = {PointPillars backbone type selection for fast and accurate LiDAR object detection},
     author = {Lis, Konrad and Kryjak, Tomasz},
     year = {2023},
     series = {Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw]},
     pages = {99–119},
     doi = {10.1007/978-3-031-22025-8_8},
     collection = {Computer Vision and Graphics : proceedings of the International Conference on Computer Vision and Graphics ICCVG 2022 : [19-21 September 2022, Warsaw]},
   }

2022

1. JSPS

   Implementation of the PointPillars network for 3D object detection in reprogrammable heterogeneous devices using FINN

   Joanna Lis, Konrad Lis, and Marek Gorgon

   Journal of Signal Processing Systems for Signal, Image, and Video Technology, Sep 2022

   Abstract DOI Bib

   In this paper, we present a hardware-software implementation of a deep neural network for object detection based on a point cloud obtained by a LiDAR sensor. The PointPillars network was used in the research, as it is a reasonable compromise between detection accuracy and calculation complexity. The Brevitas / PyTorch tools were used for network quantisation (described in our previous paper) and the FINN tool for hardware implementation in the reprogrammable Zynq UltraScale+ MPSoC device. The obtained results show that quite a significant computation precision limitation along with a few network architecture simplifications allows the solution to be implemented on a heterogeneous embedded platform with maximum 19% AP loss in 3D, maximum 8% AP loss in BEV and execution time 375ms (the FPGA part takes 262ms). We have also compared our solution in terms of inference speed with a Vitis AI implementation proposed by Xilinx (19 Hz frame rate). Especially, we have thoroughly investigated the fundamental causes of differences in the frame rate of both solutions. The code is available at https://github.com/vision-agh/pp-finn .

   @article{badap-agh:141196,
     title = {Implementation of the PointPillars network for 3D object detection in reprogrammable heterogeneous devices using FINN},
     author = {Lis, Joanna and Lis, Konrad and Gorgon, Marek},
     year = {2022},
     journal = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
     series = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
     volume = {94},
     pages = {659–674},
     doi = {10.1007/s11265-021-01733-4},
     issn = {1939-8018},
     collection = {Journal of Signal Processing Systems for Signal, Image, and Video Technology},
   }

2. Book chapter Preludium
   

   Systemy wizyjne w zastosowaniach przemyslowych

   Marek Gorgon, Maciej Aleksandrowicz, Krzysztof Blachut, Andrzej Brodzicki, Zbigniew Bublinski, and 20 more authors

   In Nauka – technika – technologia : seria wydawnicza AGH, Sep 2022

   Abstract DOI Bib

   Streszczenie: Systemy wizyjne to dynamicznie rozwijająca się dziedzina nauki i techniki.W niniejszym artykule przedstawiono przegląd zagadnień, nad którymi pracuje zespół Laboratorium Systemów Wizyjnych.Obejmuje on systemy wizyjne dla pojazdów autonomicznych, aplikacje analizy obrazów medycznych, systemy wspomagania szkolenia pracowników, analizę obrazów termowizyjnych, implementacje i akceleracje algorytmów wizyjnych w układach SoC FPGA i GPU, przetwarzanie strumienia wizyjnego w rozdzielczości 4K, a także analizę sygnałów z wykorzystaniem głębokich sieci neuronowych i współpracę przy eksperymencie ALICE w CERN-ie.

   @inbook{badap-agh:141409,
     title = {Systemy wizyjne w zastosowaniach przemyslowych},
     author = {Gorgon, Marek and Aleksandrowicz, Maciej and Blachut, Krzysztof and Brodzicki, Andrzej and Bublinski, Zbigniew and Cyba, Artur and Danilowicz, Michal and Glowacz, Adam and Jablonski, Miroslaw and Jaworek-Korjakowska, Joanna and Kostuch, Aleksander and Kowalczyk, Marcin and Kryjak, Tomasz and Kucharski, Dariusz and Lis, Konrad and Machura, Michal and Mikrut, Zbigniew and Pawlik, Piotr and Piekarski, Michal and Przewlocka-Rus, Dominika and Lis, Joanna and Szolc, Hubert and Wasala, Mateusz and Wojcicka, Anna Malgorzata and Wzorek, Piotr},
     year = {2022},
     projects = {Preludium-Kowalczyk},
     series = {Nauka – technika – technologia : seria wydawnicza AGH},
     volume = {T.~5},
     pages = {17–31},
     doi = {10.7494/978-83-67427-00-5_1},
     collection = {Nauka – technika – technologia : seria wydawnicza AGH},
   }

2021

1. FPL

   Hardware-software implementation of a DNN for 3D object detection using FINN - a demo

   Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In FPL 2021 : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings, Sep 2021

   Abstract DOI Bib

   In this demo, we present a hardware-software system for 3D object detection in LiDAR point clouds based on a deep neural network. The PointPillars architecture was used in this research, as it is a reasonable compromise between detection accuracy and computational complexity. The Brevitas tool was used for network quantisation and the FINN tool for hardware-software implementation in the reprogrammable Zynq UltraScale+ MPSoC device.

   @inproceedings{badap-agh:137413,
     title = {Hardware-software implementation of a DNN for 3D object detection using FINN - a demo},
     author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2021},
     series = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings},
     pages = {398–398},
     doi = {10.1109/FPL53798.2021.00083},
     collection = {FPL~2021 [Dokument elektroniczny] : 31st international conference on Field-Programmable Logic and Applications : Dresden, Germany 30 August – 3 September 2021 : proceedings},
   }

2. DASIP

   Hardware-software implementation of the PointPillars network for 3D object detection in point clouds

   Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In DASIP 2021 : workshop on Design and Architectures for Signal and Image Processing - 14th edition : January 18–20 2021, Budapest, Hungary : proceedings, Sep 2021

   Abstract DOI Bib

   In this paper, we present a hardware-software implementation of a deep neural network for object detection based on a point cloud obtained by a LiDAR sensor. The Brevitas / PyTorch tools were used for network quantisation and the FINN tool for hardware implementation in the reprogrammable Zynq UltraScale+ MPSoC device. The PointPillars network was used in the research, as it is a reasonable compromise between detection accuracy and calculation complexity. The obtained results show that quite a significant computation precision limitation along with a few network architecture simplifications allows the solution to be implemented on an heterogeneous embedded platform with reasonable detection accuracy.

   @inproceedings{badap-agh:132404,
     title = {Hardware-software implementation of the PointPillars network for 3D object detection in point clouds},
     author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2021},
     series = {DASIP~2021 [Dokument elektroniczny] : workshop on Design and Architectures for Signal and Image Processing - 14th~edition : January 18–20 2021, Budapest, Hungary : proceedings},
     pages = {44–51},
     doi = {10.1145/3441110.3441150},
     collection = {DASIP~2021 [Dokument elektroniczny] : workshop on Design and Architectures for Signal and Image Processing - 14th~edition : January 18–20 2021, Budapest, Hungary : proceedings},
   }

2020

1. SPA

   Optimisation of the PointPillars network for 3D object detection in point clouds

   Joanna Lis, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In SPA 2020 : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings, Sep 2020

   Abstract DOI arXiv Bib

   In this paper we present our research on the optimisation of a deep neural network for 3D object detection in a point cloud. Techniques like quantisation and pruning available in the Brevitas and PyTorch tools were used. We performed the experiments for the PointPillars network, which offers a reasonable compromise between detection accuracy and calculation complexity. The aim of this work was to propose a variant of the network which we will ultimately implement in an FPGA device. This will allow for real-time LiDAR data processing with low energy consumption. The obtained results indicate that even a significant quantisation from 32-bit floating point to 2-bit integer in the main part of the algorithm, results in 5%-9% decrease of the detection accuracy, while allowing for almost a 16-fold reduction in size of the model.

   @inproceedings{badap-agh:130350,
     title = {Optimisation of the PointPillars network for 3D object detection in point clouds},
     author = {Lis, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2020},
     series = {SPA~2020 [Dokument elektroniczny] : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings},
     pages = {122–127},
     doi = {10.23919/SPA50552.2020.9241265},
     collection = {SPA~2020 [Dokument elektroniczny] : Signal Processing : Algorithms, Architectures, Arrangements and Applications : Poznan, 23rd–25th September 2020 : conference proceedings},
   }

2019

1. Conference

   Sprzętowo-programowy system detekcji obiektów na podstawie fuzji danych z czujnika LiDAR oraz kamery : [streszczenie]

   Konrad Lis and Joanna Lis

   In 56. konferencja studenckich kół naukowych pionu hutniczego : [streszczenie], Sep 2019

   Bib

   @inproceedings{badap-agh:133502,
     title = {Sprzętowo-programowy system detekcji obiektów na podstawie fuzji danych z czujnika LiDAR oraz kamery : [streszczenie]},
     author = {Lis, Konrad and Lis, Joanna},
     year = {2019},
     series = {56. konferencja studenckich kół naukowych pionu hutniczego [Dokument elektroniczny] : [streszczenie]},
     pages = {57},
     collection = {56. konferencja studenckich kół naukowych pionu hutniczego [Dokument elektroniczny] : [streszczenie]},
   }

2. DASIP

   Hardware-software implementation of car detection system based on LiDAR sensor data - a demo

   Joanna Stanisz, Konrad Lis, Tomasz Kryjak, and Marek Gorgon

   In dasip : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montréal, Canada, Sep 2019

   Abstract Bib

   The article presents an object detection system based on LiDAR sensor data. The solution allows real-time detection of cars using information about the distance and intensity of the reflection obtained from LiDAR. The module has been implemented in programmable logic resources of the Xilinx Zynq SoC (System on Chip) device. The detected objects were projected onto the camera image for visualization. The system is dedicated to Velodyne HDL-64E LiDAR.

   @inproceedings{badap-agh:126248,
     title = {Hardware-software implementation of car detection system based on LiDAR sensor data - a demo},
     author = {Stanisz, Joanna and Lis, Konrad and Kryjak, Tomasz and Gorgon, Marek},
     year = {2019},
     series = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
     pages = {1–2},
     url = {https://dasip-conference.org/wp-content/uploads/sites/2/2019/10/DASIP2019_paper_30.pdf},
     collection = {dasip [Dokument elektroniczny] : the conference on Design and Architectures for Signal and Image Processing : TEXPO graduate student research competition : 16–18 October, 2019, Montr\'{e}al, Canada},
   }