- Course name in Hungarian: Autonóm járművek és robotok programozása
- Course name in English: Autonomous robots and vehicles software engineering
- Neptun course code:
GKNB_AUTM078
The course is offered in several engineering programs and builds on minimal existing programming knowledge. It starts with the following prerequisites: Vehicle Engineering (BSc) GKNB_INTM023 Basics of Programming, Software Engineering (BSc) GKNB_MSTM032 Python Programming, Computer Engineering (BSc) GKNB_INTM114 Programming, Mechatronics Engineering (BSc) GKNB_MSTM032 Python Programming, Electrical Engineering (BSc) GKNB_INTM023 Basics of Programming.
Course Objective
After successfully completing the course, students will have a comprehensive understanding of the software modules and other essential components of autonomous vehicles.
They will be able to develop simpler related software modules in C++ and Python. Additionally, students will be capable of applying the appropriate code, interpreting its functions, and further developing it within the ROS 2 framework.
The course content is based on the curricula of internationally recognized universities such as MIT, ETH Zurich, TU Munich, University of Virginia, and Stanford University. Certain parts of the curriculum were adopted or used as inspiration, adhering to the appropriate licenses, detailed here. In 2023, the course uses the ROS 2 Humble version, which is supported until May 2027.
Completing the course can be advantageous for employment at the following automotive/autonomous vehicle companies (in alphabetical order):
- AiMotive Kft. (Budapest)
- AVL Hungary Kft. (Budapest, Érd, Kecskemét)
- Bosch: Robert Bosch Kft. (Budapest, Győr, Zalaegerszeg)
- Continental Autonomous Mobility Hungary Kft. (Budapest, Győr, Debrecen)
- Vehicle Industry Research Center, SZE (Győr)
Further career-related insights can be found on statista, glassdoor, and linkedin.
Note
Based on the knowledge presented in the course, students can prepare theses, dissertations, project work, and TDK papers, and there is also an opportunity to complete the mandatory internship.
| Instructors | . | ||
|---|---|---|---|
 |
Dr. Ernő Horváth Course Leader github.com/horverno |
 |
Dr. Áron Ballagi Curriculum, not teaching github.com/aronball |
 |
Rudolf Krecht Simulation, Robotics github.com/rudolfkrecht |
 |
Miklós Unger Environmental Sensing github.com/umiklos |
 |
Gergő Ignéczi Control Engineering github.com/gfigneczi1 |
 |
Norbert Markó AI, Neural Networks github.com/norbertmarko |
In the fall semester of 2023/24, classes will be held in room A2 and the computer lab B6.
Theory¶
| Class | Date | Material |
|---|---|---|
| 1 | Sep. 6 | Introduction: Course structure. Knowledge of robotics and autonomous vehicles. Sensing, perception, planning, control, actuation. |
| 2 | Sep. 13 | ROS2 Concepts: Introduction to university robots and vehicles. Basics of ROS 2. |
| 3 | Sep. 20 | Sensing: Operation and signal processing of sensors such as cameras, LIDAR, GNSS (GPS), IMU, CAN. Main ROS 2 topics, ROS 2 time management. |
| 4 | Sep. 27 | Midterm Assignment: Introduction to the midterm assignment, grading criteria, ideas, Q&A. |
| 5 | Oct. 4 | Transformations: Rigid body motion, matrix multiplication review, demonstration of homogeneous coordinates with short code snippets, concept of quaternions. |
| 6 | Oct. 11 | Perception: Object recognition, object classification, object tracking and prediction, SLAM and LOAM. |
| 7 | Oct. 25 | Simulation: Overview of ROS 2 compatible simulators (e.g., Gazebo, Carla, SVL, OSSDC SIM, AirSim, AWSIM, CoppeliaSim, MVSim). |
| 8 | Nov. 8 | Planning: Global planning, local planning. Local planning: lateral and longitudinal planning. |
| 9 | Nov. 15 | Control: Vehicle control solutions (inverse models, predictive models, closed-loop models). |
| 10 | Nov. 22 | Artificial Intelligence: Neural networks with a focus on vehicles and robotics. |
Practice¶
| Class | Date | Material |
|---|---|---|
| 1 | Sep. 6 | Introduction + Linux + Computer Lab Knowledge: Using WSL2 on Windows OS. Basic computer lab knowledge. Linux commands that may be needed later. |
| 2 | Sep. 13 | Installation + Setting Up Development Environment + ROS2 Communication: First ROS 2 nodes, using ROS commands, build and source. |
| 3 | Sep. 20 | Sensing Practice: Common formats of sensor data: sensor_msgs/PointCloud2, sensor_msgs/Image, geometry_msgs/Pose, etc. Handling and playing back bag .mcap files. Creating a simple package that subscribes to position data. |
| 4 | Sep. 27 | Version Control, Git, Copilot, VS Code, ROS 2 Launch: Using Copilot for ROS 2 development, introduction to the template repo, using it, writing launch files in Python. |
| 5 | Oct. 4 | Transformations Practice: Creating a node that advertises transformations. Displaying markers, independent task with launch. |
| 6 | Oct. 11 | Perception Practice: Simple LIDAR filtering based on X, Y, and Z coordinates. |
| 7 | Oct. 25 | Simulation Introduction: Gazebo Fortress and ROS 2, Simulation Practice: Creating your own robot simulation. |
| 8 | Nov. 8 | Planning Practice: Implementing a polynomial-based local planner. Using Nav2 with a simulator. |
| 9 | Nov. 15 | Control Practice: PID tuning. Trajectory following with Gazebo simulator. Custom-developed controller and vehicle model. |
| 10 | Nov. 22 | Artificial Intelligence Practice: Neural networks practice. |
Grades¶
- Signature: Completion of a small assignment (simple home programming task)
- 1 (fail): \(0\% \leq ZH_{average} < 60\%\)
- 2 (pass): \(60\% \leq ZH_{average} < 85\%\)
- 3 (satisfactory): \(85\% \leq ZH_{average} \leq 100\%\)
- 4 (good): Large midterm assignment completed with major errors according to the specified guidelines
- 5 (excellent): Large midterm assignment completed with only a few minor errors according to the specified guidelines
Conventions¶
- practical material
- theoretical material
- supplementary material, may be necessary for a complete understanding
Topics Covered (not in the order of processing)¶
- Introduction: Introduction to autonomous vehicles: current status, past, and future. Sensors, actuators, communication technologies. (LIDAR, radar, active and passive cameras, GPS, odometry, IMU, CAN) Foxglove studio and own measurements for illustration.
- Software System: Software for autonomous vehicles: sensing, perception, planning, tracking. Simulation technologies, user interfaces. Frameworks: ROS/ROS2/MATLAB/LabVIEW roles, real-time systems (FPGA, real-time operating systems).
- Sensing: SLAM, object detection, object tracking and prediction. Curb detection, lane detection, road defect detection, vehicle and pedestrian detection/tracking, etc. Advantages and disadvantages of artificial intelligence (especially neural networks) and traditional (e.g., C++ based) algorithms, their fusion.
- Technological Knowledge: Linux, Git: Linux knowledge: Terminal handling, Git handling, VS code, ROS installation.
- Technological Knowledge: ROS 2 Basics: topics and messages, MCAP (Rosbag) playback, handling topics, accessing topic content from Python, rviz, rqt_plot, creating MCAP (Rosbag). ROS 2 ecosystem and development: Creating ROS 2 nodes in Python and C++: ROS 2 nodes, rqt_graph, Publisher / Subscriber node in Python, Publisher / Subscriber node in C++. First consultation on the individual project task.
- ROS 2 Programming: Processing ROS sensor data with a C++ node: Writing ROS nodes,
visualization_msgs, LIDAR sensor data:sensor_msgs/PointCloud2,sensor_msgs/LaserScan, etc. - Simulation and Control: Simulation: Using ROS nodes for simulation (gazebo) F1/10, rviz, finalizing individual project tasks. Planning block: types of trajectory planners, kinematic challenges, Control block: vehicle modeling, introduction to controllers, vehicle and actuator-level control, implementing motion (braking systems, steering systems, etc.).
- Outlook, Doctoral Research, University Student Teams: Nissan Leaf, Lexus, and Szenergy autonomous projects, introduction with selected code snippets. Sensing: point cloud handling or object detection based on camera. Perception / Planning: route determination, free space determination, trajectory planning. Control: closed-loop modeled vehicle, building a controller (e.g., PID or pure pursuit).
- Artificial Intelligence: Software for autonomous vehicles, summary, outlook on neural networks (artificial intelligence, AI).
- Technological Knowledge: Using ROS 2, its innovations compared to ROS.
- Project Work: Presentation of individual project tasks.
