Skip to main content

Hardware Access for Robotics, Vision & Multimedia Workloads

This guide outlines best practices for building containerized workloads on the Admiral platform that require direct, low-latency hardware access for Robotics (ROS/ROS2), Computer Vision, Motor Controllers, GPU/NPU Acceleration, and Multimedia Subsystems.

Unlike standard cloud containers, Admiral workloads run on physical edge nodes, autonomous mobile robots (AMRs), interactive kiosks, and embedded SBCs. This architecture bypasses heavyweight hypervisors and abstraction layers to communicate directly with hardware interfaces (SocketCAN, V4L2, Serial/UART, DRM/KMS, ALSA, and Evdev).

1. Robotics & Autonomous Hardware Access​

Robotics systems require real-time determinism, low-latency sensor ingestion, and high-bandwidth inter-process communication (IPC).

Serial & Motor Controllers (/dev/ttyUSB*, /dev/ttyACM*)​

Motor controllers, microcontrollers (STM32, Arduino, Teensy), and IMUs typically communicate over USB serial bridges.

Configuration in Admiral:

  • Full device access is the default (Device Isolation off), so every node under /dev is already visible.
  • With Device Isolation on, bind individual device nodes back in via System Mounts: 
    /dev/ttyUSB0  → Motor Controller / Odometry Board
    /dev/ttyACM0  → Microcontroller / IMU Sensor
    /dev/ttyS0    → Hardware UART Header

CAN Bus & Actuators (SocketCAN)​

Industrial robots, mobile robotic bases (AGVs/AMRs), and vehicle automation systems rely on Controller Area Network (CAN) bus protocols.

  • Admiral OS natively loads the kernel can and can_raw drivers.
  • Workloads configured with Host Network (hostNetwork: true) can directly bind to can0 and can1 interfaces using standard Linux SocketCAN utilities (candump, cansend, or python-can).

Vision Sensors, LiDAR & Depth Cameras (V4L2 / USB)​

Autonomous navigation and perception workloads depend on high-throughput cameras (Intel RealSense, OAK-D, StereoLabs ZED, industrial GigE/UVC cameras):

/dev/video0    → RGB Color Stream
/dev/video1    → Depth Map / IR Stream
/dev/video2    → Metadata Stream
/dev/bus/usb   → Direct USB Device Tree (Firmware upload & USB3 transfers)
InterfaceLinux device nodeTypical use
USB 3.0 depth cameras/dev/video*, /dev/bus/usbIntel RealSense D435/D455, Luxonis OAK-D stereo cameras with raw depth maps and IMU sync.
MIPI CSI-2/dev/video*, /dev/media*Direct sensor deserialisation (GMSL2 / FPD-Link III) for high-bandwidth automotive and drone cameras.
Network IP camerasHost Network (UDP/RTSP)Multi-camera RTSP and WebRTC decoding.

Dockerfile Best Practice for Camera Drivers: Ensure user permissions and udev rule compatibility by running workloads with rootless or properly mapped device groups:

# Grant container process access to video and dialout groups
RUN usermod -aG video,dialout,plugdev root

USB Peripherals, Adapters & Dynamic Hotplug (libusb / USB Devices)​

Edge nodes deploying industrial serial bridges, USB sensors, smartcard readers, hardware dongles, and specialized peripherals interface via raw USB interfaces (libusb) or Linux subsystem drivers.

Dynamic USB Hotplug & Reconnection​

Under conventional Docker, binding a static device path like --device /dev/bus/usb/001/004 breaks as soon as a peripheral power-cycles or disconnects, because the Linux kernel re-enumerates the hardware with a new device node number (e.g. 001/005).

On Admiral:

  • Default /dev Bind Mount Strategy: Admiral mounts the host /dev tree into the workload using recursive slave propagation (rbind, rslave).
  • When a USB device is plugged in, power-cycled, or re-enumerated, the new /dev/bus/usb/<bus>/<devnum> node is immediately visible inside the running workload namespace without requiring a container restart.
  • Cgroup Device Rules: Admiral's cgroup v2 controller permits all character USB devices (c 189:* rwm) by default unless Device Isolation (isolateDevices: true) is explicitly configured.

User & Device Permissions​

Inside your container userspace, ensure the application or daemon user belongs to the necessary device groups:

# Ensure system groups exist and add application service user
RUN usermod -aG dialout,plugdev,lp appuser

Persistent Peripheral State & Configurations​

Hardware calibration profiles, sensor databases, and persistent logs should survive container restarts and software updates. In your Admiral configuration, bind dedicated Custom Volumes:

Volume: device-data    → Mount Path: /var/lib/device-service
Volume: device-config  → Mount Path: /etc/device-service

Multi-Service Appliance Pattern (Application + Background Daemon + SSH)​

Because Admiral treats OCI images as a full userspace rootfs, you can run an application, a local hardware daemon, and an emergency SSH daemon together under an entrypoint script:

#!/bin/bash
set -e

# 1. Generate or restore SSH host keys on a persistent volume
if [ ! -f /etc/ssh/keys/ssh_host_rsa_key ]; then
    mkdir -p /etc/ssh/keys
    ssh-keygen -A
    cp -n /etc/ssh/ssh_host_* /etc/ssh/keys/
else
    cp -u /etc/ssh/keys/* /etc/ssh/
fi
/usr/sbin/sshd -D &

# 2. Start any local background services or hardware daemons
/usr/bin/hardware-daemon --background &

# 3. Launch your primary application using absolute executable paths
exec /opt/venv/bin/python -m app.service --port 8080

High-Speed Zero-Copy Shared Memory (/dev/shm)​

Robotics perception stacks (like ROS2 point cloud processing and OpenCV video pipelines) pass gigabytes of data per second between processes.

  • Docker defaults /dev/shm to 64MB, which drops ROS 2 shared-memory messages under load.
  • Admiral default: every workload gets a private /dev/shm sized to half of the workload's memory limit, which is roughly 40% of physical RAM on most boards. CycloneDDS, FastDDS, and Iceoryx zero-copy transports work without extra configuration.
  • Camera and LiDAR drivers can write raw frames straight into shared memory, so perception and SLAM nodes read them with no serialisation or copy.
  • Explicit sizing: to pin a specific size, add a System Mount of type tmpfs at /dev/shm with a size= option. It replaces the default: 
    Mount: tmpfs  →  /dev/shm (Type: tmpfs, Options: size=4g)

ROS 2 DDS Discovery & Multicast​

ROS 2 nodes discover each other using DDS discovery over UDP multicast, which the default bridged network blocks. Enable Host Network (hostNetwork: true) in your configuration. There is then no network namespace, bridge, or NAT between the workload and the robot's network:

  • FastDDS and CycloneDDS multicast discovery reaches on-chassis compute, other robots, operator stations, and charging docks on the same network.
  • Unicast peer lists and discovery servers (for networks that block multicast) work as they do on a bare Linux host.
  • Off-board tools such as RViz2, PlotJuggler, and Foxglove Studio see the robot's topics when ROS_DOMAIN_ID matches.

See Networking for the network modes.

Real-Time Scheduling​

Control loops (ros2_control, drive controllers, balancers) need bounded scheduling latency. Workloads have CAP_SYS_NICE and CAP_SYS_RESOURCE with unlimited locked memory, so nodes can set SCHED_FIFO / SCHED_RR priorities and lock their memory to avoid page-fault stalls. Admiral kernels are fully preemptible; for hard real-time guarantees, see Real-time kernels.

2. GPU & NPU Acceleration​

Computer vision, depth perception, and neural inference need direct access to GPUs and NPUs. Admiral supplies the device nodes and, on NVIDIA hardware, the matching host driver libraries, so images do not bundle host kernel drivers. Frameworks such as CUDA, TensorRT, and PyTorch stay in your image.

NVIDIA x86 GPUs​

On x86 machines with NVIDIA GPUs, Admiral OS detects the host driver and injects kernel device nodes into the workload namespace:

/dev/nvidia0          # Primary GPU execution device
/dev/nvidiactl        # NVIDIA control device
/dev/nvidia-uvm       # Unified memory management device
/dev/nvidia-uvm-tools # Profiling and diagnostic interface
  • No container driver bloat: Images do not need to bundle host kernel drivers. CUDA, TensorRT, and ROS perception stacks run against host-injected device nodes and libraries.
  • Unified memory (/dev/nvidia-uvm): Zero-copy transfers between system RAM and GPU memory for point-cloud processing and camera-frame inference.

Admiral injects device nodes and host driver libraries. Application frameworks stay in your image.

NVIDIA Jetson Orin (developer preview)​

Jetson AGX Orin, Orin NX, and Orin Nano use a native NVGPU path rather than the x86 /dev/nvidia0 + UVM layout.

  • The platform supplies GPU firmware, kernel modules, and libcuda.so.1, mounted directly into standard multiarch paths (/usr/lib/aarch64-linux-gnu/ and /usr/lib/) for zero-configuration drop-in support.
  • Your linux/arm64 glibc image supplies the CUDA runtime, application, and any graphics userspace (EGL, OpenGL, Vulkan).
  • CUDA compute and GPU display output share this driver path. nvidia-smi is not part of the Orin layout.

See Running GPU applications on Jetson Orin for the image contract, mount paths, group membership, and verification steps.

Rockchip RK3588 NPU​

For low-power autonomous systems, smart cameras, and drone companion computers operating under strict 15W thermal budgets, Admiral provides native pass-through for the Rockchip RK3588 6 TOPS NPU:

  • Direct /dev/rknpu Node: The Rockchip Neural Processing Unit is mounted directly into container workloads with full read/write permissions.
  • RKNN runtime in your image: Ship librknnrt.so in the container and run INT8 and FP16 quantized YOLOv8, MobileNet, and custom vision models with sub-10ms inference times at minimal power draw.

3. Fullscreen Graphics & DRM Display Pipelines​

When running graphical user interfaces (Qt, Chromium, Flutter, or custom OpenGL/Vulkan apps) on bare metal without a desktop environment (GNOME/XFCE):

Direct Rendering Manager (DRM/KMS)​

Workloads talk directly to the Linux Kernel Mode Setting (KMS) display drivers:

/dev/dri/card0       → Display Output Pipeline (Resolution & Mode Setting)
/dev/dri/renderD128  → 3D Hardware Acceleration & Video Decode

Eliminating X11 Overhead with Wayland​

For touch displays and modern UI frameworks, deploy lightweight Wayland compositors (such as cage or weston) directly inside the container:

# Install lightweight kiosk compositor
RUN apt-get update && apt-get install -y cage

Set container environment variables:

WLR_DRM_NO_MODIFIERS=1
XDG_RUNTIME_DIR=/tmp

4. Audio Strategy: Direct ALSA Implementation​

To guarantee reliable audio playback and microphone capture across industrial PCs, kiosks, and robotic voice interaction systems, communicate directly with the ALSA kernel drivers.

The "PulseAudio Trap"​

Standard desktop Linux images include libasound2-plugins, which attempt to route all audio through a non-existent PulseAudio/PipeWire daemon.

Best Practice: Remove the plugin package in your Dockerfile:

RUN apt-get update && apt-get remove -y libasound2-plugins || true

The "Plug" Conversion Layer​

Hardware sound cards often demand strict sample formats (e.g., Integer 16-bit @ 48kHz). If your application streams 32-bit float audio, the driver will fail.

Configure /etc/asound.conf inside your container to enable transparent format and sample rate conversion:

pcm.!default {
    type plug
    slave {
        pcm "hw:0,0"
        rate 48000
    }
}
ctl.!default {
    type hw
    card 0
}

5. Input Devices & Emergency Controls (Evdev)​

Touchscreens, rotary dials, hardware keypads, and physical Emergency Stop (E-Stop) switches interface via the Linux event interface (/dev/input/event*).

  • Bind /dev/input into the container.
  • In-container event listeners read raw evdev events with zero windowing latency: 
    from evdev import InputDevice, categorize, ecodes
    dev = InputDevice('/dev/input/event0')
    for event in dev.read_loop():
        if event.type == ecodes.EV_KEY:
            print(f"Hardware Input Event: {event.code} Value: {event.value}")

Next Steps​

Learn how to bundle these hardware capabilities into versioned Configurations and deploy them to Fleets. For what stays in your workload versus the platform (Wi-Fi, reboots, kernel patches, carrier boards, boot branding), see Platform Boundary & Customisation.