Mixed criticality¶

Automotive systems handle workloads with mixed criticality, meaning some tasks are more important than others. Isolating these tasks is crucial to:

Ensure that critical functions (like lane departure or blind spot warnings, rearview camera, or automatic headlight control) operate without interference from non-critical functions (like infotainment, interior lighting, and climate control)
Determine which applications have priority when they share resources
Provide freedom from interference (FFI)

Different levels of criticality in automotive are categorized by ASIL (Automotive Safety Integrity Level) and QM (Quality Management). ASIL levels (A, B, C, D) are assigned to safety-critical systems. QM refers to functions that are not safety-critical: they are lower priority than ASIL-rated systems.

Conventional mixed criticality in automotive¶

The automotive industry traditionally uses a combination of separate physical or virtual devices to achieve FFI:

Isolation by hardware: Using separate devices for critical and non-critical applications, such as dedicated ECUs for airbags and braking and separate ECUs for infotainment, guarantees that a failure in one system does not affect the functioning of a safety-critical system in another.

This diagram shows separate hardware devices with separate instances of the OS, each running an application with a different level of criticality.

FFI with separate physical devices
Virtualization: Software partitioning allows multiple applications on the same hardware, but critical and non-critical applications can be in isolated virtual environments. This strategy accomplishes FFI with fewer physical devices in the vehicles.

The “FFI with separate virtual machines” diagram shows separate VMs managed by a hypervisor on a single hardware device: each separate VM includes a separate instance of the OS, each running an application with a different level of criticality.

FFI with separate virtual machines

Red Hat’s approach to mixed criticality¶

Red Hat’s automotive software offering, AutoSD (public preview of Red Hat In-Vehicle-OS), and Red Hat In-Vehicle OS provide the necessary technology and tools to support mixed-criticality workloads.

With AutoSD and Red Hat In-Vehicle OS, you can continue running existing VMs dedicated to non-critical applications if you prefer to continue using your current application isolation mechanism. However, because they are Linux-based operating systems, you can also use containers as an isolation mechanism.

Note

This documentation presents the technology underlying AutoSD and Red Hat In-Vehicle-OS, but the safety-related requirements and documents are not covered here. The safety-related documents are available only to Red Hat In-Vehicle-OS customers. The safety documents impose some limitations and additional activities necessary for safety workloads.

Containers fully isolate a process or application from the host running them and from other containerized applications. They rely on features of the Linux kernel and a container image. Containers and container images ensure full consistency as they are deployed in different environments, such as:

Dev, test, and production
Public, private, and hybrid cloud or onsite data centers

The “FFI with virtual machines and containers” diagram shows separate VMs managed by a hypervisor on a single hardware device. Each VM is designated for a different level of criticality, and one VM is running a containerized application within a separate partition. These concepts are described in the topics below.

A shared hardware platform with a hypervisor that hosts two VMs. One VM is deployed on AutoSD OS. In the AutoSD OS VM, Podman serves as the container management tool. A containerized application workload runs in the root partition and a separate containerized application workload runs in the QM partition. A second VM runs a single application.

FFI with virtual machines and containers

The “Red Hat In-Vehicle OS 1.0” diagram shows separate VMs managed by a hypervisor on a single hardware device. The Red Hat In-Vehicle OS VM runs a containerized application workload in the root partition and a separate, containerized application workload in the QM partition. The second VM runs an ASIL D OS with a single application.

Red Hat In-Vehicle OS 1.0 with containers alongside another virtual machine

Container-based isolation and protection¶

Containers are lightweight and isolated environments. They provide isolation at the process level without running a complete operating system. Containers can be effectively used for mixed criticality if you include proper isolation, resource management, and scheduling mechanisms.

These isolation measures are essential for protecting against threats, including attempts by malicious actors to monopolize resources or to DDoS attack other users.

Containers have the following advantages in the automotive context:

Process-level mixed criticality: Configure mixed criticality at a process level by running your software stack on a single OS in a way that isolates applications from each other and from the rest of the system
Operational integrity: Protect against resource abuse and malicious attacks, to ensure operational integrity and uninterrupted service
Cost savings and simplified maintenance: Save costs and simplify maintenance by reducing the number of operating systems deployed in the vehicle
Resource efficiency: Save resources by running more workloads on the same hardware or by reducing the hardware capability for your existing workloads
Independent runtime environments: Run different applications that have different runtime environments without conflicts and without requiring several VMs
Decoupled application lifecycles: Decouple the application lifecycle from the OS lifecycle
OCI container standards: Use standardized Open Container Initiative (OCI) container technologies to benefit from the entire ecosystem dedicated to containerized applications

Container-orchestrated mixed criticality¶

A container-orchestrated mixed criticality orchestrates the following aspects of the system:

Resource management

Computing, graphics, audio, input
Shared devices, such as requests, limits, priorities, ownership

Security

Isolation
Minimized attack surface

Lifecycle management

Start, stop, resume, reboot, update, manage the software stack

Heterogeneous environments

Android, Linux, and other workloads

In a container-orchestrated mixed criticality system, to ensure the non-critical applications do not interfere with the critical applications, you must configure the containers such that no two containers interfere with each other.

To prevent accidental misconfiguration for non-critical applications, AutoSD and Red Hat In-Vehicle OS contain a dedicated preconfigured feature called the QM partition. This design ensures non-critical (QM) applications do not interfere with critical applications.

In this framework, you can assign the resources, networking interfaces, or user IDs that are available for all of the QM applications. The QM partition includes configuration to enable the termination of QM applications if the system comes under memory pressure. As a last resort, the kernel can stop the entire QM partition to free memory for critical applications.

Important

The QM partition is the technology – available in AutoSD for experimentation – that enables Red Hat In-Vehicle OS to run mixed-criticality workloads when the system is configured according to safety-related requirements and documents, which are available to Red Hat In-Vehicle-OS customers.

The preconfigured QM partition¶

To support FFI between safety-critical ASIL applications and non-critical QM applications, the QM workloads are configured within the preconfigured QM partition.

A preconfigured QM partition is a subsection of the host filesystem with its own root that has containerized applications nested within it. By default, this subsection of the host filesystem is /usr/lib/qm.

The “Host filesystem with a QM partition” diagram shows how a QM partition contains its own infrastructure to support containers.

Filesystem diagram that shows a root file system with an instance of systemd and /usr, /bin, /etc, /var, /lib, and /home directories and a Podman instance within /var/lib/containers. The rootfs for the preconfigured QM partition is stored in /usr/lib/qm. The QM partition has its own instances of systemd, Podman, cgroups, SELinux, namespaces, and SECcomp

Host filesystem with a QM partition

The QM partition provides dedicated instances of systemd and Podman that are unique and distinct from the systemd and Podman instances in the host partition. These dedicated instances guarantee that starting, stopping, or reloading an application in the QM partition cannot affect a similar action taken for an application in the host partition. AutoSD’s QM partition is preconfigured to constrain its applications and support FFI.

The next diagram shows high-priority (ASIL-rated) workloads running on the root partition, and a separate QM partition with lower-priority (QM) workloads.

The container isolation architecture diagram shows how to run an entire software stack on a single OS in a way that isolates containers, applications, and guest OS instances from one another and from the rest of the system

Container isolation in root and QM partitions

Enhancing security for containers with SELinux in the QM partition¶

The QM partition isolates QM applications, which can run as nested containers. This partition also uses a special SELinux configuration.

How SELinux works in a standard Linux environment¶

In a standard Linux environment, each process and each file in the system has an SELinux label. This label enables SELinux to restrict, at the kernel level, the actions that each process can take, according to a set of policies that define the rules for each label.

A standard container configuration has just one SELinux label that encompasses all processes in a container: container_t. SELinux uses this label, combined with the unique ID of each container, to constrain the container process. However, SELinux does not differentiate between processes running inside the container.

How SELinux works in the QM partition¶

In the AutoSD QM partition, SELinux has preconfigurations and an additional rule set for QM-specific SELinux labels. The QM partition has two sets of labels:

An SELinux label for processes in the QM partition
Additional labels for each container within the QM partition

SELinux differentiates between and isolates containerized processes within the QM partition, and isolates and constrains your nested containerized software. You can run nested containers within the QM partition, which has all of the security mechanics that apply to regular containers.

With this mechanism:

Critical processes in the root partition can see, watch, and stop the non-critical processes running in the QM partition.
In contrast, non-critical processes running in the QM partition cannot see or interfere with any process outside of the QM partition.

The QM partition runs SELinux and isolated instances of systemd, Podman, and any other applications. This practice isolates all applications from each other and the rest of the system. Critical and non-critical applications run within a single system. This implementation saves resources, grants flexibility, and enables per-application lifecycle management decoupled from the OS lifecycle.

To preserve FFI by using the QM partition in your mixed-criticality OS, see Deploying applications in the QM partition.

To use containers in your mixed-criticality automotive OS, see Containerizing applications.

Additional resources