openEuler at the Base

The openEuler operating system is tailored for heterogeneous convergence. You’ll find support for a wide range of hardware environments, from x86 to ARM and RISC-V. openEuler also supports the new generation of AI-optimized chips. The Intelligence BooM stack enhances performance for AI workloads with:

• Coordinated scheduling of heterogeneous workloads

• Unified memory management for heterogeneous architectures

• Virtualization and pooling for heterogeneous computing power

A number of other components are built into the system for seamless AI integration. For instance, LMCache provides a key-value cache layer for memory optimization of large language models (LLMs). FalconFS is a high-performance distributed filesystem for AI workloads. Generalized Memory Management (GMEM) is a centralized management system for connecting heterogeneous memory in distributed environments (see the box entitled “What Is GMEM?”).

What Is GMEM?

New technologies like graphics processing units (GPUs), tensor processing units (TPUs), and field-programmable gate arrays (FPGAs) have complicated the task of memory allocation in today’s heterogeneous systems. The good news is that accelerators have their own built-in memory. The challenge is that this memory is separated from the CPU, and conventional techniques for moving data between memory locations can introduce bottlenecks. The Heterogeneous Memory Management (HMM) feature built into Linux can hamper performance and relies on manual tuning. GMEM “combines the two independent address spaces into a unified virtual memory space” (Figure 3) [2]. The GMEM APIs allow for heterogeneous memory programming without memory migrations. Figure 3: GMEM merges host and device memory into a unified virtual address space.

Data Management

Like any enterprise-grade Linux system, openEuler supports a variety of tools for database management in cloud, edge, HPC, and data center environments. However, the openEuler developers are aware that AI is a special use with special needs for managing vast quantities of data. AI runs on data, and openEuler’s BooM stack integrates a number of tools for meeting the challenge of managing data in AI environments. openEuler can work with openGauss, an open source database management system designed for high performance and high reliability. Also closely integrated with openEuler is pgvector, a PostgreSQL extension that supports the vector similarity search functions essential to many AI systems.

openEuler integrates another important component often used with LLMs and other AI applications: Data-Juicer, which is designed to efficiently generate data recipes used for training AI models.