{"id":1350,"date":"2025-09-25T14:43:05","date_gmt":"2025-09-25T11:43:05","guid":{"rendered":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/?page_id=1350"},"modified":"2025-12-15T16:19:45","modified_gmt":"2025-12-15T14:19:45","slug":"ompi-offloading-to-vulkan-gpus","status":"publish","type":"page","link":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/software\/ompi\/documentation\/ompi-offloading-to-vulkan-gpus\/","title":{"rendered":"OMPi: Offloading to Vulkan GPUs"},"content":{"rendered":"

Introduction<\/h2>\n

The latest version of OMPi enables offloading to Vulkan-supported GPUs through the vulkan<\/code> module. OMPi analyzes the OpenMP target directives appearing in a user application and generates kernel source files, written in\u00a0shader language (GLSL).<\/p>\n

The vulkan<\/code> module can:<\/p>\n

    \n
  1. Compile the shader files to SPIR-V binaries,<\/li>\n
  2. Optimize the generated binaries in terms of size and performance,<\/li>\n
  3. Initialize and finalize a Vulkan device,<\/li>\n
  4. Transfer data between the CPU and a Vulkan device,<\/li>\n
  5. Offload compute shaders to the requested Vulkan device.<\/li>\n<\/ol>\n

    Requirements<\/h2>\n

    The current requirements of the vulkan<\/code>\u00a0module are the following:<\/p>\n

      \n
    • Vulkan Loader\n
        \n
      • sudo apt-get install libvulkan-dev
        \n <\/code>OR<\/strong> <\/code><\/li>\n
      • dnf install vulkan-loader-devel<\/code><\/li>\n<\/ul>\n<\/li>\n
      • Vulkan Validation layers and SPIR-V tools\n
          \n
        • sudo apt install vulkan-validationlayers spirv-tools glslang-tools<\/code>
          \n OR<\/strong><\/li>\n
        • sudo dnf install mesa-vulkan-drivers vulkan-validation-layers-devel glslang<\/code><\/li>\n<\/ul>\n<\/li>\n
        • Vulkan Utilities\n
            \n
          • sudo apt-get install\u00a0vulkan-tools
            \n <\/code>OR<\/strong> <\/code><\/li>\n
          • dnf install vulkan-tools<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n

            On Arch Linux,ou can run\u00a0sudo pacman -S vulkan-devel<\/code>\u00a0to install all the required tools.<\/p>\n

            Before proceeding with the installation, verify the Vulkan installation and SPIR-V utilities by executing:<\/p>\n

            $ vulkaninfo<\/code><\/p>\n

            $ glslangValidator -v
            \n <\/code><\/p>\n

            The commands should display information about the Vulkan devices and Glslang\u00a0version, respectively.<\/p>\n

            Installing OMPi<\/h2>\n

            The first step is to\u00a0deploy OMPi on your system. Initially, you need to\u00a0build OMPi as usual:<\/p>\n

            meson setup build --prefix=<install-dir>\r\ncd build\/\r\nmeson compile\r\nmeson install<\/pre>\n
            During the installation process, if your system satisfies the minimum requirements and has at least one Vulkan-enabled GPU installed, OMPi will automatically build the vulkan<\/code>\u00a0module.\u00a0The correct installation of the module can be verified by running:<\/p>\n
            ompiconf --devvinfo<\/pre>\n
            which lists all the identified modules\/devices, along with their numeric device IDs<\/em>. If the above command fails to show a Vulkan GPU, please revisit the minimum requirements and ensure you have installed OMPi correctly.<\/p>\n
            <\/span>Sample output of devvinfo<\/div>
            \n
            The ompiconf --devvinfo<\/code> command should give an output similar to the following (here an Ampere A2 GPU was identified by OMPi):<\/p>\n
            MODULE [vulkan]:\r\n------\r\nOMPi Vulkan device module.\r\nAvailable devices : 1\r\n\r\ndevice id < 0 > { \r\n Name: NVIDIA A2\r\n API version: 4210991 - Driver version: 2392933632\r\n Vendor ID: 4318\r\n Maximum workgroup sizes: (2147483647, 65535, 65535)\r\n Maximum invocations: 1024\r\n 15724544 Kbytes of memory\r\n}\r\n------<\/pre>\n<\/div><\/div>\n
            Quick start<\/h2>\n
            Compiling your OpenMP application with Vulkan GPU offloading requires a simple\u00a0--devs=all<\/code>\u00a0argument:<\/p>\n
            ompicc --devs=all app.c<\/pre>\n
            <\/span>Note<\/div>
            \n
            If you use the\u00a0--devs=vulkan<\/code>\u00a0compilation option, the numeric device IDs are those printed by\u00a0ompiconf --devinfo=vulkan<\/code>. For example, in our system we get:\u00a0<\/p>\n
            MODULE [vulkan] provides device(s) : 0 1<\/code><\/p>\n
            You\u00a0may\u00a0instead compile using the\u00a0--devs=all<\/code>\u00a0option to produce kernels that may run on any GPU target,\u00a0including<\/em>\u00a0Vulkan.\u00a0Be careful that in this case the numeric device IDs may be different depending on what other modules\u00a0are active. In the case you use\u00a0--devs=all<\/code>, the\u00a0correct device IDs can be found by exectuting\u00a0ompiconf --devinfo<\/code>. In our system it reported:<\/p>\n
            MODULE [cuda] provides device(s) : 0<\/code>
            \n MODULE [vulkan] provides device(s) : 1 2<\/code><\/p>\n
            Thus the numeric IDs of the two Vulkan-capable GPUs are now 1 and 2.<\/p>\n<\/div><\/div>\n
            The compiler will produce the main application executable (a.out<\/code>) and several SPIR-V kernel binaries, one for each OpenMP target<\/code> construct in the\u00a0application.<\/p>\n
            <\/span>Sample OpenMP application utilizing the GPU<\/div>
            \n
            Here is a sample application that can be used to verify the\u00a0successful deployment of the vulkan<\/code>\u00a0module:<\/p>\n
            #include<\/strong> <stdio.h>\r\n#include<\/strong> <omp.h>\r\n\r\nint<\/strong> main(void<\/strong>)\r\n{\r\n    \/* \r\n     * Expected result (assuming that the GPU device ID is 0):\r\n     * Running on Vulkan device \r\n     *\/\r\n    int<\/strong> is_initial_device = 1;\r\n    #pragma omp target map(tofrom:is_initial_device) device(0)<\/span><\/strong>\r\n    {<\/span>\r\n        is_initial_device<\/span> =<\/span> omp_is_initial_device<\/span>();<\/span>\r\n    }\r\n<\/span><\/pre>\n
                printf<\/strong>(\"Running on %s.\\n\", (<\/span>is_initial_device == 0) ? \r\n                               \"Vulkan device\" : \"host\");<\/span><\/pre>\n
                return<\/strong> 0; \r\n} <\/span><\/pre>\n<\/div><\/div>\n
            <\/span>Advanced usage: Multiple compilation jobs<\/div>
            \n
            When an\u00a0application contains several target<\/code> regions, compilation times may become substantial. One can generate multiple compilation jobs that handle the compilation of different kernels in parallel, by using the\u00a0-j<number><\/code> device option. For example to have 4 parallel kernel compilation jobs, use ompicc<\/code> as follows:<\/p>\n
            ompicc --devs=all --devopt -j4<\/strong> app.c<\/pre>\n<\/div><\/div>\n
            Limitations<\/h2>\n
            Currently, the vulkan<\/code> module has a number of limitations:<\/p>\n
              \n
            1. Structs are allowed, but the above restriction also applies to struct fields,<\/li>\n
            2. Only static loop scheduling is supported in target teams distribute parallel for<\/code> directives,<\/li>\n
            3. Pointers and conditional jumps are not allowed in the offloaded kernels.<\/li>\n<\/ol>\n
              Notes<\/h2>\n
              The OMPi vulkan<\/code> module has been tested with the following GPUs:<\/p>\n
                \n
              • NVIDIA Ampere A2<\/li>\n
              • NVIDIA Tesla P40<\/li>\n
              • AMD RX 550<\/li>\n
              • Intel UHD 770<\/li>\n
              • VideoCore VII (Raspberry Pi 5)<\/li>\n
              • VideoCore VI (Raspberry Pi 4)<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"
                Introduction The latest version of OMPi enables offloading to Vulkan-supported GPUs through the vulkan module. OMPi analyzes the OpenMP target directives appearing in a user application and generates kernel source files, written in\u00a0shader language (GLSL). The vulkan module can: Compile the shader files to SPIR-V binaries, Optimize the generated binaries in terms of size and […]<\/p>\n","protected":false},"author":4,"featured_media":0,"parent":120,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1350","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/pages\/1350","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/comments?post=1350"}],"version-history":[{"count":16,"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/pages\/1350\/revisions"}],"predecessor-version":[{"id":1431,"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/pages\/1350\/revisions\/1431"}],"up":[{"embeddable":true,"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/pages\/120"}],"wp:attachment":[{"href":"https:\/\/paragroup.cse.uoi.gr\/wpsite\/wp-json\/wp\/v2\/media?parent=1350"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}