Mipmapping in Vulkan

Introduces how to use mipmaps and how to generate them from a source image.

Using mipmaps in Vulkan: the quad on the right shows the mip level used for each highlighted quad on the left

Note

The source for this sample can be found in samples/mipmapping in the SDK.

Introduction

This sample builds on the Rotating Texture sample. It implements mipmapping in two ways, by loading pre-scaled images and by generating mipmaps from a single image. The mip levels for each case are displayed as shown in the image above.

In this tutorial we will go through the steps needed to implement both mipmapping approaches.

Loading a mipmapped texture from pre-scaled images

Let us start with the simplest case, in which we want to create a mipmapped texture by importing already scaled images. For that purpose, we will modify the createTextureFromAsset function into createMipmappedTextureFromAssets, which accepts a vector of paths instead of a single one.

It is convenient to store the data for each mip level that we will load in a struct, MipLevel. The first part of the function is then a straightforward modification from the original one, just by iterating through the images we want to load:

vector<MipLevel> mipLevels;
unsigned mipLevelCount;

for (auto &pPath : pPaths)
{
        MipLevel mipLevel;

        if (FAILED(loadRgba8888TextureFromAsset(pPath, &mipLevel.buffer, &mipLevel.width, &mipLevel.height)))
        {
                LOGE("Failed to load texture from asset.\n");
                abort();
        }

        // Copy commands such as vkCmdCopyBufferToImage will need TRANSFER_SRC_BIT.
        mipLevel.stagingBuffer = createBuffer(mipLevel.buffer.data(), mipLevel.width * mipLevel.height * 4, VK_BUFFER_USAGE_TRANSFER_SRC_BIT);

        mipLevels.push_back(mipLevel);
}

// Get the number of mip levels based on the number of loaded sources.
mipLevelCount = mipLevels.size();

Small changes are necessary to the structures for creating the VkImage and the VkImageView, which need to be aware of the number of mip levels.

We can now move to actually loading data from the staging buffers to the mip levels of the texture. Please note that in the final version the first mip level (i.e. the iteration with i = 0) is isolated; this will be useful when generating mipmaps, but for the time being we can simplify the code.

for (unsigned i = 0; i < mipLevelCount; i++)
{
        VkBufferImageCopy region = {};
        region.bufferOffset = 0;
        region.bufferRowLength = mipLevels[i].width;
        region.imageSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
        region.imageSubresource.mipLevel = i;
        region.imageSubresource.layerCount = 1;
        region.imageExtent.width = mipLevels[i].width;
        region.imageExtent.height = mipLevels[i].height;
        region.imageExtent.depth = 1;

        // Copy each staging buffer to the appropriate mip level of our optimally tiled image.
        vkCmdCopyBufferToImage(cmd, mipLevels[i].stagingBuffer.buffer, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &region);
}

The function vkCmdCopyBufferToImage (already used in Rotating Texture) is able to copy data to specific mip level of the destination image, just by specifying it in the region structure. No memory barrier is needed between such copies, but we must keep the barrier at the end to ensure that no fragment shading is done while we are copying the texture from the staging buffers.

The rest of the function requires few other minor modifications, such as freeing all the temporary resources and passing the maximum LOD to the sampler.

Everything is ready, we can now call our new function to load a mipmapped texture:

// Load texture with pre-generated mipmaps.
vector<char const *> pPaths = { "textures/T_Speaker_512.png", "textures/T_Speaker_256.png", "textures/T_Speaker_128.png",
                                "textures/T_Speaker_64.png",  "textures/T_Speaker_32.png",  "textures/T_Speaker_16.png",
                                "textures/T_Speaker_8.png",   "textures/T_Speaker_4.png",   "textures/T_Speaker_2.png",
                                "textures/T_Speaker_1.png" };
textures[0] = createMipmappedTextureFromAssets(pPaths);

Generating mipmaps in Vulkan

We have seen the case in which we had already generated our mipmaps, but what if we only have a single image?

There is no built-in support for generating mip-maps in Vulkan, but we can use vkCmdBlitImage to achieve the same result. Here is an example, similar to the previous one, with generated mipmaps:

Automatically generated mipmaps

Given the code we have already written, we can implement mipmap generation just by changing a few things. Our function, createMipmappedTextureFromAssets, will now accept an optional parameter generateMipLevels: if true, mip levels will be generated from the first item in pPaths.

The first change is related to mipLevelCount, which must now be computed rather than obtained from the size of the input vector:

if (generateMipLevels)
{
        // Get the number of mip levels to be generated, based on the size of the source.
        mipLevelCount = floor(log2(float(min(mipLevels[0].width, mipLevels[0].height)))) + 1;
}

The size of each mip level is obtained by cutting in half each dimension of the previous level, until one of the dimensions hits 1; thus the number of mip levels is the base 2 logarithm of the smaller dimension of the input image.

We will then need to add VK_IMAGE_USAGE_TRANSFER_SRC_BIT to the usage flags for the image, as we will use the image itself as a source to generate mipmaps.

info.usage = (generateMipLevels ? VK_IMAGE_USAGE_TRANSFER_SRC_BIT : 0) | VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT;

We will generate each mip level by scaling the previous one in half. Before we can start, though, we need to load the first mip level to the image in the usual way:

VkBufferImageCopy region = {};
region.bufferOffset = 0;
region.bufferRowLength = mipLevels[0].width;
region.imageSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
region.imageSubresource.mipLevel = 0;
region.imageSubresource.layerCount = 1;
region.imageExtent.width = mipLevels[0].width;
region.imageExtent.height = mipLevels[0].height;
region.imageExtent.depth = 1;

// Copy the buffer for the first mip level to our optimally tiled image.
vkCmdCopyBufferToImage(cmd, mipLevels[0].stagingBuffer.buffer, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &region);

// Transition first mip level into a TRANSFER_SRC_OPTIMAL layout.
// We need to wait for first CopyBuffer to complete before we can transition away from TRANSFER_DST_OPTIMAL,
// so use VK_PIPELINE_STAGE_TRANSFER_BIT as the srcStageMask.
imageMemoryBarrier(cmd, image, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT,
                   VK_PIPELINE_STAGE_TRANSFER_BIT, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
                   0, 1);

Everything looks the same as before, apart from the last part: this is a critical step, as we won't be able to execute transfer commands unless the texture is in the appropriate layout. Specifically, we need to transition the first mip level from VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL to VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL; this way we will be able to use it as a source to generate the next level.

We will then loop through the mip levels and use vkCmdBlitImage to generate each level based on the previous one:

VkImageBlit region = {};
region.srcSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
region.srcSubresource.mipLevel = i - 1;
region.srcSubresource.layerCount = 1;
region.srcOffsets[1].x = max(mipLevels[0].width >> (i - 1), 1u);
region.srcOffsets[1].y = max(mipLevels[0].height >> (i - 1), 1u);
region.srcOffsets[1].z = 1;
region.dstSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
region.dstSubresource.mipLevel = i;
region.dstSubresource.layerCount = 1;
region.dstOffsets[1].x = max(mipLevels[0].width >> i, 1u);
region.dstOffsets[1].y = max(mipLevels[0].height >> i, 1u);
region.dstOffsets[1].z = 1;

// Generate a mip level by copying and scaling the previous one.
vkCmdBlitImage(cmd, image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &region, VK_FILTER_LINEAR);

The x and y components of srcOffsets[1] and dstOffsets[1] correspond to the width and height of each mip level, which we can get by shifting the width and height of the base image. We also want to ensure that each dimension is greater or equal than 1.

This time we have dependencies between each iteration of the loop, as the previous blit must be complete before we can start the next one. In order to enforce this dependency, we will need to add memory barriers within the loop.

After the blit operation we will not need the previous mip level anymore, so we can transition it to VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL. We also need to transition the current level to VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, so that it can be used as a source for the next iteration. In the last iteration, we can just transition the current level to VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL.

// Transition the previous mip level into a SHADER_READ_ONLY_OPTIMAL layout.
imageMemoryBarrier(cmd, image, VK_ACCESS_TRANSFER_READ_BIT, VK_ACCESS_SHADER_READ_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT,
                   VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
                   i - 1, 1);

if (i + 1 < mipLevelCount)
{
        // Transition the current mip level into a TRANSFER_SRC_OPTIMAL layout, to be used as the source for the next one.
        imageMemoryBarrier(cmd, image, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT,
                           VK_PIPELINE_STAGE_TRANSFER_BIT, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
                           i, 1);
}
else
{
        // If this is the last iteration of the loop, transition the mip level directly to a SHADER_READ_ONLY_OPTIMAL layout.
        imageMemoryBarrier(cmd, image, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_SHADER_READ_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT,
                           VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
                           i, 1);
}

Nothing else is required, so we can now load a texture and generate mipmaps for it:

textures[1] = createMipmappedTextureFromAssets({ "textures/T_Pedestal_512.png" }, true);