vulkan_best_practice_for_mobile_developers

Appropriate use of surface rotation

Overview

Mobile devices can be rotated, therefore the logical orientation of the application window and the physical orientation of the display may not match. Applications then need to be able to operate in two modes: portrait and landscape. The difference between these two modes can be simplified to just a change in resolution. However, some display subsystems always work on the “native” (or “physical”) orientation of the display panel. Since the device has been rotated, to achieve the desired effect the application output must also rotate.

In OpenGL ES the GPU driver can transparently handle the logical rotation of window surface framebuffers, but the Vulkan specification has made this explicit in the API. Therefore in Vulkan the application is responsible for supporting rotation.

In this sample we focus on the rotation step, and analyse the performance implications of implementing it correctly with Vulkan.

Pre-rotation

Pre-rotation

The rotation step can be carried out in different ways:

An application has no means to tell whether the current device can support a free rotation during composition, so the only guaranteed method to avoid any additional processing cost is to render into a window surface which is oriented to match the physical orientation of the display panel, thus removing the Android compositor step.

Demo application

The sample application you can find here shows how you can handle rotations in your Vulkan application in a way that avoids using the Android compositor for rotation. It allows you to enable and disable pre-rotation at run time, so you can compare these two modes using the hardware counters shown on the display. In this section we will go through the code required to carry out pre-rotation. In the analysis section below we will explain the differences in more detail. Note that not all devices will show obvious differences, as more and more include a DPU capable of performing the rotation in hardware.

In a nutshell, below are the steps required to handle pre-rotation:

No pre-rotation Pre-rotation
Destroy the Vulkan framebuffers and the swapchain Destroy the Vulkan framebuffers and the swapchain
Re-create the swapchain using the new surface dimensions i.e. the swapchain dimensions match the surface’s. Ignore the preTransform field in VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR. This will not match the value returned by vkGetPhysicalDeviceSurfaceCapabilitiesKHR and therefore the Android Compositor will rotate the scene before presenting it to the display Re-create the swapchain using the old swapchain dimensions, i.e. the swapchain dimensions do not change. Update the preTransform field in VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR so that it matches the currentTransform field of the VkSurfaceCapabilitiesKHR returned by the new surface. This communicates to Android that it does not need to rotate the scene.
Re-create the framebuffers Re-create the framebuffers
n/a Adjust the MVP matrix so that: 1) The world is rotated, 2) The field-of-view (FOV) is adjusted to the new aspect ratio
Render the scene Render the scene

Rotation in Android

Android added pre-rotation to their Vulkan Design Guidelines. However, by default, Android calls onDestroy when the screen is rotated. To disable this behavior and handle rotations in Vulkan, you must add the orientation (for API level 13 and lower) and screenSize attributes to the activity’s configChanges in the Android manifest:

<activity android:name=".BPNativeActivity"
          android:configChanges="orientation|screenSize">

To track orientation changes, use Android’s APP_CMD_CONTENT_RECT_CHANGED event:

void on_app_cmd(android_app *app, int32_t cmd)
{
	auto platform = reinterpret_cast<AndroidPlatform *>(app->userData);
	assert(platform && "Platform is not valid");

	switch (cmd)
	{
		case APP_CMD_INIT_WINDOW:
		{
			platform->get_window().resize(ANativeWindow_getWidth(app->window), ANativeWindow_getHeight(app->window));
			app->destroyRequested = !platform->prepare();
			break;
		}
		case APP_CMD_CONTENT_RECT_CHANGED:
		{
			// Get the new size
			auto width  = app->contentRect.right - app->contentRect.left;
			auto height = app->contentRect.bottom - app->contentRect.top;
			platform->get_app().resize(width, height);
			platform->get_window().resize(width, height);
			break;
		}
	}
}

Swapchain re-creation

We need to sample the current transform from the surface:

VkSurfaceCapabilitiesKHR surface_properties;
VK_CHECK(vkGetPhysicalDeviceSurfaceCapabilitiesKHR(get_device().get_physical_device(),
                                                   get_surface(),
                                                   &surface_properties));

pre_transform = surface_properties.currentTransform;

currentTransform is a VkSurfaceTransformFlagBitsKHR value. When we re-create the swapchain, we must set the swapchain’s preTransform to match this value. This informs the compositor that the application has handled the required transform so it does not have to.

To re-create the swapchain, the sample uses the helper function update_swapchain provided by the framework:

get_device().wait_idle();

auto surface_extent = get_render_context().get_surface_extent();

get_render_context().update_swapchain(surface_extent, select_pre_transform());

This function then takes care to safely destroy the framebuffers and use the new preTransform value to re-create the swapchain:

device.get_resource_cache().clear_framebuffers();

auto width  = extent.width;
auto height = extent.height;
if (transform == VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR || transform == VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR)
{
	// Pre-rotation: always use native orientation i.e. if rotated, use width and height of identity transform
	std::swap(width, height);
}

swapchain = std::make_unique<Swapchain>(*swapchain, VkExtent2D{width, height}, transform);

Note that if pre-rotation is enabled and the application has been rotated by 90 degrees, then the surface dimensions must be swapped with respect to the previous orientation. This is done to preserve the dimensions of the swapchain images, since we are planning to rotate our geometry accordingly.

The framework then takes care to re-create the framebuffers.

Rotating the scene

When rotating our geometry, normally all we need to do is adjust the Model View Projection (MVP) matrix that we provide to the vertex shader every frame. In this case we want to rotate the scene just before applying the projection transformation. Therefore we update the matrix that the camera will use to compute the projection matrix:

glm::mat4   pre_rotate_mat = glm::mat4(1.0f);
glm::vec3   rotation_axis  = glm::vec3(0.0f, 0.0f, -1.0f);
const auto &swapchain      = get_render_context().get_swapchain();

if (swapchain.get_transform() & VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR)
{
	pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(90.0f), rotation_axis);
}
else if (swapchain.get_transform() & VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR)
{
	pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(270.0f), rotation_axis);
}
else if (swapchain.get_transform() & VK_SURFACE_TRANSFORM_ROTATE_180_BIT_KHR)
{
	pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(180.0f), rotation_axis);
}

camera->set_pre_rotation(pre_rotate_mat)

The camera uses this transformation when returning the view matrix:

glm::mat4 Camera::get_view()
{
	if (!node)
	{
		throw std::runtime_error{"Camera component is not attached to a node"};
	}

	auto &transform = node->get_component<Transform>();
	return pre_rotation * glm::inverse(transform.get_world_matrix());

This way the framework will use the updated matrix before pushing the MVP to the shader:

void GeometrySubpass::update_uniform(CommandBuffer &command_buffer, sg::Node &node, size_t thread_index)
{
	GlobalUniform global_uniform;

	global_uniform.camera_view_proj = vkb::vulkan_style_projection(camera.get_projection()) * camera.get_view();

For completion, here are the relevant sections of the vertex shader:

layout(location = 0) in vec3 position;

layout(set = 0, binding = 1) uniform GlobalUniform {
    mat4 model;
    mat4 view_proj;
    vec3 camera_position;
} global_uniform;

layout (location = 0) out vec4 o_pos;

void main(void)
{
    o_pos = global_uniform.model * vec4(position, 1.0);
    gl_Position = global_uniform.view_proj * o_pos;;
}

As well as rotating the scene, we also need to adjust the Field-of-view (FOV) used by the camera to calculate the projection matrix. The FOV is the extent of the observable world that can be seen, and it can be broken down into a horizontal component and a vertical component. To calculate the projection matrix, we use the aspect ratio, the FOV, a far clipping plane and a near clipping plane. Keeping the other factors constant, increasing the FOV for a given camera position results in a ‘zoom out’ effect. Similarly, decreasing the FOV results in a ‘zoom in’ effect. Therefore, if the aspect ratio changes, in order to keep the same ‘zoom level’, the FOV must be adjusted accordingly. Since rotating the screen is effectively a swap of width and height, having set a particular horizontal FOV in a landscape display means that we need to use the corresponding vertical FOV in a portrait display. The derivation of the formula used can be found below:

FOV components

In the framework:

float PerspectiveCamera::get_field_of_view()
{
	/* Calculate vertical fov */
	auto vfov = static_cast<float>(2 * atan(tan(fov / 2) * (1.0 / aspect_ratio)));

	return aspect_ratio > 1.0f ? fov : vfov;
}

glm::mat4 PerspectiveCamera::get_projection()
{
	return glm::perspective(get_field_of_view(), aspect_ratio, near_plane, far_plane);
}

Performance impact

The surface_rotation Vulkan sample allows you to toggle between pre-rotation mode and compositor mode. Below is a screenshot of the sample running on a device that does not support native (DPU) rotation, but instead includes a separate 2D block which rotates the GPU output before presenting it to the display.

Android compositor handling the rotation

Compare this to the same scene rendered using pre-rotation:

Pre-rotating the scene

As you can see there is a significant increase in the stall rate on the external memory bus if pre-rotation is not enabled, because the framebuffer is being read and written to the 2D rotation block. For this device the additional system memory bandwidth generated by the 2D block increases the use of external memory, which is visible as an increase in memory back-pressure seen by the GPU.

This is more obvious if we trace both modes using Streamline. If you enable all Mali counters and use the relevant template (Mali-G72 in this case) to visualize the data, we can see that we go from an average 12% read stall / 7% write stall to 22% read stall / 17% write stall. In the image below pre-rotation is enabled and disabled every second (using the auto-toggle option). The absolute traffic per cycle drops, but this is because of the drop in performance associated to the increased memory pressure.

Streamline capture. Pre-rotate is enabled/disabled every second

In this case the 2D rotation block is using a significant portion of the bandwidth, causing a drop in performance. Note however that this scene is rendered in a memory-heavy fashion (no culling, no compressed textures) to make the effect of pre-rotation more visible. Even if your scene is not memory-heavy, the extra load on the system resulting from performing the rotation during composition will have a negative impact on the battery life of the device.

In order to save battery life in those devices without a rotation-capable DPU, always ensure that your Vulkan renderer performs pre-rotation.

Best-practice summary

Do

Don’t

Impact

Debugging

This site is open source. Improve this page.