• Launchpad Entry: arm-m-graphics-stack-on-x

• Created:

• Contributors: Alexandros Frantzis

• Packages affected:

Summary

Drive the effort to improve the X graphics stack on ARM, with emphasis on 3D features. Achieve this by defining a clear common goal for all involved teams: get the Unity UI to run smoothly on the ubuntu-on-arm platform on the vendors' hardware.

Rationale

The user experience is one of the most important aspects of any product and vendors are really keen on providing smooth and rich user interfaces. This requires that the proper functionality and performance can be delivered by the underlying graphics stack (on X in this case).

Design

We need to make sure that they graphics stack on X can deliver the features and performance needed to support a modern, rich UI experience (using the Unity UI as an example).

• Package and put in the archive builds of Clutter and Unity with OpenGL ES support.
• Coordinate with DXTeam and vendors to ensure Unity is working well with their stacks.

Implementation

• Mesa EGL/OpenGL
  • Package and make available in the archive a build of Mesa with EGL/OpenGL support.

• Package and make available in the archive an OpenGL ES->OpenGL wrapper: git://gitorious.org/qemu-maemo/gles-libs.git

• Clutter OpenGL ES
  • Investigate the feasibility of having Clutter use runtime loadable plugins for various backends (OpenGL, OpenGL ES 2.0)
    • If plugins are feasible, package and make available in the archive plugins for a Clutter OpenGL and OpenGL ES 2.0 backends.
    • Provide a mechanism for loading the correct plugin depending on the platform.
  • Package and make available in the archive a build of Clutter with OpenGL ES 2.0 support (either using plugins or a separate build for that backend).
• Unity
  • Work with Unity team to make Unity work with Clutter OpenGL ES (eg eliminate direct GL calls)
  • Package and make available in the archive a build of Unity that works with OpenGL ES
  • Coordinate the efforts of DX Team and vendors to
    • produce a specification of minimum driver/hardware requirement to support Unity on ARM
    • solve any issues related to the functionality or performance of their stacks, using Unity as a "benchmark"

BoF agenda and discussion

Topics

• requirements for graphics drivers for a good X11 experience
  • version alignment
• performance benchmarking on X
• 3D story on X and arm
• video/multimedia on X and arm

Driver Requirements for X

• Kernel driver:
  • Compatibility with kernel 2.6.32 (Lucid) or higher.
  • KMS (Kernel Mode Setting) - hard requirement
    • no transitioning resolution of the bootsplash and X
    • seamless visual experience from bootsplash to X
    • Mode setting happens in the LCD driver
    • faster and more reliable suspend/resume

  • Compatibility with the userspace libdrm 2.4.18 (Lucid) or higher -> usually backwards compatible -- no general need version alignment

    • separation of LVDS control and GPU (due to licencing of external GPU tech) makes DRM/KMS integration harder
  • Support for RandR output hotplug events (so that the xserver receives an event when external screens are connected or disconnected)
  • GEM (Graphics Execution Manager), TTM (Translation Table Maps) or an alternative kernel video memory manager
    • needed for DRI2 to work (aka accelerated redirected/offscreen 3D/GL)
    • needed for OpenGL extensions like FBOs (GL_ARB_framebuffer_object)
    • GEM is usually safe wrt backwards compatiblility
    • GLX_texture_from_pixmap needed because of the window-manager/compositor mutter used by unity
    • XDamage, XFixes, XComposite (might be needed to be supported on the driver side)
    • TTM can be looked at as an implementation detail, as a subset of GEM support
  • Dynamic front buffer resizing as in -intel's UXA (so that we don't have to set the virtual resolution and restart X when dealing with external screens using high resolutions)
• DDX (X.Org) driver:
  • Compatibility with xserver 1.7.x (Lucid) or higher (1.8.x)
    • needs to be aligned version wise
  • Support for RandR 1.3 (especially transformations, scaling, etc.)

    • version does not change frequently -- minimum should be >= 1.2 to get good multi head support

    • current development is ongoing for xinerama support
  • Use of the E-EDID functions in X.org (this is useful for HDMI)
    • currently not well supported as typically arm devices have been directly connected to the display
    • likely will need some other mechanism to get the resolution choices to xrandr
    • currently the resolution is hard coded into the kernel
  • DRI 2 (Direct Rendering Infrastructure) which allows GLX applications to do direct rendering to redirected windows. This makes OpenGL and XV (X video extension) work correctly with compositing managers.
• What is needed to get fast graphics to 'clutter/mutter' level??
  • seems to be a limitation that the graphics drivers are mostly OpenGL ES (sp?) which is a sub-set of that required by these layers
  • graphics performance is also suspected to be lower, particularly compositing off screen is slow
  • one option may be to require full OpenGL drivers and exclude OpenGL ES
    • OpenGL is a software API though; doesn't imply any minimum level of hardware support
    • concerns expressed that requiring too high a level will blow the power budget for some use cases
• Multimedia:
  • Support for either VA-API (Video Acceleration API) or VDPAU (Video Decode and Presentation API for Unix) to allow video programs to offload portions of the video decoding process and video post-processing to the GPU video-hardware.
    • does not really matter to use gstreamer
• 3D driver:
  • OpenGL 2.1 (or later) compliant driver, more specifically we would like to have the following extensions available:
    • GL_ARB_framebuffer_object
    • GL_ARB_vertex_program
    • GL_ARB_fragment_program
    • GL_ARB_texture_non_power_of_two
    • GL_EXT_stencil_two_side
    • GL_ARB_vertex_buffer_object
  • Support for GLSL
  • chromium has bad performance with opengles even with
  • minimum hardware requirements
    • opengl hardware with opengles 2.0 or better

Performance / Benchmarking

Goals:

• Provide a way to easily test and benchmark their hardware and drivers
  • Custom Images with Benchmarks being auto run
  • Report results to a database
• Provide easy way to demo graphics capabilities of devices
  • Full desktop
  • Benchmarks

Performance tests:

• 3D tests
  • phoronix-test-suite (openGL based tests)
    • both performance and correctness tests, is already in the archive for use
    • there are not any OpenGL ES specifc tests, though there are some at Intel not yet committed to the repo
      • will be needed for this effort
    • some of this is available from Khronos group perhaps
    • [Jay] To write some simple tests to test the OpenGL ES 2.0 implementation . Tests to cover what Unity needs as a first step

    • unity relevant tests -> dx team could develop test suites

    • powervr published a testsuite for opengles included in their SDK

    • webos seems to have some simple test tool under http://www.webos-internals.org/wiki/GLES_1.1_3D_Cube_Demo

  • games: needs porting to opengles --
• 2D tests
  • gtkperf (to test performance with GTK+)

  • cairo-perf-trace (to test performance of apps which use cairo e.g. firefox) http://cworth.org/intel/performance_measurement/

    • make it easy to select the backend for validation: opengl, xrandr, etc.

    • a good game to test OpenGL ES is ioquake3 (ACTION: investigate/package up) - http://github.com/Cpasjuste/quake3_pandora_gles

Discussion/Notes Session 1

Discussion notes inline above.

Primary deliverables

• clear documentation of the expected version compatibility levels to have good performance on Ubuntu
• [Jay Taoko] good test suites to confirm performance and expose choke points
  • allow easy running of those benchmarks with automatic result
• maybe package opegles games for showcasing the hardware
• Conduct a survey of the current partners graphical rendering models (immediate vs. deferred)

Discussion/Notes Session 2

• Unity should be ported to OpenGL ES pretty easily.
• Plan is to support OpenGL ES 2.0.
• Mutter is to be built with ES 2.0.
  • Custom stuff in Unity and Mutter make it not compatable with ES 2.0 yet.
• Clutter in Lucid doesn't work with OpenGL ES yet.
• No more than a week to make Unity ES 2.0 compatable. Would need testing afterwards.
• We need:
  • Clutter
• Blending is a bottle neck (confirm with Yuval)
• Step 1 get the drivers in place

• Step 2 get the test cases run on all available SoCs

• Step 3 decide what to change in clutter or the driver to meet the performance required
• Some Parts in GTK require XRandR otherwise they fail Unity OpenGL ES extension required
  • Framebuffer Objects
  • Non power of two textures
  • GLSL shader support (included in OpenGL ES 2.0)
  • Rectangle textures
  ARM Graphics

GL/ES                                                |           GL
                                                     |
    GPU                                              |       GPU
      |                                              |         |
      |                                              |         V
      |                                              |     DRI/DRM             NEEDS
      |                                              |         |               TO CHG
      V                                              |         V                FROM
F    DRIVER                                          |      DRIVER        VENDOR OF SOC
R   |                                                |         |                    
O   V                                                |         V
M   OPENGL/ES                                        |     X11/XORG
V        /EGL                                        |       |
E     |                                              |       V
N     |                                              |     GLX
D     |                                              |      |
O     | X11                                          |      |
R     |                                              |      |
      V                                              |      V
  CLUTTER(ES)                                        |  CLUTTER(GL)
      |                                              |     |
      V                                              |     V
    UNITY                                            |    UNITY

Action Points

• Jay/Neil/Alexander Sack: Will make Clutter with OpenGL ES 2.0 available in a PPA for Lucid
• Jay: Will fix unity so it doesn't depend on Open GL only to make it compatible with OpenGL ES
• Anmar: Discuss setting up a mailing list or something to communicate
• David Barth: to outline the acceleration requirements. Similar to Google's hardware requirements
• Neil: to find out what features required in XRandR. Graham said it is very simple to put in a basic XRandR

---

CategorySpec