docs/DISPLAY-STATE.TXT - emulator-unstable-refactoring - Git at Google

 This tries to document the mess that is DisplayState in QEMU.
 See "console.h" for the main definitions, and below for some
 explanations:


 DISPLAY STATE OBJECTS:
 ======================

 A DisplayState holds state for stuff to be displayed on QEMU. More
 precisely:

  - A DisplayState owns a 'DisplaySurface' which is nothing more than a
    pixel buffer with specific dimensions, pitch and format plus bytes
    to carry its content.

  - A DisplayState also holds a 'DisplayAllocator' which allows it to
    allocate its surface through a proper API. For example, this is
    used in the upstream sdl UI backend to allocate the surface pixels
    through SDL_SetVideoMode(). The default allocator simply uses
    'malloc' to do the allocation (with 32-bits/pixel).

  - A DisplayState also holds a list of DisplayChangeListener object.
    Each listener contains a small set of callbacks that will be called
    whenever an "event" happens on the display state. Events examples
    are:

       dpy_update:  a rectangular portion of the surface has been updated.
       dpy_resize:  the hardware decided to resize the framebuffer.
       dpy_refresh: called periodically by the GUI timer.
       dpy_copy:    the hardware performed a rectangular copy operation.
       dpy_fill:    the hardware performed a rectangular fill operation.
       dpy_setdata: the hardware decided to change the framebuffer address.
       dpy_text_cursor: the hardware placed the text cursor at a given (x,y).

    NOTE: dpy_setdata is essentially the same than dpy_resize except that
          there is a guarantee that the size didn't change.

    More on DisplayChangeListeners below.

  - The file "console.h" provides many helper functions to call all listeners
    registered for a given DisplayState. For example, dpy_update(ds,x,y,w,h)
    will call the 'dpy_update' callback of all listeners for the display
    state 'ds'.


 CONSOLES:
 =========

 A "console" is something that can write pixels into a DisplayState.
 There are two kinds of consoles, and they're fairly different in usage.

   GRAPHICAL CONSOLE:
   ------------------

   A "Graphical console" creates and owns a DisplayState. It is used when one
   needs to write directly to the DisplaySurface pixel buffer. A typical
   hardware framebuffer emulator (e.g. hw/vga-pic.c) will call the
   function graphic_console_init() to create the DisplayState. Note that
   this functions accepts several callbacks and is defined as:

         typedef void (*vga_hw_update_ptr)(void *);
         typedef void (*vga_hw_invalidate_ptr)(void *);
         typedef void (*vga_hw_screen_dump_ptr)(void *, const char *);
         typedef void (*vga_hw_text_update_ptr)(void *, console_ch_t *);

         DisplayState *graphic_console_init(vga_hw_update_ptr update,
                                         vga_hw_invalidate_ptr invalidate,
                                         vga_hw_screen_dump_ptr screen_dump,
                                         vga_hw_text_update_ptr text_update,
                                         void *opaque);

   The update/invalidate/screen_dump/text_update functions must be provided
   by the hardware framebuffer emulation, and will be called under various
   circumstances:

       'update' is called periodically to check for any hw framebuffer
       updates (and then copy them to the DisplayState, to finally send
       them through dpy_update() to the listeners).

       'invalidate' is called to indicate that the next call to 'update'
       should send the content of _all_ the framebuffer, instead of only
       the smallest possible update.

       'screen_dump' is called to force a screen dump (i.e. print the
       content of the framebuffer to a ppm file, which name is passed as
       a parameter).

       'text_update' is called to display one single character. XXX: Code is
       not very clear, but probably related to text console.


   TEXT CONSOLES:
   --------------

   A "Text console" attaches to an existing DisplayState, and is able to
   take over its rendering in order to display a text *terminal*. It's not
   sure whether this emulates VT101 or something else (see the code inside
   the console_putchar() for all the gory details), but the main idea is
   that you create a console with a call to:

         CharDriverState*  text_console_init(const char* p);

   The function returns a CharDriverState* (see docs/CHAR-DEVICES.TXT) that
   will be connected to a host device identified by the string in 'p'. This
   allows you, for example, to connect the console to stdio.

   The text console code is capable of producing a bitmap each time you update
   its content (i.e. it includes code to manage fixed-size font rendering,
   scrolling, escape sequences, color, blinking cursor, etc...).

   - By default, the graphics console writes to its DisplayState, but you can
     use console_select() to change that at runtime. This function can be used
     to force switching between virtual terminals and the graphics display.
     There can be several text consoles associated to a single DisplayState
     object.


 DISPLAY CHANGE LISTENERES:
 ==========================

 There QEMU sources provide the implementation for various
 DisplayChangeListeners, most notables are the following:

  - In sdl.c: This one uses the SDL library to display the content of a
    DisplaySurface through a SDL_Window. The implementation also supports
    zooming the output to an arbitrary size (using SDL functions).

  - In vnc.c: This listener implements a VNC Server that can be used to
    display the DisplaySurface remotely through the RDP protocol.

  - In curses.c: This listener is used to display text consoles through the
    "curses" library on Unix systems. It cannot be used to display any
    graphics though.

 NOTE: The initialization sequence in vl.c only allows for a single listener
 on the main display state, but the rest of the code deals with several
 listeners per DisplayState just fine.

 Each DisplayChangeListener can specify a refresh period (e.g. every 1/60th
 of second). QEMU will then create a timer that will be programmed to called
 the listener's 'dpy_refresh' callback periodically. The point of this
 callback is to perform the following:

 - poll for new user input events from the underlying UI (e.g. from the SDL
   event loop, or from the network for VNC). These should be translated into
   guest event codes with functions like 'kbd_put_keycode' or 'kbd_mouse_event'.

 - call the global vga_hw_update() function. It will, if the graphics console
   is being displayed, call the 'update' callback that was passed to
   graphic_console_init(). If a text console is being displayed, the does
   nothing.

 - eventually call the global vga_hw_invalidate() to indicate that the whole
   framebuffer content should be resent as an update. This can happen when a
   UI window was minimized and is made visible again, for example.


 INITIALIZATION AND RUNTIME EXECUTION:
 =====================================

 Initialization happens in the qemu main() function in the vl.c source file.

 First, the hardware machine is initialized. The hardware fraembuffer emulation
 shall call graphic_console_init() to create a new DisplayState. Note that the
 object returned by this function has a default DisplaySurface of 640x480 pixels
 allocated through malloc(). In other words, the hardware emulation does not
 set the size of the display state by default!

 After that, the listener's initialization function (e.g. sdl_display_init)
 is called. It is passed the DisplayState object and can replace the
 corresponding DisplaySurface with another one with proper dimensions, and
 eventually created with a different DisplayAllocator. It also registers a
 DisplayChangeListener to receive later state changes.

 Note that the VNC listener doesn't change the dimension of the DisplayState
 surface it is initialized with. However, it will react to dpy_resize events
 accordingly.

 NOTE: dpy_resize()s are currently only generated when switching between
 consoles, or when the framebuffer's size is modified by the guest kernel.


 The GUI timer, corresponding to the first listener than has one refresh
 period, drives the whole update process (if no listener provides a refresh
 period, a default 'no_graphic' timer is setup with a default refresh period
 of 30 frame/s).

 Things happen in this order:

  - the GUI timer kicks in, and calls the 'dpy_refresh()' callback of
    the listener (each listener has its own timer, btw).

  - the listener callback polls for user events, and calls vga_hw_update()
    to see if there are hardware framebuffer updates.

  - vga_hw_update() checks that the graphics console is displayed (otherwise
    it exits)

  - it then calls the graphics console's 'update' callback

  - the callback, implemented by the framebuffer hw emulation, checks for
    dirty pages in order to detect what changed since it was invoked.

    For every rectangle of the hw framebuffer that was modified, it copies
    the pixels from VRAM into the DisplayState's surface buffer (eventually
    performing format conversion at the same time).

    After that, it calls dpy_update() to send the update to all registered
    listeners for the DisplayState.

  - The listener's 'dpy_update' callback is called and receives a pointer
    to the DisplayState, and the rectangle corresponding to the update. Its
    implementation can then update the content of the screen (or the internal
    VNC framebuffer).

 Eventually, hardware emulation can also trigger other dpy_xxxx events (e.g.
 dpy_resize, dpy_copy, dpy_fill....)

 Here's a simplified diagram of what happens in the typical case:

             _____________
             |             |
             |  hardware   |
             | framebuffer |-------+
             |             |       |
             |_____________|       |
                ^                  |
                |                  |
                | 3/ ds.update()   | 4/ dpy_update(ds,x,y,w,h)
                |                  |
                |                  |
             _____________         |
             |             |       |
             |  Display    | <-----+
             |   State     |
             |             | ----------+
             |_____________|           |
                 ^                     |
                 |                     |
                 | 2/ vga_hw_update()  |
                 |                     |
                 |                     |
                 |                     |
                 |     +---------------+
                 |     |               |
                 |     | 5/listener.dpy_update(ds,x,y,w,h)
                 |     |               |
                 |     |               | 6/listener.dpy_update(...)
                 |     |               |
                 |     v               v
              _____________     _____________
             |             |   |             |
             |  SDL        |   |  VNC        |
             |   Listener  |   |   Listener  |
             |             |   |             |
             |_____________|   |_____________|
                 ^
                 |
                 | 1/ listener.dpy_refresh()
                 |

                GUI timer
	This tries to document the mess that is DisplayState in QEMU.
	See "console.h" for the main definitions, and below for some
	explanations:


	DISPLAY STATE OBJECTS:
	======================

	A DisplayState holds state for stuff to be displayed on QEMU. More
	precisely:

	- A DisplayState owns a 'DisplaySurface' which is nothing more than a
	pixel buffer with specific dimensions, pitch and format plus bytes
	to carry its content.

	- A DisplayState also holds a 'DisplayAllocator' which allows it to
	allocate its surface through a proper API. For example, this is
	used in the upstream sdl UI backend to allocate the surface pixels
	through SDL_SetVideoMode(). The default allocator simply uses
	'malloc' to do the allocation (with 32-bits/pixel).

	- A DisplayState also holds a list of DisplayChangeListener object.
	Each listener contains a small set of callbacks that will be called
	whenever an "event" happens on the display state. Events examples
	are:

	dpy_update: a rectangular portion of the surface has been updated.
	dpy_resize: the hardware decided to resize the framebuffer.
	dpy_refresh: called periodically by the GUI timer.
	dpy_copy: the hardware performed a rectangular copy operation.
	dpy_fill: the hardware performed a rectangular fill operation.
	dpy_setdata: the hardware decided to change the framebuffer address.
	dpy_text_cursor: the hardware placed the text cursor at a given (x,y).

	NOTE: dpy_setdata is essentially the same than dpy_resize except that
	there is a guarantee that the size didn't change.

	More on DisplayChangeListeners below.

	- The file "console.h" provides many helper functions to call all listeners
	registered for a given DisplayState. For example, dpy_update(ds,x,y,w,h)
	will call the 'dpy_update' callback of all listeners for the display
	state 'ds'.


	CONSOLES:
	=========

	A "console" is something that can write pixels into a DisplayState.
	There are two kinds of consoles, and they're fairly different in usage.

	GRAPHICAL CONSOLE:
	------------------

	A "Graphical console" creates and owns a DisplayState. It is used when one
	needs to write directly to the DisplaySurface pixel buffer. A typical
	hardware framebuffer emulator (e.g. hw/vga-pic.c) will call the
	function graphic_console_init() to create the DisplayState. Note that
	this functions accepts several callbacks and is defined as:

	typedef void (vga_hw_update_ptr)(void );
	typedef void (vga_hw_invalidate_ptr)(void );
	typedef void (vga_hw_screen_dump_ptr)(void , const char *);
	typedef void (vga_hw_text_update_ptr)(void , console_ch_t *);

	DisplayState *graphic_console_init(vga_hw_update_ptr update,
	vga_hw_invalidate_ptr invalidate,
	vga_hw_screen_dump_ptr screen_dump,
	vga_hw_text_update_ptr text_update,
	void *opaque);

	The update/invalidate/screen_dump/text_update functions must be provided
	by the hardware framebuffer emulation, and will be called under various
	circumstances:

	'update' is called periodically to check for any hw framebuffer
	updates (and then copy them to the DisplayState, to finally send
	them through dpy_update() to the listeners).

	'invalidate' is called to indicate that the next call to 'update'
	should send the content of _all_ the framebuffer, instead of only
	the smallest possible update.

	'screen_dump' is called to force a screen dump (i.e. print the
	content of the framebuffer to a ppm file, which name is passed as
	a parameter).

	'text_update' is called to display one single character. XXX: Code is
	not very clear, but probably related to text console.


	TEXT CONSOLES:
	--------------

	A "Text console" attaches to an existing DisplayState, and is able to
	take over its rendering in order to display a text terminal. It's not
	sure whether this emulates VT101 or something else (see the code inside
	the console_putchar() for all the gory details), but the main idea is
	that you create a console with a call to:

	CharDriverState* text_console_init(const char* p);

	The function returns a CharDriverState* (see docs/CHAR-DEVICES.TXT) that
	will be connected to a host device identified by the string in 'p'. This
	allows you, for example, to connect the console to stdio.

	The text console code is capable of producing a bitmap each time you update
	its content (i.e. it includes code to manage fixed-size font rendering,
	scrolling, escape sequences, color, blinking cursor, etc...).

	- By default, the graphics console writes to its DisplayState, but you can
	use console_select() to change that at runtime. This function can be used
	to force switching between virtual terminals and the graphics display.
	There can be several text consoles associated to a single DisplayState
	object.


	DISPLAY CHANGE LISTENERES:
	==========================

	There QEMU sources provide the implementation for various
	DisplayChangeListeners, most notables are the following:

	- In sdl.c: This one uses the SDL library to display the content of a
	DisplaySurface through a SDL_Window. The implementation also supports
	zooming the output to an arbitrary size (using SDL functions).

	- In vnc.c: This listener implements a VNC Server that can be used to
	display the DisplaySurface remotely through the RDP protocol.

	- In curses.c: This listener is used to display text consoles through the
	"curses" library on Unix systems. It cannot be used to display any
	graphics though.

	NOTE: The initialization sequence in vl.c only allows for a single listener
	on the main display state, but the rest of the code deals with several
	listeners per DisplayState just fine.

	Each DisplayChangeListener can specify a refresh period (e.g. every 1/60th
	of second). QEMU will then create a timer that will be programmed to called
	the listener's 'dpy_refresh' callback periodically. The point of this
	callback is to perform the following:

	- poll for new user input events from the underlying UI (e.g. from the SDL
	event loop, or from the network for VNC). These should be translated into
	guest event codes with functions like 'kbd_put_keycode' or 'kbd_mouse_event'.

	- call the global vga_hw_update() function. It will, if the graphics console
	is being displayed, call the 'update' callback that was passed to
	graphic_console_init(). If a text console is being displayed, the does
	nothing.

	- eventually call the global vga_hw_invalidate() to indicate that the whole
	framebuffer content should be resent as an update. This can happen when a
	UI window was minimized and is made visible again, for example.


	INITIALIZATION AND RUNTIME EXECUTION:
	=====================================

	Initialization happens in the qemu main() function in the vl.c source file.

	First, the hardware machine is initialized. The hardware fraembuffer emulation
	shall call graphic_console_init() to create a new DisplayState. Note that the
	object returned by this function has a default DisplaySurface of 640x480 pixels
	allocated through malloc(). In other words, the hardware emulation does not
	set the size of the display state by default!

	After that, the listener's initialization function (e.g. sdl_display_init)
	is called. It is passed the DisplayState object and can replace the
	corresponding DisplaySurface with another one with proper dimensions, and
	eventually created with a different DisplayAllocator. It also registers a
	DisplayChangeListener to receive later state changes.

	Note that the VNC listener doesn't change the dimension of the DisplayState
	surface it is initialized with. However, it will react to dpy_resize events
	accordingly.

	NOTE: dpy_resize()s are currently only generated when switching between
	consoles, or when the framebuffer's size is modified by the guest kernel.


	The GUI timer, corresponding to the first listener than has one refresh
	period, drives the whole update process (if no listener provides a refresh
	period, a default 'no_graphic' timer is setup with a default refresh period
	of 30 frame/s).

	Things happen in this order:

	- the GUI timer kicks in, and calls the 'dpy_refresh()' callback of
	the listener (each listener has its own timer, btw).

	- the listener callback polls for user events, and calls vga_hw_update()
	to see if there are hardware framebuffer updates.

	- vga_hw_update() checks that the graphics console is displayed (otherwise
	it exits)

	- it then calls the graphics console's 'update' callback

	- the callback, implemented by the framebuffer hw emulation, checks for
	dirty pages in order to detect what changed since it was invoked.

	For every rectangle of the hw framebuffer that was modified, it copies
	the pixels from VRAM into the DisplayState's surface buffer (eventually
	performing format conversion at the same time).

	After that, it calls dpy_update() to send the update to all registered
	listeners for the DisplayState.

	- The listener's 'dpy_update' callback is called and receives a pointer
	to the DisplayState, and the rectangle corresponding to the update. Its
	implementation can then update the content of the screen (or the internal
	VNC framebuffer).

	Eventually, hardware emulation can also trigger other dpy_xxxx events (e.g.
	dpy_resize, dpy_copy, dpy_fill....)

	Here's a simplified diagram of what happens in the typical case:

	_____________
	\| \|
	\| hardware \|
	\| framebuffer \|-------+
	\| \| \|
	\|_____________\| \|
	^ \|
	\| \|
	\| 3/ ds.update() \| 4/ dpy_update(ds,x,y,w,h)
	\| \|
	\| \|
	_____________ \|
	\| \| \|
	\| Display \| <-----+
	\| State \|
	\| \| ----------+
	\|_____________\| \|
	^ \|
	\| \|
	\| 2/ vga_hw_update() \|
	\| \|
	\| \|
	\| \|
	\| +---------------+
	\| \| \|
	\| \| 5/listener.dpy_update(ds,x,y,w,h)
	\| \| \|
	\| \| \| 6/listener.dpy_update(...)
	\| \| \|
	\| v v
	_____________ _____________
	\| \| \| \|
	\| SDL \| \| VNC \|
	\| Listener \| \| Listener \|
	\| \| \| \|
	\|_____________\| \|_____________\|
	^
	\|
	\| 1/ listener.dpy_refresh()
	\|

	GUI timer