An overview of memory management in QEMU:

I. RAM Management:
==================

I.1. RAM Address space:
-----------------------

All pages of virtual RAM used by QEMU at runtime are allocated from
contiguous blocks in a specific abstract "RAM address space".
|ram_addr_t| is the type of block addresses in this space.

A single block of contiguous RAM is allocated with 'qemu_ram_alloc()', which
takes a size in bytes, and allocates the pages through mmap() in the QEMU
host process. It also sets up the corresponding KVM / Xen / HAX mappings,
depending on each accelerator's specific needs.

Each block has a name, which is used for snapshot support.

'qemu_ram_alloc_from_ptr()' can also be used to allocated a new RAM
block, by passing its content explicitly (can be useful for pages of
ROM).

'qemu_get_ram_ptr()' will translate a 'ram_addr_t' into the corresponding
address in the QEMU host process. 'qemu_ram_addr_from_host()' does the
opposite (i.e. translates a host address into a ram_addr_t if possible,
or return an error).

Note that ram_addr_t addresses are an internal implementation detail of
QEMU, i.e. the virtual CPU never sees their values directly; it relies
instead of addresses in its virtual physical address space, described
in section II. below.

As an example, when emulating an Android/x86 virtual device, the following
RAM space is being used:

  0x0000_0000 ... 0x1000_0000   "pc.ram"
  0x1000_0000 ... 0x1002_0000   "bios.bin"
  0x1002_0000 ... 0x1004_0000   "pc.rom"


I.2. RAM Dirty tracking:
------------------------

QEMU also associates with each RAM page an 8-bit 'dirty' bitmap. The
main idea is that whenever a page is written to, the value 0xff is
written to the page's 'dirty' bitmap. Various clients can later inspect
some of the flags and clear them. I.e.:

  VGA_DIRTY_FLAG (0x1) is typically used by framebuffer drivers to detect
  which pages of video RAM were touched since the latest VSYNC. The driver
  typically copies the pixel values to the real QEMU output, then clears
  the bits. This is very useful to avoid needless copies if nothing
  changed in the framebuffer.

  MIGRATION_DIRTY_FLAG (0x8) is used to tracked modified RAM pages during
  live migration (i.e. moving a QEMU virtual machine from one host to
  another)

  CODE_DIRTY_FLAG (0x2) is a bit more special, and is used to support
  self-modifying code properly. More on this later.


II. The physical address space:
===============================

Represents the address space that the virtual CPU can read from / write to.
|hwaddr| is the type of addresses in this space, which is decomposed
into 'pages'. Each page in the address space is either unassigned, or
mapped to a specific kind of memory region.

See |phys_page_find()| and |phys_page_find_alloc()| in translate-all.c for
the implementation details.


II.1. Memory region types:
--------------------------

There are several memory region types:

  - Regions of RAM pages.
  - Regions of ROM pages (similar to RAM, but cannot be written to).
  - Regions of I/O pages, used to communicate with virtual hardware.

Virtual devices can register a new I/O region type by calling
|cpu_register_io_memory()|. This function allows them to provide
callbacks that will be invoked every time the virtual CPU reads from
or writes to any page of the corresponding type.

The memory region type of a given page is encoded using PAGE_BITS bits
in the following format:

        +-------------------------------+
        |    mem_type_index     | flags |
        +-------------------------------+

Where |mem_type_index| is a unique value identifying a given memory
region type, and |flags| is a 3-bit bitmap used to store flags that are
only relevant for I/O pages.

The following memory region type values are important:

  IO_MEM_RAM (mem_type_index=0, flags=0):
    Used for regular RAM pages, always all zero on purpose.

  IO_MEM_ROM (mem_type_index=1, flags=0):
    Used for ROM pages.

  IO_MEM_UNASSIGNED (mem_type_index=2, flags=0):
    Used to identify unassigned pages of the physical address space.

  IO_MEM_NOTDIRTY (mem_type_index=3, flags=0):
    Used to implement tracking of dirty RAM pages. This is essentially
    used for RAM pages that have not been written to yet.

Any mem_type_index value of 4 or higher corresponds to a device-specific
I/O memory region type (i.e. with custom read/write callbaks, a
corresponding 'opaque' value), and can also use the following bits
in |flags|:

  IO_MEM_ROMD (0x1):
    Used for ROM-like I/O pages, i.e. they are backed by a page from
    the RAM address space, but writing to them triggers a device-specific
    write callback (instead of being ignored or faulting the CPU).

  IO_MEM_SUBPAGE (0x02)
    Used to indicate that not all addresses in this page map to the same
    I/O region type / callbacks.

  IO_MEM_SUBWIDTH (0x04)
    Probably obsolete. Set to indicate that the corresponding I/O region
    type doesn't support reading/writing values of all possible sizes
    (1, 2 and 4 bytes). This seems to be never used by the current code.

Note that cpu_register_io_memory() returns a new memory region type value.

II.2. Physical address map:
---------------------------

QEMU maintains for each assigned page in the physical address space
two values:

  |phys_offset|, a combination of ram address and memory region type.

  |region_offset|, an optional offset into the region backing the
  page. This is only useful for I/O pages.

The |phys_offset| value has many interesting encoding which require
further clarification:

  - Generally speaking, a phys_offset value is decomposed into
    the following bit fields:

      +-----------------------------------------------------+
      |         high_addr               |     mem_type      |
      +-----------------------------------------------------+

    where |mem_type| is a PAGE_BITS memory region type as described
    previously, and |high_addr| may contain the high bits of a
    ram_addr_t address for RAM-backed pages.

More specifically:

  - Unassigned pages always have the special value IO_MEM_UNASSIGNED
    (high_addr=0, mem_type=IO_MEM_UNASSIGNED)

  - RAM pages have mem_type=0 (i.e. IO_MEM_RAM) while high_addr are
    the high bits of the corresponding ram_addr_t. Hence, a simple call to
    qemu_get_ram_ptr(phys_offset) will return the corresponding
    address in host QEMU memory.

    This is the reson why IO_MEM_RAM is always 0:

    RAM page phys_offset value:
      +-----------------------------------------------------+
      |   high_addr                     |           0       |
      +-----------------------------------------------------+


  - ROM pages are like RAM pages, but have mem_type=IO_MEM_ROM.
    QEMU ensures that writing to such a page is a no-op, except on
    some target architectures, like Sparc, this may cause a CPU fault.

    ROM page phys_offset value:
      +-----------------------------------------------------+
      |   high_addr                     |     IO_MEM_ROM    |
      +-----------------------------------------------------+

  - Dirty RAM page tracking is implemented by using special
    phys_offset values with mem_type=IO_MEM_NOTDIRTY. Note that these
    values do not appear directly in the physical page map, but in
    the CPU TLB cache (explained later).

    non-dirty RAM page phys_offset value (CPU TLB cache only):
      +-----------------------------------------------------+
      |   high_addr                     |  IO_MEM_NOTDIRTY  |
      +-----------------------------------------------------+

   - Other pages are I/O pages, and their high_addr value will
     be 0 / ignored:

    I/O page phys_offset value:
      +----------------------------------------------------------+
      |  0                              | mem_type_index | flags |
      +----------------------------------------------------------+

    Note that when reading from or writing to I/O pages, the lowest
    PAGE_BITS bits of the corresponding hwaddr value will be added
    to the page's |region_offset| value. This new address is passed
    to the read/write callback as the 'i/o address' for the operation.

   - As a special exception, if the I/O page's IO_MEM_ROMD flag is
     set, then high_addr is not 0, but the high bits of the corresponding
     ram_addr_t backing the page's contents on reads. On write operations
     though, the I/O region type's write callback will be called instead.

     ROMD I/O page phys_offset value:
      +----------------------------------------------------------+
      |  high_addr                      | mem_type_index | flags |
      +----------------------------------------------------------+

     Note that |region_offset| is ignored when reading from such pages,
     it's only used when writing to the I/O page.