android/docs/DEVELOPMENT.TXT

Introduction:
-------------

This document provides important information about the Android
emulator codebase, which will be useful for anyone planning to
contribute patches to it.


I. Getting the sources:
-----------------------

I.1) Get the 'repo' tool:
- - - - - - - - - - - - -

See https://source.android.com/source/downloading.html for download and
installation instructions.

I.2) Determine the current Android Studio development branch:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

See the content of http://tools.android.com/build

As of this writing, the major emulator development happens on emu-master-dev

I.3) Checkout the sources with 'repo':
- - - - - - - - - - - - - - - - - - - -

    mkdir emu-master-dev
    cd emu-master-dev
    repo init -u https://android.googlesource.com/platform/manifest \
            -b emu-master-dev
    repo sync

Replace 'emu-master-dev' with another branch name if necessary.


II. Building the binaries:
--------------------------

II.1) Prebuilt binaries:
- - - - - - - - - - - -

Building the emulator now depends on many prebuilt libraries that are stored
under $TOP/prebuilts/android-emulator-build/. Normally, you don't need to
care about them, but read the section below named "Rebuilding prebuilts"
for more information, or if you need to add a new dependency.

II.2) Build the emulator binaries:
- - - - - - - - - - - - - - - - - -

After building all the dependencies, do:

  cd $TOP/external/qemu
  ./android/rebuild.sh

Note that android/rebuild.sh will build all sources, but also run a small
series of unit-tests to check that everything works as expected. In case
of problem, use the --verbose flag to see what's failing.

Final binaries will be located under the 'objs' directory. You can select
a different location with --out-dir=<path>.

All scripts available to build emulator binaries take a --help option that
describe them in detail, and provide an explanation of all supported options.

On Linux, it is possible to build Windows binaries by using the --mingw option
as in:

  ./android/rebuild.sh --mingw

If you have Wine installed, android/rebuild.sh will detect it and run the
unit-tests with it.

IMPORTANT: Unfortunately, it's not possible to run the Windows emulator
           binaries under Wine anymore. This seems related to OpenGL API
           emulation issues in Wine.

android/rebuild.sh also supports other interesting options (e.g. --debug to
generate debug versions of the binaries). For all details, use:

  ./android/rebuild.sh --help

And read the output.


II.3) Packaging binary releases:
- - - - - - - - - - - - - - - - -

You can create redistributable tarballs by using:

  cd $TOP/external/qemu
  android/scripts/package-release.sh

This will create /tmp/android-emulator-<date>-<system>.tar.bz2 containing
all binaries, ready to be uncompressed into an Android SDK installation
directory.

Alternatively, you can update the prebuilt binaries that are stored in
an AOSP checkout under:

   $AOSP/prebuilts/android-emulator/

These are the binaries that are launched when you type 'emulator' just
after a platform build, in the root AOSP directory.

To update them, do:

  android/scripts/package-release.sh  --copy-prebuilts=/path/to/aosp

Note that this also creates a README file in the target directory that
describes the git log of each source git tree since the last update.


II.4) Remote darwin builds:
- - - - - - - - - - - - - -

The scripts describe above work on Linux and Darwin systems only.

On Linux, binaries for both Linux and Windows are generated by default.

You can however build Darwin binaries from Linux, provided that you have
a Darwin machine available through SSH. To do that, do either one of the
following:

  - Set ANDROID_EMULATOR_DARWIN_SSH=<hostname> in your environment.
  - Use --darwin-ssh=<hostname> when calling one of the build scripts.

It is recommended to always use this when using package-release.sh, this
makes it an ideal way to check that a source code change actually compiles
for all 3 host platforms.


II.5) Speeding up builds with 'ccache':
- - - - - - - - - - - - - - - - - - - -

It is highly recommended to install the 'ccache' build tool on your development
machine(s). The Android emulator build scripts will probe for it and use it
if available, which can speed up incremental builds considerably.

Build scripts also support the --ccache=<program> and --no-ccache options to
better control ccache usage.



III. Testing:
-------------

III.1) Unit-testing:
- - - - - - - - - - -

./android/rebuild.sh already runs a small set of unit test programs and
verification checks. If they fail, it will print an error message about it.

Most unit-tests are in C++ and use the GoogleTest library. Whenever you add
a new Android-specific feature to the code base, that doesn't directly
modifies the QEMU-based code, add corresponding unit tests to avoid
regressions in the future.


III.2) Booting existing AVDs:
- - - - - - - - - - - - - - -

Otherwise, a simple way to test a emulator code change is trying to boot an
existing AVD, by doing the following after calling android/rebuild.sh:

   export ANDROID_SDK_ROOT=/path/to/installed/android/sdk
   objs/emulator <your-options>

III.3) Booting an Android platform build:
- - - - - - - - - - - - - - - - - - - - -

It is also possible to try booting an Android system image built from a fresh
AOSP platform checkout. One has to select an emulator-compatible build product,
e.g.:

   cd $AOSP/
   . build/envsetup.sh
   lunch aosp_arm-userdebug
   make -j8

Recommended build products are 'aosp_<cpu>-userdebug' or 'aosp_<cpu>-eng',
where <cpu> can be one of 'arm', 'x86', 'mips', 'x86_64', 'arm64' or 'mips64'.

To boot the generated system image:

   cd $TOP/external/qemu
   ./android/rebuild.sh
   export ANDROID_BUILD_TOP=/path/to/aosp
   objs/emulator <your-options>


IV. Overall source layout:
--------------------------

IV.1) Common Android-emulation related sources:
- - - - - - - - - - - - - - - - - - - - - - - -

The following directories contain generic libraries used for Android
emulation. By 'generic', it is meant that the corresponding sources
do not depend on a specific emulation engine (e.g. QEMU1 or QEMU2 as
described below).

Under $TOP/external/qemu/ they are located at:

  android/android-emu/
    Android-specific code, used to implement a UI and functional layer on
    top of the QEMU code base. Ideally, anything that shouldn't impact the
    QEMU internals should be here.

  android/android-emugl/
    Android-specific GPU emulation library, also called "EmuGL".
    Read the DESIGN.TXT file in this directory for more information
    on its design and how it is used at runtime. Also read
    android/docs/GPU-EMULATION.TXT.

  android/third_party/
    Contains various third-party libraries build scripts and/or sources that
    the Android emulation libraries depend on (e.g. zlib, libpng, libSDL).


IV.1) "Classic" emulator source layout:
- - - - - - - - - - - - - - - - - - - -

The 'classic' emulator programs are used to run legacy Android SDK system
images (from Cupcake to Lollipop) and are based on a very ancient version of
QEMU1 (1.10) that has been drastically patched and altered.

Its sources lie under $TOP/external/qemu/android/qemu1/, with the following
important sub-directories:

  android-qemu1-glue/
    Glue code between QEMU1 and the Android emulation libraries. Ideally,
    it is the only part of the code that is supposed to know about both
    android/android-emu/ and android/qemu1/ at the same time.
    (In practice some of the android/qemu1/ includes android/android-emu/
    headers directly, but since this is a mostly-deprecated version of
    QEMU, we don't really bother about cleaning this up).

  include/hw/android/goldfish/
    Headers for the Android-specific goldfish virtual devices.
    See docs/GOLDFISH-VIRTUAL-PLATFORM.TXT for more details.

  hw/android/goldfish/
    Implementation files for the Android-specific goldfish virtual devices.

  hw/android/
    Implementation files for the Android-specific virtual ARM and MIPS
    boards. For x86, the content of hw/i386/pc.c was modified instead.

  slirp-android/
    Modified version of the slirp/ directory, which adds various Android-specific
    features and modifications.

Generally speaking, some QEMU source files have been rewritten in so signficant
ways that they gained an -android prefix (e.g. vl-android.c versus vl.c). The
original file is typically kept as a reference to make it easier to see
modifications and eventually integrate upstream changes.

Note that these sources are considered 'legacy', i.e. there is no effort to
port Android-specific changes here to upstream QEMU.


IV.2) New 'qemu2' emulator source layout:
- - - - - - - - - - - - - - - - - - - - - - - - -

The 'new' emulator programs are used to run more version Android system images
(e.g. Lollipop on 64-bit ARM CPUs), and are based on a much more recent version
of upstream QEMU (2.7.0 at this time), which has been slightly modified with
Android-specific changes.

The sources are under $TOP/external/qemu/, and match the upstream QEMU source
tree layout, to ease future rebases and cherry-picks of upstream fixes. This
is why all other sources are under $TOP/external/qemu/android/ too.

Note that these sources will be periodically rebased on top of upstream QEMU
changes / releases. It is our goal to ultimately send patches upstream to
enable Android support in QEMU, but this is lower priority task compared with
getting things to work.

One of the major differences between the classic and new code bases is that
they do not support the same virtual hardware.

- The classic emulator supports the 'goldfish' virtual platform.
  See docs/GOLDFISH-VIRTUAL-HARDWARE.TXT for a full description.

- The new emulator supports the 'ranchu' virtual platform, which is an
  evolution of 'goldfish' that replaces NAND and MMC devices with
  virtio-block instead, as well as a few other differences.

NOTE: 'ranchu' is still a work in progress and isn't completely documented
      yet, for full details, one should refer to the sources under
      external/qemu as well as the android-goldfish-3.10 branch
      found on https://android.googlesource.com/kernel/goldfish.git


V) Overview of all binaries:
----------------------------

The Android emulator is really made of several binaries that work together
to implement all required emulation features. This is a detailed description
of them:

  - The 'emulator' program launcher, which is the main entry point to
    launch a virtual Android device (AVD). Typically located under:

        $EXEC_DIR/emulator

    The role of this tiny program is to read the AVD's configuration, and
    device which specific emulation engine program to run (see below). It must
    also setup the environment to ensure that GPU emulation libraries are
    properly loaded when needed (which typically involves modifying
    LD_LIBRARY_PATH on Posix systems, and PATH on Windows).

  - CPU-specific "classic" (a.k.a QEMU1) emulation engines, which are
    independent program that can emulate a single virtual CPU. 32-bit
    and 64-bit versions are typically available, and are called by
    'emulator' based on AVD configuration, for example:

        $EXEC_DIR/emulator-arm   ->
                32-bit host program to emulate 32-bit ARM CPUs.

        $EXEC_DIR/emulator-x86   ->
                32-bit host program to emulate 32-bit x86 and
                64-bit x86_64 CPUs.

        $EXEC_DIR/emulator64-arm ->
                64-bit host program to emulate 32-bit ARM CPUs.

    These are tagged as "classic" because they are generated from the
    sources under external/qemu/android/qemu1, and can be used to emulate old
    Android system images (e.g. anything before the L release).

  - CPU-specific "newer" QEMU2 emulation engines, based on a newer version
    of QEMU (2.2.0), located under:

        $EXEC_DIR/qemu/$HOST_SYSTEM/qemu-system-$ARCH

    Where $HOST_SYSTEM is a host system tag (e.g. linux-x86_64), and $ARCH
    is a virtual cpu name, following QEMU convention (e.g. 'aarch64' for
    64-bit ARM CPUs).

    They are built by android/rebuild.sh from the sources under
    external/qemu/ that are _not_ under external/qemu/android/.

    Note that these are slightly-modified QEMU programs, and thus accept
    emulator-specific command-line options, not the original QEMU ones.

  - GPU Emulation libraries, which implement guest EGL/GLES to desktop GL
    translation. This is the default GPU emulation mode also known as 'host'
    mode. The files are:

        $EXEC_DIR/lib/                    -> for 32-bit host programs.
                libOpenglRender.so
                libEGL_translator.so
                libGLES_CM_translator.so
                libGLESv2_translator.so

        $EXEC_DIR/lib64/                  -> for 64-bit host programs.
                lib64OpenglRender.so
                lib64EGL_translator.so
                lib64GLES_CM_translator.so
                lib64GLESv2_translator.so

    The libraries are loaded by the emulation engine programs when needed
    (i.e. if the AVD has 'hw.gpu.enabled' set to 'yes'). In particular,
    the 'emulator' launcher takes care of adding either $EXEC_DIR/lib/
    or $EXEC_DIR/lib64/ to LD_LIBRARY_PATH (or PATH on Windows) before
    launching the real emulation engine.

    There is also the possibility of supporting additional GPU emulation
    modes, through optional 'backends'. For example, the 'mesa' backend
    is used to perform software-based OpenGL rendering, and can be activated
    by using '-gpu mesa' or setting 'hw.gpu.mode' to 'mesa' in the AVD
    configuration.

    The files of a given backend <name> must be under:

        $EXEC_DIR/lib/gles_<name>/ or
        $EXEC_DIR/lib64/gles_<name>/

    For more details, see docs/GPU-EMULATION.TXT

  - The 'query answering machine', a small application which only purpose is
    to answer queries from Android Studio:

        $EXEC_DIR/emulator-check

    Currently the only parameter is '-accel', which returns the status of
    the hypervisor support on the current machine (if emulator can use
    hardware acceleration).

VI. Rebuilding prebuilt binaries:
---------------------------------

The emulator prebuilt binaries are stored in git repositories under:

  $TOP/prebuilts/android-emulator-build/

In particular, this contains an 'archive' sub-directory:

  $TOP/prebuilts/android-emulator-build/archive/
       A directory containing a PACKAGES.TXT file describing all source
       tarballs needed to rebuild the prebuilts from scratch. The directory
       also contains copies of said tarballs to avoid re-downloading them
       from the Internet every time.

  $TOP/prebuilts/android-emulator-build/<name>/

    A directory containing the binaries for prebuilt dependency <name>.
    These binaries are generated from the sources from .../archive/ through
    scripts provided under $TOP/external/qemu/android/scripts/.

The most important files here are the following:

   external/android/scripts/download-sources.sh:
       A script used to parse PACKAGES.TXT and download or check all
       packages into the .../archive one.

       If one wants to add a new prebuilt library, start by updating
       PACKAGES.TXT, then run download-sources.sh to update the
       .../archive/ content.

       IMPORTANT: In other words: IF YOU UPDATE PACKAGES.TXT, ALWAYS RUN
                  download-sources.sh BEFORE RUNNING ONE OF THE BUILD
                  SCRIPTS BELOW.

    external/android/scripts/build-<name>.sh:
       A script used to rebuild a given set of prebuilt binaries.
       Note that by default, when invoked on Linux, this builds Linux
       and Windows binaries at the same time. It is also possible to
       build for all supported host platforms with the --darwin-ssh=<hostname>
       option, that will direct the script to perform a remote build on
       a Darwin machine through ssh.

       The corresponding binaries are then stored under one of:

           $TOP/prebuilts/android-emulator-build/<name>/<system>/
           $TOP/prebuilts/android-emulator-build/common/<name>/<system>/

       Where <name> is the prebuilt name, and <system> is a host system tag
       like windows-x86 or linux-x86_64.

    external/android/scripts/build-qt.sh:
       As a special exception, this script works like the other build-<name>.sh
       scripts, except that the Qt source tarball are not listed in
       PACKAGES.TXT, nor is a copy stored under .../archive/.

       The reason for this is simply that the archive file is too large
       (several hundreds MBs). As such, one should call this script with
       the --download option on the first time. This will grab the tarball
       from official Qt servers and put it under .../archive/ where it
       will *not* be put under git.

    external/android/scripts/build-qemu-android.sh:
       This is another special exception because this script does not build
       something that is put on git repositories. The purpose of this script
       is to rebuild the QEMU2 binaries from sources under
       $TOP/external/qemu, using the original QEMU2 configure/build
       system (unlike android/rebuild.sh, which uses the emulator build
       system to compile them). The main difference is that said binaries
       cannot use the AndroidEmu library (see the documentation for it in
       docs/ANDROID-EMULATION-LIBRARY.TXT).

       The reason to use this script is to ease the task of tracking
       upstream QEMU changes, and rebasing our own modifications on top
       of it (mainly because the Android-specific features added on top
       of it are way too large to be accepted by upstream).

       In particular, build-qemu-android.sh generates LINK-<system>.TXT
       files describing all the object files used during a link of the
       final QEMU2 binaries. These files are later processed by a helper
       script located under:

          external/qemu/android-qemu2-glue/ \
                  scripts/gen-qemu2-sources-mk.sh

       to generate the correspond Android .mk rules used with
       android/rebuild.sh, i.e.:

          external/qemu/android-qemu2-glue/ \
                  build/Makefile.qemu2-sources.mk

       The new QEMU2 binaries are copied into:

          $TOP/prebuilts/android-emulator-build/qemu-android/

       Note that this directory is *not* backed by a git repository.

VII. Other related sources:
---------------------------

A full SDK depends on other notable sources from different locations:

* https://android.googlesource.com/platform/sdk:

    This repository has an emulator/ directory that contains the sources for
    various emulator-related sources and binaries:

       sdk/emulator/opengl/
          No longer exists, but used to host the sources for the host-side
          GPU emulation libraries. These are now located under
          external/qemu/android-emugl instead. Read the DESIGN.TXT document
          in this directory to learn more about how GPU emulation works.

       sdk/emulator/mksdcard/
          Sources for the 'mksdcard' SDK host tool, used to generate an empty
          SDCard partition image that can be used by the emulator. This
          executable is never called or used by the Android emulator itself,
          but by the AVD Manager UI tool instead.

       sdk/emulator/snapshot/snapshot.img
          An empty QCOW2 partition image file, this is copied by the AVD
          Manager into a new AVD's content directory if the user enables
          snapshotting.

    The rest of sdk/ is not related to emulation and can be ignored here.


* https://android.googlesource.com/platform/device/generic/goldfish/

    Contains emulation support libraries that must be in the system image
    for emulation to work correctly. Essentially HAL (Hardware Abstraction
    Layer) modules for various virtual devices, and the GPU EGL/GLES
    system libraries that communicate with the host-side ones during
    emulation.

* https://android.googlesource.com/platform/prebuilts/qemu-kernel/

    Prebuilt kernel images configured to run under emulation. Read
    docs/ANDROID-KERNEL.TXT for more information.

* https://android.googlesource.com/platform/prebuilts/android-emulator/

    Prebuilt emulator and GPU emulation libraries for various host platforms.
    These are used only when building an emulator-compatible system image
    with the platform build (e.g. 'aosp_<cpu>-userdebug').

    The 'lunch' command adds this directory to the path so you can just
    type 'emulator' after a succesful build, and have the corresponding
    system image started under emulation.

    Updating these binaries requires using the script under
    android/scripts/package-release.sh with the --copy-prebuilts option
    (see above).