ui: fix setup of VNC websockets auth scheme with TLS

The way the websockets TLS code was integrated into the VNC server
made it essentially useless. The only time that the websockets TLS
support could be used is if the primary VNC server had its existing
TLS support disabled. ie QEMU had to be launched with:

  # qemu -vnc localhost:1,websockets=5902,x509=/path/to/certs

Note the absence of the 'tls' flag. This is already a bug, because
the docs indicate that 'x509' is ignored unless 'tls' is given.

If the primary VNC server had TLS turned on via the 'tls' flag,
then this prevented the websockets TLS support from being used,
because it activates the VeNCrypt auth which would have resulted
in TLS being run over a TLS session. Of course no websockets VNC
client supported VeNCrypt so in practice, since the browser clients
cannot setup a nested TLS session over the main HTTPS connection,
so it would not even get past auth.

This patch causes us to decide our auth scheme separately for the
main VNC server vs the websockets VNC server. We take account of
the fact that if TLS is enabled, then the websockets client will
use https, so setting up VeNCrypt is thus redundant as it would
lead to nested TLS sessions.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2 files changed
tree: 0c44102d8d9abb3c5e337d982b0dd7fbee3b24b5
  1. audio/
  2. backends/
  3. block/
  4. bsd-user/
  5. default-configs/
  6. disas/
  7. docs/
  8. fpu/
  9. fsdev/
  10. gdb-xml/
  11. hw/
  12. include/
  13. libcacard/
  14. libdecnumber/
  15. linux-headers/
  16. linux-user/
  17. migration/
  18. net/
  19. pc-bios/
  20. po/
  21. qapi/
  22. qga/
  23. qobject/
  24. qom/
  25. roms/
  26. scripts/
  27. slirp/
  28. stubs/
  29. sysconfigs/
  30. target-alpha/
  31. target-arm/
  32. target-cris/
  33. target-i386/
  34. target-lm32/
  35. target-m68k/
  36. target-microblaze/
  37. target-mips/
  38. target-moxie/
  39. target-openrisc/
  40. target-ppc/
  41. target-s390x/
  42. target-sh4/
  43. target-sparc/
  44. target-tricore/
  45. target-unicore32/
  46. target-xtensa/
  47. tcg/
  48. tests/
  49. trace/
  50. ui/
  51. util/
  52. .exrc
  53. .gitignore
  54. .gitmodules
  55. .mailmap
  56. .travis.yml
  57. accel.c
  58. aio-posix.c
  59. aio-win32.c
  60. arch_init.c
  61. async.c
  62. balloon.c
  63. block.c
  64. blockdev-nbd.c
  65. blockdev.c
  66. blockjob.c
  67. bootdevice.c
  68. bt-host.c
  69. bt-vhci.c
  70. Changelog
  71. CODING_STYLE
  72. configure
  73. COPYING
  74. COPYING.LIB
  75. coroutine-gthread.c
  76. coroutine-sigaltstack.c
  77. coroutine-ucontext.c
  78. coroutine-win32.c
  79. cpu-exec.c
  80. cpus.c
  81. cputlb.c
  82. device-hotplug.c
  83. device_tree.c
  84. disas.c
  85. dma-helpers.c
  86. dump.c
  87. exec.c
  88. gdbstub.c
  89. HACKING
  90. hmp-commands.hx
  91. hmp.c
  92. hmp.h
  93. iohandler.c
  94. ioport.c
  95. iothread.c
  96. kvm-all.c
  97. kvm-stub.c
  98. LICENSE
  99. main-loop.c
  100. MAINTAINERS
  101. Makefile
  102. Makefile.objs
  103. Makefile.target
  104. memory.c
  105. memory_mapping.c
  106. module-common.c
  107. monitor.c
  108. nbd.c
  109. numa.c
  110. os-posix.c
  111. os-win32.c
  112. page_cache.c
  113. qapi-schema.json
  114. qdev-monitor.c
  115. qdict-test-data.txt
  116. qemu-bridge-helper.c
  117. qemu-char.c
  118. qemu-coroutine-io.c
  119. qemu-coroutine-lock.c
  120. qemu-coroutine-sleep.c
  121. qemu-coroutine.c
  122. qemu-doc.texi
  123. qemu-img-cmds.hx
  124. qemu-img.c
  125. qemu-img.texi
  126. qemu-io-cmds.c
  127. qemu-io.c
  128. qemu-log.c
  129. qemu-nbd.c
  130. qemu-nbd.texi
  131. qemu-options-wrapper.h
  132. qemu-options.h
  133. qemu-options.hx
  134. qemu-seccomp.c
  135. qemu-tech.texi
  136. qemu-timer.c
  137. qemu.nsi
  138. qemu.sasl
  139. qjson.c
  140. qmp-commands.hx
  141. qmp.c
  142. qtest.c
  143. README
  144. rules.mak
  145. savevm.c
  146. softmmu_template.h
  147. spice-qemu-char.c
  148. tcg-runtime.c
  149. tci.c
  150. thread-pool.c
  151. thunk.c
  152. tpm.c
  153. trace-events
  154. translate-all.c
  155. translate-all.h
  156. user-exec.c
  157. VERSION
  158. version.rc
  159. vl.c
  160. xen-common-stub.c
  161. xen-common.c
  162. xen-hvm-stub.c
  163. xen-hvm.c
  164. xen-mapcache.c