Keymap for virtual keyboard should be immutable

[Xyene]

Sway config:

input * {
    xkb_layout "myus"
}

Then, connecting with wayvnc and opening up a shell, the keyboard layout is still us.

$ swaymsg -t get_inputs                                                                                                                                                                     
Input device: virtual pointer
  Type: Mouse
  Identifier: 0:0:virtual_pointer
  Product ID: 0
  Vendor ID: 0

Input device: virtual keyboard
  Type: Keyboard
  Identifier: 0:0:virtual_keyboard
  Product ID: 0
  Vendor ID: 0
  Active Keyboard Layout: English (US)

Switching it to myus is ineffective.

$ swaymsg input '*' xkb_layout myus   
$ swaymsg -t get_inputs                                                                                                                                                                     
Input device: virtual pointer
  Type: Mouse
  Identifier: 0:0:virtual_pointer
  Product ID: 0
  Vendor ID: 0

Input device: virtual keyboard
  Type: Keyboard
  Identifier: 0:0:virtual_keyboard
  Product ID: 0
  Vendor ID: 0
  Active Keyboard Layout: English (US)

To get things working, I first have to set the layout to us (this should be a noop, but appears to not be one), and then to myus.

$ swaymsg input '*' xkb_layout us                                                                                                                                                                
$ swaymsg input '*' xkb_layout myus                                                                                                                                                           
$ swaymsg -t get_inputs 
Input device: virtual pointer
  Type: Mouse
  Identifier: 0:0:virtual_pointer
  Product ID: 0
  Vendor ID: 0

Input device: virtual keyboard
  Type: Keyboard
  Identifier: 0:0:virtual_keyboard
  Product ID: 0
  Vendor ID: 0
  Active Keyboard Layout: English (Mine)

At this point everything works.

[Xyene]

/cc @any1 since I figure you're most familiar with this. Do you reckon the bug is:

• that the layout isn't applied at connection time, or
• that the layout of the virtual keyboard can be changed at all?

The protocol doesn't seem to suggest what should happen in this case. Is the keymap specified in the virtual keyboard protocol supposed to always take precedence?

FWIW, specifying --keyboard=myus to wayvnc doesn't quite do the right thing either, but I haven't looked far into it. The layout reported by Sway is correct at connection time

$ swaymsg -t get_inputs 
Input device: virtual pointer
  Type: Mouse
  Identifier: 0:0:virtual_pointer
  Product ID: 0
  Vendor ID: 0

Input device: virtual keyboard
  Type: Keyboard
  Identifier: 0:0:virtual_keyboard
  Product ID: 0
  Vendor ID: 0
  Active Keyboard Layout: English (Mine)

but the actual keymap is not applied, and appears to be us.

[any1]

Virtual keyboards get their keyboard layout from the client. Changing their layout from elsewhere can mess with the client's expectations, so things might misbehave. Trying to change the layout of a virtual keyboard should not have any effect and perhaps it would even be prudent to output a warning somewhere.

Unless the VNC client supports qemu extended key events, wayvnc will do a reverse mapping of symbols to key codes for the chosen layout. In this mode, it doesn't really matter so much which layout you choose as long as it has all the symbols that you use. With the qemu extension, things behave in a simpler and much saner way where key codes are passed to the virtual keyboard instead of symbols. In this mode, it matters which layout you choose.

[Xyene]

Virtual keyboards get their keyboard layout from the client. Changing their layout from elsewhere can mess with the client's expectations, so things might misbehave. Trying to change the layout of a virtual keyboard should not have any effect and perhaps it would even be prudent to output a warning somewhere.

OK, this makes sense to me. This is easy enough to fix in Sway, but I wonder if we shouldn't also have a guard within wlr_keyboard_set_keymap to abort if you try to set a keymap on a virtual keyboard?

If I understand correctly the only instance a virtual keyboard's keymap should be changed should be within wlroots itself, in response to zwp_virtual_keyboard_v1_interface.keymap. Does it ever make sense for a compositor to change it otherwise?

wayvnc will do a reverse mapping of symbols to key codes for the chosen layout. In this mode, it doesn't really matter so much which layout you choose as long as it has all the symbols that you use.

Hmm, I don't entirely follow.

My use-case is the a/s keys on my travel laptop are broken, and I have a keymap to rebind {z → a, x → s, Alt+z → z, Alt+x → x}. I have this layout installed on both my laptop and the host I'm VNCing to, and specify this layout to wayvnc's --keyboard.

When I press the z key on my laptop with gedit open locally, a gets typed. If I do the same to gedit running in wayvnc, I instead get z, as if no layout was applied. Is this intended behavior? What mapping happens here to get this result? My client is Remmina, which as far as I can tell from a quick Google doesn't seem to support the QEMU extended key events. (What's a client that does?)

[any1]

You shouldn't need to set the keyboard layout on the remote host as the symbols just get translated to key codes for the selected layout anyhow. The basic VNC (RFB) protocol uses key symbols, but wayland clients get key codes sent to them and translate them to symbols using xkbcommon. This means that wayvnc has to translate the symbols it gets from the VNC client to key codes that it can send to the compositor via the virtual keyboard protocol.

It is odd that you're getting a {z} in the scenario that you described. This is more likely to be a bug in wayvnc or remmina than sway.

What happens with wlvncc? It supports the qemu extension, which does require you to set the keyboard layout on both ends. I think tigervnc also supports it.

[Xyene]

Thanks for the explanation, I understand now.

What happens with wlvncc? It supports the qemu extension, which does require you to set the keyboard layout on both ends. I think tigervnc also supports it.

I couldn't get wlvncc running because it's 2021 and my compositor still doesn't support xdg-shell. Grumble, grumble. It does however work as expected in tigervnc, as well as krdc (which does not appear to support extended key events). So, it's probably a Remmina bug.

I suppose the only actionable thing in this ticket is that the keymap of a virtual keyboard should be immutable.

[emersion]

Ref swaywm/wlroots#2420