RawKeyboard events support

[erickzanardo]

Hello,

This may be a little out of scope of the project, but would be cool if there were support to RawKeyboard Events on this project.

I am a collaborator at a flutter game engine (https://github.com/luanpotter/flame), and with this kind of support, we could bring flutter games to desktop.

Thanks

[pchampio]

Some documentation about keyboard events:

• The dart raw_keyboard's handlers are located here
• The GLFW linux & windows implementation flutter/engine handler registration & flutter/engine handler definition
• MacOS specific flutter/flutter/#29231, google/FDE & flutter/engine#8219

[pchampio]

#140 is ready to be merged, @erickzanardo, @luanpotter, you can test the pending feature using: hover run -b "@feature/key-events" (You might need to upgrade hover).
Any feedback is greatly appreciated!!

Example Widget receiving Rawkeyevents (from FDE) (Clickable)

// Copyright 2018 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import 'package:flutter/material.dart';
import 'package:flutter/scheduler.dart';
import 'package:flutter/services.dart';

/// Keyboard test page for the example application.
class KeyboardTestPage extends StatefulWidget {
  @override
  State<StatefulWidget> createState() {
    return _KeyboardTestPageState();
  }
}

class _KeyboardTestPageState extends State<KeyboardTestPage> {
  final List<String> _messages = [];

  final FocusNode _focusNode = FocusNode();
  final ScrollController _scrollController = new ScrollController();

  @override
  void didChangeDependencies() {
    super.didChangeDependencies();
    FocusScope.of(context).requestFocus(_focusNode);
  }

  @override
  Widget build(BuildContext context) {
    return new Scaffold(
        appBar: AppBar(
            title: new Text('Keyboard events test'),
            leading: new IconButton(
                icon: new Icon(Icons.arrow_back),
                onPressed: () {
                  Navigator.of(context).pop();
                })),
        body: new RawKeyboardListener(
                  focusNode: _focusNode,
                  onKey: onKeyEvent,
                  child: Container(
                      padding: EdgeInsets.symmetric(horizontal: 16.0),
                      child: SingleChildScrollView(
                          controller: _scrollController,
                          child: Column(
                              crossAxisAlignment: CrossAxisAlignment.start,
                              children: _messages.map((m) => new Text(m)).toList())),
                  ),
              ),
    );
  }

  void onKeyEvent(RawKeyEvent event) {
    bool isKeyDown;
    switch (event.runtimeType) {
      case RawKeyDownEvent:
        isKeyDown = true;
        break;
      case RawKeyUpEvent:
        isKeyDown = false;
        break;
      default:
        throw new Exception('Unexpected runtimeType of RawKeyEvent');
    }

    int keyCode;
    switch (event.data.runtimeType) {
      case RawKeyEventDataLinux:
        final RawKeyEventDataLinux data = event.data;
        keyCode = data.keyCode;
        break;
      default:
        throw new Exception('Unsupported platform ${event.data.runtimeType}');
    }

    _addMessage('${isKeyDown ? 'KeyDown' : 'KeyUp'}: $keyCode\n- Modifiers: ${event.data.modifiersPressed}\n- KeyLabel: ${event.data.logicalKey.keyLabel}\n- debugName: ${event.data.logicalKey.debugName}');
  }

  void _addMessage(String message) {
    setState(() {
      _messages.add(message);
    });
    SchedulerBinding.instance.addPostFrameCallback((_) {
      _scrollController.jumpTo(
          _scrollController.position.maxScrollExtent,
      );
    });
  }
}

[erickzanardo]

That is some awesome news! I will be testing this tomorrow and report here.

Thanks a lot

[GeertJohan]

I will review the PR soon:tm:!

[pchampio]

@erickzanardo make sure to also checkout: #144 ;)

[erickzanardo]

@Drakirus I tested and it worked great! I have already added an example on Flame's repository if you want to check out: flame-engine/flame#92

Just a couple of comments, I tested on MacOs, and I have received instances of RawKeyEventDataLinux instead of RawKeyEventDataMacOs, it did not caused any issues though.

Also, I noticed a lot of these messages on the console while I kept the key pressed: go-flutter: Unknown key event type: 2, this also did not caused any issues despite the console been spammed with those messages.

About #144 I took a lot on the PR but did not figured it out how to activate de fullscreen.

Thanks a lot for implementing this, and great work on this library.

[GeertJohan]

@erickzanardo

Using go-flutter, you will receive RawKeyEventDataLinux on each platform (mac, windows, linux). This is because go-flutter "proxies" GLFW key events, and those are only supported by RawKeyEventDataLinux.

To start an app in fullscreen, add to the options list: flutter.WindowMode(flutter.WindowModeBorderlessFullscreen).

Make sure the app can close itself from full screen, or you'll be stuck in fullscreen ;)
Using the PopBehavior option may also be interesting for Flame, it allows you to configure how go-flutter should handle the app asking for a "pop". It can be configured to minimize (aka iconify) or close the app. https://godoc.org/github.com/go-flutter-desktop/go-flutter#PopBehavior

[pchampio]

@erickzanardo Tanks for your feedback!

The go-flutter: Unknown key event type: 2 error message occurs when go-flutter detects a gfw.Repeat Action.

The flutter framework supports only keyDown & keyUp events, go-flutter matches the same configuration.

[erickzanardo]

Awesome, thanks for the clarifications @Drakirus and @GeertJohan!

I just tested the fullscreen feature and it worked pretty well.

Awesome work!

[GeertJohan]

Thanks @erickzanardo

@Drakirus Perhaps we can 'silence' the Unknown event type when it is glfw.Repeat?
We know that glfw.Repeat exists, and we know we cannot send it to the Flutter Framework, so it can be ignored. Ofcourse it should still report for actions we do not know (perhaps are added in the future).

[pchampio]

I was about to submit a PR on flutter/flutter that adds a new RawKeyRepeatEvent event to RawKeyEvent, but then I thought about the current implementation.

• On Android, there is a KeyEvent.getRepeatCount() that retrieve the repeat count of the event.
  From what I can tell the above getter isn't used by the android shell meaning the keyDown event is sent even if the return value of getRepeatCount() is superior to 0 (event occurred multiples times, glfw.Repeat on GLFW).

• On MacOS, there is a NsEvent.isARepeat() that indicates whether the key event is a repeat.
  On the current MacOS shell (From FDE) the NsEvent.isARepeat() getter isn't used meaning keyDown even is sent event if the key is held down.

• On iOS, there is no support for keyPress events flutter/engine#8606

• Linux/Windows, (from FDE) are only supporting glfw.Press and glfw.Release events.
  But I'll like to point out that the Linux/Windows (from FDE) impl was always behind MacOS impl.

Adding support for RawKeyRepeatEvent in flutter/flutter involves changes on multiples shells.

For now, what I think we should do is to match the Android and MacOS implementation where multiples successive RawKeyDownEvent event of the same key can be sent (translating glfw.Repeat to a glfw.Press action).

@GeertJohan do you have any objections? (we'll keep the comment saying RawKeyRepeatEvent may be ported to the Flutter Framework, and that we are synthesizing a glfw.Press event)

If interested: Source of the the patch adding `RawKeyRepeatEvent` (Clickable)

From 2fb1a185761dc85c79aa2a75815f23cc9cc194d3 Mon Sep 17 00:00:00 2001
From: Drakirus <[email protected]>
Date: Wed, 22 May 2019 17:37:02 +0200
Subject: [PATCH] Add RawKeyRepeatEvent

---
 .../lib/src/services/raw_keyboard.dart        | 25 ++++++++++++++++---
 1 file changed, 21 insertions(+), 4 deletions(-)

diff --git a/packages/flutter/lib/src/services/raw_keyboard.dart b/packages/flutter/lib/src/services/raw_keyboard.dart
index cf449ab27..b785a78b5 100644
--- a/packages/flutter/lib/src/services/raw_keyboard.dart
+++ b/packages/flutter/lib/src/services/raw_keyboard.dart
@@ -105,8 +105,8 @@ enum ModifierKey {
 ///
 ///  * [RawKeyEventDataAndroid], a specialization for Android.
 ///  * [RawKeyEventDataFuchsia], a specialization for Fuchsia.
-///  * [RawKeyDownEvent] and [RawKeyUpEvent], the classes that hold the
-///    reference to [RawKeyEventData] subclasses.
+///  * [RawKeyDownEvent], [RawKeyUpEvent] and [RawKeyRepeatEvent], the classes
+///    that hold the reference to [RawKeyEventData] subclasses.
 ///  * [RawKeyboard], which uses these interfaces to expose key data.
 @immutable
 abstract class RawKeyEventData {
@@ -230,6 +230,8 @@ abstract class RawKeyEventData {
 ///    pressing a key.
 ///  * [RawKeyUpEvent], a specialization for events representing the user
 ///    releasing a key.
+///  * [RawKeyRepeatEvent], a specialization for events representing the user
+///     holding down a key.
 ///  * [RawKeyboard], which uses this interface to expose key data.
 ///  * [RawKeyboardListener], a widget that listens for raw key events.
 @immutable
@@ -294,6 +296,8 @@ abstract class RawKeyEvent {
         return RawKeyDownEvent(data: data, character: message['character']);
       case 'keyup':
         return RawKeyUpEvent(data: data);
+      case 'keyrepeat':
+        return RawKeyRepeatEvent(data: data);
       default:
         throw FlutterError('Unknown key event type: $type');
     }
@@ -434,6 +438,19 @@ class RawKeyUpEvent extends RawKeyEvent {
   }) : super(data: data, character: character);
 }
 
+/// The user has held down a key on the keyboard until it repeated.
+///
+/// See also:
+///
+///  * [RawKeyboard], which uses this interface to expose key data.
+class RawKeyRepeatEvent extends RawKeyEvent {
+  /// Creates a key event that represents the user holding down a key.
+  const RawKeyRepeatEvent({
+    @required RawKeyEventData data,
+    String character,
+  }) : super(data: data, character: character);
+}
+
 /// An interface for listening to raw key events.
 ///
 /// Raw key events pass through as much information as possible from the
@@ -446,8 +463,8 @@ class RawKeyUpEvent extends RawKeyEvent {
 ///
 /// See also:
 ///
-///  * [RawKeyDownEvent] and [RawKeyUpEvent], the classes used to describe
-///    specific raw key events.
+///  * [RawKeyDownEvent], [RawKeyUpEvent] and [RawKeyRepeatEvent], the classes
+///    used to describe specific raw key events.
 ///  * [RawKeyboardListener], a widget that listens for raw key events.
 ///  * [SystemChannels.keyEvent], the low-level channel used for receiving
 ///    events from the system.
-- 
2.21.0

[GeertJohan]

Sounds good @drakirus