OpenSSH for Windows hangs while writing to STDERR asynchronously

[lizhaoliu]

Troubleshooting steps
https://github.com/PowerShell/Win32-OpenSSH/wiki/Troubleshooting-Steps

Terminal issue? please go through wiki
https://github.com/PowerShell/Win32-OpenSSH/wiki/TTY-PTY-support-in-Windows-OpenSSH

Please answer the following

"OpenSSH for Windows" version
7.9p1, 8.0p1.

Server OperatingSystem
Debian rodete rodete (x86-64).

Client OperatingSystem
Windows 10, Windows Server 2016.

What is failing
Reproduced reliably with OpenSSH 7.9p1, 8.0p1:

1. We have an SSH client (named "test runner") written in Go running on Linux.
2. Test runner SSH'es into Windows 10 and Server 2016 machines (OpenSSH server) to run commands which will create another SSH session connecting to other Linux machines. In short, the SSH sessions propagate like this: test runner (Linux) => Win (OpenSSH) => device (Linux).
3. Test runner gets indefinitely blocked while trying to read from STDERR in the Windows SSH session since there is no output to STDERR.

I collected the call tack of ssh.exe on Windows:

        [External Code]
        ssh.exe!wait_for_multiple_objects_enhanced(unsigned long nCount, void * const * lpHandles, unsigned long dwMilliseconds, int bAlertable) Line 96        C
        ssh.exe!wait_for_any_event(void * * events, int num_events, unsigned long milli_seconds) Line 289       C
        ssh.exe!fileio_write(w32_io * pio, const void * buf, unsigned __int64 max_bytes) Line 749       C
        ssh.exe!w32_write(int fd, const void * buf, unsigned __int64 max) Line 562      C
        ssh.exe!do_log(LogLevel level, const char * fmt, char * args) Line 468  C
        ssh.exe!error(const char * fmt, ...) Line 170   C
        ssh.exe!check_host_key(char * hostname, sockaddr * hostaddr, unsigned short port, sshkey * host_key, int readonly, char * * user_hostfiles, unsigned int num_user_hostfiles, char * * system_hostfiles, unsigned int num_system_hostfiles) Line 1165    C
        ssh.exe!verify_host_key(char * host, sockaddr * hostaddr, sshkey * host_key) Line 1272  C
        ssh.exe!verify_host_key_callback(sshkey * hostkey, ssh * ssh) Line 98   C
        ssh.exe!kex_verify_host_key(ssh * ssh, sshkey * server_host_key) Line 1071      C
        ssh.exe!input_kex_gen_reply(int type, unsigned int seq, ssh * ssh) Line 160     C
        ssh.exe!ssh_dispatch_run(ssh * ssh, int mode, volatile int * done) Line 114     C
        ssh.exe!ssh_kex2(ssh * ssh, char * host, sockaddr * port, unsigned short) Line 224      C
        ssh.exe!ssh_login(ssh * ssh, Sensitive * sensitive, const char *) Line 1320     C
        ssh.exe!main(int ac, char * * av) Line 1499     C
        ssh.exe!wmain(int argc, wchar_t * * wargv) Line 61      C
        [External Code] 

For some reason the asyn write WriteFileEx gets stuck writing to STDERR and the main thread hangs waiting for the write to be completed.

I tried forcing the STDERR write to be sync and the problem goes away, i.e. in w32_write function, insert the following code:

if (fd == STDERR_FILENO) {
	fd_table.w32_ios[fd]->type = NONSOCK_SYNC_FD;
}

Expected output
The SSH sessions complete without hanging.

Actual output
The Windows SSH session hangs while writing to STDERR asynchronously.

[maertendMSFT]

Can you attempt this manually, if you still see the issue, can you provide your manual repro steps?