一次 ssh 互信免密码登陆失败的调试经历

小民 - 2021-08-23 19:32:46

1 问题背景描述：

k8s 节点间(包括与自己的互信)建立互信后，发现登陆并没有免密。
互信建立操作如下

# ssh-keygen -t rsa
# ssh-copy-id -i .ssh/id_rsa.pub root@<remote ip>

为方便描述：这里设定为A与B互信操作后，A机器上ssh B不需要密码，B机器上ssh A需要密码

2 排查

这里百度参考了一位大神的调试步骤，跟着验证了一遍

2.1 debug 日志

首先要拿到明细的 Debug 日志，看看卡在那里。linux很多命令都带又调试功能，ssh就自带debug功能：
B机器上连A （调试的时候将localhost换为A的IP）

# ssh -vvv localhost
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to localhost [127.0.0.1] port 22.
debug1: Connection established.
debug1: identity file /home/work/.ssh/identity type -1
debug1: identity file /home/work/.ssh/identity-cert type -1
...
debug3: remaining preferred: keyboard-interactive,password
// 启用公钥登录
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/work/.ssh/identity
debug3: no such identity: /home/work/.ssh/identity
debug1: Offering public key: /home/work/.ssh/id_rsa
debug3: send_pubkey_test
// 发送公钥包，等待服务器认证响应
debug2: we sent a publickey packet, wait for reply
debug3: Wrote 368 bytes for a total of 1741
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/work/.ssh/id_dsa
debug3: no such identity: /home/work/.ssh/id_dsa
debug1: Trying private key: /home/work/.ssh/id_ecdsa
debug3: no such identity: /home/work/.ssh/id_ecdsa
// 没通过认证，禁用该认证方法
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
// 下一个认证方法：启用密码登录
debug1: Next authentication method: password
work@localhost's password:

可以看到，确实是认证失败了，但是仅凭一句 we did not send a packet, disable method，咱们还是无法看到失败的深层次原因，那咱们再对比下正常的认证流程应该是怎样的：

2.2 检查配置

打开A服务器的 /etc/ssh/sshd_config

确认下面几行是这样的:

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile  .ssh/authorized_keys
#GSSAPIAuthentication yes
#GSSAPICleanupCredentials yes

配置没问题，此路还是不通。

2.3 Debugging SSH public key

从上面的对比流程可以看到，正常互信之后"we sent a publickey packet, wait for reply"之后应该紧跟着"debug1: Server accepts key: pkalg ssh-rsa blen 277"，所以可以看出互信失败的原因：A 的 sshd 不认可 publickey。
至于为什么不认可，参照大神的笔记，在 google 上搜索了关键字"ssh publickey ignore debug diagnose"，此处我查阅了第二条：https://unix.stackexchange.co...
大概意思就是A机器上的相关文件或目录的权限不对

~ 
~/.ssh
~/.ssh/authorized_keys

，调试方法：

If you have root access to the server, the easy way to solve such problems is to run sshd in debug mode, by issuing something like /usr/sbin/sshd -d -p 2222 on the server (full path to sshd executable required, which sshd can help) and then connecting from the client with ssh -p 2222 user@host. This will force the SSH daemon to stay in the foreground and display debug information about every connection. Look for something like
debug1: trying public key file /path/to/home/.ssh/authorized_keys
...
Authentication refused: bad ownership or modes for directory /path/to/home/
If it isn't possible to use an alternative port, you can temporarily stop the SSH daemon and replace it with one in debug mode. Stopping the SSH daemon does not kill existing connections so it is possible to do this through a remote terminal, but somewhat risky - if the connection does get broken somehow at a time when the debug replacement is not running, you are locked out of the machine until you can restart it. The commands required:
service ssh stop
/usr/sbin/sshd -d
#...debug output...
service ssh start
(Depending on your Linux distribution, first / last line might be systemctl stop sshd.service / systemctl start sshd.service instead.)

就是在A机器上执行 /usr/sbin/sshd -d -p 2222 （在2222端口启动一个带debug输出的sshd，2222端口可以用任意端口代替）
然后在B机器执行 ssh -vv localhost -p 2222
在A端可以看到

debug1: trying public key file /path/to/home/.ssh/authorized_keys
...
Authentication refused: bad ownership or modes for directory /path/to/home/
也就是说因为你的/path/to/home/目录权限有问题导致（不一定设这个目录，比如我遇到的时root目录权限有问题。这些debug信息也可以在 /var/log/secure 中查看到。

3 最终解决方案

Your home directory ~, your ~/.ssh directory and the ~/.ssh/authorized_keys file on the remote machine must be writable only by you
Check the permissions on your home directory, .ssh directory, and the authorized_keys file: If your ssh server is running with ‘StrictModes on’, it will refuse to use your public keys in the ~/.ssh/authorized_keys file. Your home directory should be writable only by you, ~/.ssh should be 700, and authorized_keys should be 600.

ssh 为了保证通信安全，防止key被串改或窃取，对目录和文件的权限又求相当严格
也就是说当你的 /etc/ssh/sshd_config 中 StrictModes 为 yes（默认为yes）时，要求