SSH

Synopsis

HELP:   Remote SSH (SFTP)
TYPE:   OBJECT
SYNTAX: SSH(HOST/SERVER/DAEMON='str',PORT/SERVICE=num,USER='str',PASSWORD='str',CONFIGFILE='str',NODEFAULTCONFIG,KNOWNHOSTS='str',PUBKEYFILE='str',PRIKEYFILE='str',PWDKEYFILE='str',HOSTKEYCHECK='str'/STRICT/ASK/WARN/ACCEPTANY,PCAPFILE='str',VERBOSITY/VERBOSE[=SYSTEM/INFO/PROTOCOL/PACKET/FUNCTION/ALL],TIMEOUT=num,ALGORITHMS())

Description

The SSH method provides secure remote file access through the SSH file transfer protocol (SFTP). Due to the transport encryption of the SSH protocol, the transferred data is safe from eavesdroppers even if the data itself is unencrypted. The remote host only needs a running SSH server (e.g. OpenSSH).

FLAM can read and write most locally supported types of files also via SSH, including the use of wildcards and recursion into directories. Only FLAMFILEs (work in progress) and record-oriented host datasets are currently not supported via SSH (limitation of the SSH server).

Authentication methods supported are password and public key authentication. There are two ways of specifying authentication parameters: via the NET-SSH object or the file URL.

The file URL may contain the SSH username, the password (is using password authentication), hostname and port. The NET-SSH object also allows to specify the paths of the known hosts, public and private key files as well as the private key password (if applicable) when using password authentication. Parameters specified in the file URL always take precedence over parameters set via the NET-SSH object.

To successfully establish an SSH connection, at least a hostname/IP is required. Other options can be specified or can be loaded from a OpenSSH config file.

By default, options are loaded from the per-user (~/.ssh/config) and system-wide (/etc/ssh/ssh_config) OpenSSH config files, if they exist. To disable loading those default config files, use the NODEFAULTCONFIG switch. A custom OpenSSH configuration file can be set with the CONFIGFILE option.

SSH configuration options are priorized in the following order:

  1. Command string options (net.ssh())
  2. Custom configuration file (net.ssh(configfile=...))
  3. Per-user configuration file (~/.ssh/config)
  4. System-wide configuration file (/etc/ssh/ssh_config)

Example: The known_hosts file path is set via net.ssh(knownhosts=...) and in one of the config files with the UserKnownHostsFile keyword. Since the options specified in the command string has higher priority, it is used.

Below is an (incomplete) list of keywords of the most important keywords that are supported in SSH config files. They follow the OpenSSH syntax. Please refer to OpenSSH's documentation for a description:

The port defaults to 22 and the username defaults to the local username unless specified in the command or one of the loaded config files.

If a password is specified, authentication via password is attempted.

For public key authentication, the paths to a public and/or private key file can be specified (via the net.ssh() object). A private key is mandatory. There are three cases:

  1. Public and private key file specified: Public key authentication with this keypair is performed
  2. Only a private key file specified: The public key file path is built by appending the extension .pub to the private key file path. If no such file exists, the public key is extracted from the private key. This has a security implication: If the server rejects the public key for authentication, the private key was decrypted unnecessarily. If the public key is stored in a separate file, the private key would remain encrypted in the case of public key rejection by the server.
  3. Only a public key is specified: It is not possible to authenticate with only a public key. If the public key filename ends with .pub, this extension is stripped to build the private key filename. If no such file exists or the public key filename suffix is not .pub, authentication will fail.

If neither a password nor a public/private key file are specified, public key authentication in automatic mode is attempted. First, authentication via an SSH agent (e.g. from OpenSSH) is attempted if one is running and the environment variable SSH_AUTH_SOCK is set to the socket path of the SSH agent. If authentication via SSH agent fails, the keys defined in ~/.ssh/config and /etc/ssh/ssh_config are tried (keyword: IdentityFile) if present. If that fails as well, a series of default keys of the local user account is used (stored in ~/.ssh/id_*, e.g. id_ed25519, id_ecdsa, id_rsa, id_dsa).

When establishing a connection, a known host check is performed to prevent an attacker from performing a man-in-the-middle attack and impersonate the destination host. For this purpose, the per-user known_hosts file (~/.ssh/known_hosts) and the system-wide file (/etc/ssh/ssh_known_hosts) are used by default. A different per-user known_hosts file can be specified with the KNOWNHOSTS option. A known_hosts consists of one line per previously seen host, each containing the hostname (or hash), key type and public key of the host. Establishing a connection will fail by default (strict hostkey checking) if the host is not in the known_hosts file or if the public key presented by the server during connection establishment does not match the key in known_hosts file.

FLAM is designed to run without human interaction. Therefore, by default, connecting fails if no known_hosts entry is found. To connect to a host you have never connected to before, a known_hosts entry can be added manually. FLAM can also add the missing host entry for you to the per-user known_hosts. This, however, requires to activate interactive mode (hostkeycheck=ask). When you connect to a new host in interactive mode, the fingerprint of the remote host is displayed. You should verify that the fingerprint is correct (to avoid adding a key from a man-in-the-middle attacker that might have control over the communication path to the server) and confirm the key to have it added to your known hosts file. There are more hostkey check mechanisms implemented and described in a dedicated chapter.

The parameter VERBOSITY (or VERBOSE) activates connection debug logging (CONDBG) in the log output. There are several levels of verbosity to control the amount of logged messages. The SYSTEM level logs some details about the connection (e.g. negotiated algorithms). Levels other than SYSTEM additonally log messages from the SSH library (libssh).

Examples:

Establish a connection to an SSH host for the first time by trying to read a dummy file (does not necessarily have to exist) in interactive mode:

flcl xcnv "inp(net.ssh(host=host.example.com hostkeycheck=ask) sav.fil(fio.blk(name=somefile.dat)))"

Read a file "myfile.bin" from the home directory of a remote host using the default username (either from config file or the local username) in different but equivalent ways by using default public key authentication:

flcl conv "read.bin(file='ssh://host.example.com/myfile.bin')"
flcl conv "read.bin(file=myfile.bin net.ssh(host=host.example.com))"
flcl xcnv "inp(net.ssh(host=host.example.com) sav.fil(fio.blk(name=myfile.bin)))"
flcl xcnv "inp(sav.fil(fio.blk(name='ssh://host.example.com/myfile.bin')))"
flcl xcnv "inp(net.ssh(port=22) sav.fil(fio.blk(name=ssh://host.example.com/myfile.bin')))"
flcl xcnv "inp(sav.fil(fio.blk(name='ssh://host.example.com:22/myfile.bin')))"

Read a file with password authentication and a custom username:

flcl conv "read.bin(file=myfile.bin net.ssh(host=host.example.com user=admin pass=123456))"
flcl conv "read.bin(file='ssh://admin:123456@host.example.com/myfile.bin'))"
flcl xcnv "inp(net.ssh(host=host.example.com user=admin pass=123456) sav.fil(fio.blk(name=myfile.bin)))"
flcl xcnv "inp(net.ssh(pass=123456) sav.fil(fio.blk(name='ssh://admin@host.example.com/myfile.bin')))"
flcl xcnv "inp(sav.fil(fio.blk(name='ssh://admin:123456@host.example.com/myfile.bin')))"

Read from an absolute path:

flcl conv "read.bin(file='ssh://host.example.com//my/path/file.bin'))"
flcl xcnv "inp(net.ssh(host=host.example.com) sav.fil(fio.blk(name=/my/path/file.bin)))"
flcl xcnv "inp(sav.fil(fio.blk(name='ssh://host.example.com//my/path/file.bin')))"

Read and write from/to different remote hosts, one with public key, the other with password authentication and a non-default port:

flcl conv "read.bin(file='ssh://user@host1.example.com//my/path/file.bin')) write.bin(file='ssh://admin:123456@host2.example.com:2222//other/path/file.bin'))"
flcl conv "read.bin(file='/my/path/file.bin' net.ssh(host=host1.example.com user=user))) write.bin(file='/other/path/file.bin' net.ssh(host=host2.example.com port=2222 user=admin pass=123456)))"

Get information for all text files in a remote home directory with several connection debug information and specification of the known host file:

flcl info "get.file='ssh://user@host1.example.com/*.txt' net.ssh(knownhosts='/home/user/.shh/known_hosts' verbose #=system#)"
flcl info "get.file='ssh://user@host1.example.com/*.txt' net.ssh(knownhosts='~/.shh/known_hosts' verbose=info)"
flcl info "get.file='ssh://user@host1.example.com/*.txt' net.ssh(knownhosts='/home/user/.shh/known_hosts' verbose=protocol)"
flcl info "get.file='ssh://user@host1.example.com/*.txt' net.ssh(knownhosts='~/.shh/known_hosts' verbose=packet)"
flcl info "get.file='ssh://user@host1.example.com/*.txt' net.ssh(knownhosts='/home/user/.shh/known_hosts' verbose=all #=function#)"

Arguments