File Transfer on Linux Servers

This wiki provides a quick overview of common tools for transferring files between local and remote servers, using zhuyf@192.168.5.200 as an example.

---

1. scp, sftp, rsync

These three tools are all based on SSH, which means they provide encrypted and secure file transfer without requiring extra configuration if SSH access is available.

scp (Secure Copy)

Used to copy files securely between local and remote systems. Example:

# Copy local file to remote server
scp file.txt zhuyf@192.168.5.200:/home/zhuyf/

# Copy file from remote to local
scp zhuyf@192.168.5.200:/home/zhuyf/file.txt ./

sftp (SSH File Transfer Protocol)

Interactive file transfer tool, similar to FTP but encrypted with SSH. Example:

sftp zhuyf@192.168.5.200
sftp> put localfile.txt
sftp> get remotefile.txt

rsync

Efficient tool for syncing files or directories, supports incremental transfer.

Example:

# Sync local directory to remote
rsync -avz ./project/ zhuyf@192.168.5.200:/home/zhuyf/project/

# Sync local directory to remote (specify the 22 port)
rsync -avz -e "ssh -p 22" ./project/ zhuyf@192.168.5.200:/home/zhuyf/project/

# Sync remote to local
rsync -avz zhuyf@192.168.5.200:/home/zhuyf/project/ ./project/

Core Essential Parameters (Must Know)

These parameters, especially the -av combination, are used in the vast majority of rsync scenarios.

Flag	Long Form	Description	Example
-a	--archive	Archive mode. This is the most important flag. It is a combination of several other flags (-rlptgoD), which means it: • Copies directories recursively (-r) • Preserves symbolic links (-l) • Preserves permissions (-p) • Preserves modification times (-t) • Preserves group (-g) • Preserves owner (-o) • Preserves device files and special files (-D) Use this to preserve all file attributes for a perfect backup.	rsync -a source/ dest/
-v	--verbose	Verbose mode. Increases the amount of information output to the terminal, showing you which files are being transferred. Can be used multiple times (-vv, -vvv) for even more detail.	rsync -av source/ dest/
-z	--compress	Compress during transfer. Reduces the amount of data sent over the network, significantly speeding up remote transfers. Note: Has minimal effect on already compressed files (e.g., .zip, .jpg).	rsync -avz source/ user@host:dest/
-n	--dry-run	Perform a trial run with no changes. Shows what would be transferred without actually making any changes. A critical safety feature. Always use this before a new or important sync to verify the command's behavior.	rsync -avn source/ dest/
--progress		Show progress during transfer. Displays a progress bar and transfer rate for each individual file, which is useful for tracking the status of large transfers.	rsync -av --progress source/ dest/
-h	--human-readable	Output numbers in a human-readable format. Displays file sizes in K, M, G for easier reading, rather than just in bytes. Often combined with --progress or -v.	rsync -avh --progress source/ dest/

Advanced Practical Parameters

Flag	Long Form	Description	Example
-P		Combination flag for --partial --progress. • --partial: Keeps partially transferred files, enabling resumption of interrupted transfers. • --progress: Shows progress. Essential for transferring large files or over unstable connections.	rsync -avP source/ dest/
--delete		Delete extraneous files from the destination. Makes the destination an exact mirror of the source. If a file exists on the destination but not on the source, it will be deleted. Use with extreme caution! Always test with -n first.	rsync -av --delete source/ dest/
-e		Specify the remote shell to use. Most commonly used to specify a custom SSH port or SSH options.	rsync -av -e "ssh -p 2222" source/ user@host:dest/
--exclude=PATTERN		Exclude files matching PATTERN. Can use wildcards (e.g., *.log, temp/).	rsync -av --exclude='*.tmp' source/ dest/
--exclude-from=FILE		Read exclude patterns from a specified FILE, one pattern per line. Good for complex exclusion rules.	rsync -av --exclude-from='exclude-list.txt' source/ dest/
--include=PATTERN		Include files matching PATTERN. Typically used in combination with --exclude for complex filtering logic.	rsync -av --include='*.txt' --exclude='*' source/ dest/
--max-size=SIZE		Do not transfer any file larger than SIZE. (e.g., --max-size=100m, --max-size=1G).	rsync -av --max-size=50m source/ dest/
--min-size=SIZE		Do not transfer any file smaller than SIZE.	rsync -av --min-size=10k source/ dest/

Classic Usage Scenarios & Examples

1. Local Backup (Most Common)

# Synchronize the ~/documents directory to a backup drive, preserving all attributes and showing progress.
# Note the trailing slash on the source path.
rsync -avh --progress ~/documents/ /mnt/backup/documents/

2. Remote Backup (via SSH)

# Push a local directory to a remote server (Upload)
rsync -avz -e ssh /local/path/ username@remote_host:/remote/path/

# Pull a remote directory to the local machine (Download)
rsync -avz -e ssh username@remote_host:/remote/path/ /local/path/

3. Safe Trial Run and Deletion (Mirroring)

# 1. First, do a dry run to see what would be deleted. THIS IS A SAFETY STEP.
rsync -avn --delete source/ dest/

# 2. After verifying the output, execute the command for real to mirror the destination.
rsync -av --delete source/ dest/

4. Resuming Interrupted Transfers

# The -P flag allows rsync to resume the transfer of a large file if it gets interrupted.
rsync -avzP -e ssh large_file.tar.gz user@host:~/backups/

5. Excluding Specific Files and Directories

# Exclude all .log files and any directory named 'cache'
rsync -av --exclude='*.log' --exclude='cache/' source/ dest/

# Use a file for a complex list of excludes
rsync -av --exclude-from='exclude_list.txt' source/ dest/

⚠️ Important Note: Trailing Slash (/) Behavior

Understanding the trailing slash on the source path is critical: * rsync -av /home/user/data /backup/ * This will create a directory called data inside /backup/. * Result: /backup/data/contents_of_data * rsync -av /home/user/data/ /backup/ * This will copy the contents of the data directory directly into /backup/. * Result: /backup/contents_of_data

Best Practice: Always be intentional with your trailing slashes. Using a trailing slash on the source is common for backups to avoid creating an extra directory level at the destination.

---

🔹 Comparison

Tool	Transport	Protocol/Mechanism	Best for
scp	SSH	Remote copy over SSH	Quick one-time transfers
sftp	SSH	SFTP protocol (subsystem of SSH)	Interactive file management
rsync	SSH or rsync daemon	Rsync algorithm (delta transfer)	Incremental sync & backup

---

2. sz and rz

Unlike scp, sftp, and rsync (which are based on SSH), sz and rz use the Zmodem protocol, a legacy file transfer protocol originally designed for serial communication. They are commonly integrated into terminal emulators (e.g., Xshell, SecureCRT) to allow quick file upload/download inside a terminal session.

🔹 Comparison with SSH-based tools

Feature	SSH (scp/sftp/rsync)	Zmodem (sz/rz)
Type	Secure network protocol	File transfer protocol
Transport	TCP/IP (port 22)	Serial/terminal stream
Security	Encrypted (RSA/AES)	No encryption (CRC only)
Main Use	Remote login + file transfer	File transfer in terminal emulators
Modernity	Still mainstream	Legacy, niche use

---

Usage Examples

• sz → Send file from server to client
• rz → Receive file from client to server

Example (on server side):

# Send a file to client
sz file.txt

# Receive file from client
rz

3. Automation with expect

expect automates interactive SSH or file transfer commands (e.g., typing password).

The expect command offers various advanced parameters and features. Readers who wish to explore further may search for additional resources.

Example 1: Automated SSH Login

#!/usr/bin/expect -f
# Description: Auto-login to SSH server with password authentication

# Configuration
set host "192.168.1.100"
set user "root"
set password "123456"

# Start SSH session
spawn ssh $user@$host

# Handle interaction
expect {
    "yes/no" { 
        send "yes\r"      # Confirm SSH fingerprint prompt
        exp_continue      # Continue expecting other patterns
    }
    "password:" { 
        send "$password\r" # Send password 
    }
    timeout {
        puts "Connection timed out"
        exit 1
    }
}

# Return control to user after login
interact

Example 2: Automated SCP File Transfer

#!/usr/bin/expect -f
# Description: Securely copy files via SCP with password automation

# Configuration
set host "example.com"
set user "admin"
set password "secret"
set local_file "/data/test.txt"
set remote_dir "/tmp/"

# Initiate SCP transfer
spawn scp $local_file $user@$host:$remote_dir

# Handle authentication
expect {
    "password:" { 
        send "$password\r"  # Send SCP password
    }
    timeout {
        puts "Error: Operation timed out"
        exit 1
    }
}

# Wait for transfer completion
expect eof

Example 3: Batch Remote Command Execution

#!/usr/bin/expect -f
# Description: Execute commands on multiple servers sequentially

# Server list
set hosts { "host1" "host2" "host3" }
set user "admin"
set password "123456"

foreach host $hosts {
    puts "Connecting to $host..."

    # Establish SSH connection
    spawn ssh $user@$host

    expect {
        "password:" { 
            send "$password\r"  # Authenticate
            exp_continue
        }
        "$ " {                 # Wait for shell prompt
            send "uptime\r"     # Execute command
            expect "$ "
            send "exit\r"       # Close session
        }
        timeout {
            puts "Timeout connecting to $host"
            continue           # Skip to next host
        }
    }

    expect eof  # Ensure clean session closure
}

---

Appendix: wget and curl

• wget: Non-interactive downloader, supports HTTP, HTTPS, FTP.
• curl: More flexible tool for data transfer and API requests.

---

Comments