Site Deployment Via ftp(s)

A few of my projects are on shared hosting, where I do not have shell access and cannot deploy using git. Rsync is also not an option. So I used to resort to using Krusader file manager (part of KDE ecosystem), which has a nice option to sync files over SFTP, FTP and other protocols.

And then one of the hosting providers dropped SFTP for FTP over TLS – which KDE ecosystem does not support… I had to switch to Filezilla (which does not have sync in their free ftp client) or look for other alternatives.

So I found lftp, Unix command-line ftp client, which supports multiple protocols, including FTP over TLS, and, with some help of Gemini and ChatGPT wrote a script to automate deployment.

Here is how it works:

$ ./deploy.sh app
Syncing app folder...
Synchronizing directory: '/home/me/Programavimas/myproject/app' to '/myproject/app'
source: Aplankas
Removing old file `Config/Filters.php'               
Transferring file `Config/Filters.php'
Removing old file `Controllers/Auth.php'                               
Transferring file `Controllers/Auth.php'
Removing old file `Controllers/Texts.php'       
Transferring file `Controllers/Texts.php'
Transferring file `Filters/IsLoggedIn.php'                           
Successfully synchronized '/home/me/Programavimas/myapp/app' to '/myapp/app'

Actually, the AI was great with Bash part, but not very helpful with lftp itself, giving me only non-working code, so I had to Google and read in the man page about it.

So here I share the working script, maybe it will benefit someone. [Update: the script has been turned into a package installable via composer, see https://github.com/dgvirtual/deploy-via-ftp/ There you will find an updated version of the script].

First, you will want a config file for FTP credentials, which are better kept outside of your Git repository. Also, to make the script itself more universal, the other options particular to the project should be kept in it. I named it .ftp_config :

# .ftp_config
USERNAME="myname"
PASSWORD="mySecretPass"
PROTOCOL="ftps"  # or "ftp"
PORT=21
SERVER="ftp.example.com"

# Remote directories
REMOTE_APP="/myproject/app"
REMOTE_VENDOR="/myproject/vendor"
REMOTE_PUBLIC_HTML="/public_html"

# options for excluding files/directories from sync; things that rarely
# change, should not be in production, or that should be updated manually
APP_EXCLUDE_GLOBS=""
VENDOR_EXCLUDE_GLOBS=""
PUBLIC_EXCLUDE_GLOBS="--exclude-glob='uploads/*' --exclude-glob='index.php'"

# Local directories
LOCAL_APP="app"
LOCAL_VENDOR="vendor"
LOCAL_PUBLIC_HTML="public"

Be sure to use SERVER address which which matches the server certificate (as many providers do not care to configure server to use your site sertificate). So, in my case, I could not use the real address of my website, instead using the generic address of the server.

And here is my deploy.sh script, which can be run in command line by specifying the directory to be deployed (in my case: public, app or vendor, or any combination of those), like this:

./deploy.sh app vendor

#!/bin/bash

# retrieve all variables for server, protocol and directories
source ./.ftp_config

# Base directory (where the script is executed)
BASE_DIR="$(pwd)"

# Function to synchronize a directory using lftp
sync_dir() {
  local local_dir="$1"
  local remote_dir="$2"
  local excludes="$3"  # Additional exclude patterns

  # Construct the full local directory path
  local full_local_path="$BASE_DIR/$local_dir"

  # Check if directory exists locally
  if [[ ! -d "$full_local_path" ]]; then
    echo "Error: Local directory '$full_local_path' does not exist."
    return 1
  fi

  # Debugging output to verify directory paths
  echo "Synchronizing directory: '$full_local_path' to '$remote_dir'"

  # Construct lftp command based on protocol
  if [[ "$PROTOCOL" == "ftps" ]]; then
    lftp -f "
    set ftp:ssl-force true
    set ftp:ssl-protect-data true
    open $SERVER
    user $USERNAME $PASSWORD
    lcd $BASE_DIR
    mirror -c --continue --reverse --delete --verbose $excludes $local_dir $remote_dir
    bye
    "
  else
    lftp -f "
    open $SERVER
    user $USERNAME $PASSWORD
    lcd $BASE_DIR
    mirror -c --continue --reverse --delete --verbose $excludes $local_dir $remote_dir
    bye
    "
  fi

  # Check if lftp command succeeded
  if [[ $? -eq 0 ]]; then
    echo "Successfully synchronized '$full_local_path' to '$remote_dir'"
  else
    echo "Error: Synchronization of '$full_local_path' failed."
    exit 1
  fi
}

# Function to upload a single file using curl
upload_single_file() {
  local local_file="$1"
  local remote_file="$2"

  # Construct the full local file path
  local full_local_path="$BASE_DIR/$local_file"

  # Check if file exists locally
  if [[ ! -f "$full_local_path" ]]; then
    echo "Error: Local file '$full_local_path' does not exist."
    return 1
  fi

  # Determine the URL scheme and port based on protocol
  if [[ "$PROTOCOL" == "ftps" ]]; then
    CURL_OPTS="--ftp-ssl-reqd --ftp-create-dirs --ftp-pasv"
  else
    CURL_OPTS="--ftp-pasv"
  fi

  # Upload the file using curl
  curl $CURL_OPTS --ftp-create-dirs -T "$full_local_path" --user "$USERNAME:$PASSWORD" ftp://$SERVER:$PORT$remote_file

  # Check if curl command succeeded
  if [[ $? -eq 0 ]]; then
    echo "Successfully uploaded '$full_local_path' to '$remote_file'"
  else
    echo "Error: Upload of '$full_local_path' failed."
    exit 1
  fi
}

# Check for arguments and call the appropriate function
if [[ $# -eq 0 ]]; then
  echo "Usage: $0 (app|vendor|public|onefile) [relative_local_path] [/absolute_remote_path]"
  echo "  app: Synchronize app directory only."
  echo "  vendor: Synchronize vendor directory only."
  echo "  public: Synchronize public_html directory only."
  echo "  onefile: Upload a single file using curl."
  echo "  for example:"
  echo "  ./deploy.sh onefile app/Controllers/Test.php /project/app/Controllers/Test.php"
  exit 1
fi

# Check the first argument to decide which function to call
case $1 in
  app)
    echo 'Syncing app folder...'
    sync_dir "$LOCAL_APP" "$REMOTE_APP" "$APP_EXCLUDE_GLOBS"
    ;;
  vendor)
    echo 'Remove vendor dev dependencies...'
    composer install --no-dev
    echo 'Syncing vendor folder...'
    sync_dir "$LOCAL_VENDOR" "$REMOTE_VENDOR"  "$VENDOR_EXCLUDE_GLOBS"
    echo 'Restore vendor dev dependencies...'
    composer install
    ;;
  public)
    echo 'Syncing public folder...'
    sync_dir "$LOCAL_PUBLIC_HTML" "$REMOTE_PUBLIC_HTML" "$PUBLIC_EXCLUDE_GLOBS"
    ;;
  onefile)
    if [[ $# -ne 3 ]]; then
      echo "Usage: $0 onefile [local_path] [remote_path]"
      exit 1
    fi
    echo 'Uploading single file...'
    upload_single_file "$2" "$3"
    ;;
  *)
    echo "Invalid argument: '$1'"
    exit 1
    ;;
esac

exit 0

As you see, there is an option „onefile” with additional argumens, that runs a curl command (instead of lftp). It is handy if you only need to update one file; or, as in my case, sometimes lftp gets stuck on a file for a long time failing to sync (in my case there are two such files in the project). It is some bug in lftp, which is no longer actively maintained. So, in case I change such file and it gets stuck, I apply the „onefile” command as a workaround:

./deploy.sh onefile app/Controllers/Test.php /project/app/Controllers/Test.php

If anyone finds this useful, or as some improvements to suggest – please tell me in the comments.