WordPress Performance Profiling with Sail CLI

Having to deal with performance problems on a WordPress site is never too pleasant, partly because the tooling is not great. Things like Query Monitor and the Debug Bar series of plugins can certainly help to some extent, but often times they’re not enough, because they do things in PHP, which is limited to, well… PHP.

Moreover, when reporting on database queries or remote HTTP requests, these tools will rely on the WordPress database and HTTP APIs, so if some theme or plugin happens to do a direct call to mysqli_query(), or file_get_contents(), etc., which is often the case with external third-party libraries, then you’re out of luck.

This is why you often need an actual profiler to get the job done.

Continue reading

wp_mail() is NOT broken

So why is everybody trying to fix it? I’ll try to explain.

A few of months ago I started working on a new open source project called Sail, which is a CLI tool to provision WordPress servers on DigitalOcean. I wanted the best possible e-mail configuration I could have, fully transparent to the WordPress user, so I did some research. A lot of research actually.

The first thing I noticed, is that Google is just filled with search results about how broken the wp_mail() function is in WordPress, and how you need a ton of plugins and paid services to fix it. And yet, when I looked deeper into that “fix” that was offered, I was quite surprised, and also a bit disappointed with some of our beloved hosting providers.

But what do hosting providers have to do with e-mail? Well, nothing. But also everything.

Continue reading

Configuring Mailgun for WordPress with Sail CLI

Mailgun is a robust e-mail delivery service with both API and SMTP support, and a generous trial of up to 5000 e-mails/mo for three months. Together with Mailgun, we’ve developed a couple of default blueprints for Sail to ease the configuration.

  • mailgun-dns.yaml: this blueprint adds all necessary DNS records for mail delivery through Mailgun, as well as domain verification at Mailgun
  • mailgun.yaml: extends the default Postifx configuration with Mailgun-specific settings

If haven’t used sail before, it’s a free and open source CLI tool, that helps you provision and deploy WordPress applications to the DigitalOcean cloud. Sail is available for Windows, macOS and Linux. Click here to learn more about Sail.

Add and Verify a Domain

Before running any Sail blueprints you’ll need to get an account at Mailgun. Then add your domain name to Mailgun under SendingDomainsAdd New Domain. After the domain is added you’ll see a list of DNS records, required for mail delivery and domain verification (SPF, DKIM, MX and CNAME records).

Next, add this domain to your DigitalOcean account using Sail:

$ sail domain add example.org

Make sure the domain name server records at your domain registrar are pointed to the DigitalOcean DNS servers:

  • ns1.digitalocean.com
  • ns2.digitalocean.com
  • ns3.digitalocean.com

Here’s how to do that with common domain registrars.

To add the Mailgun DNS records to your domain, run the mailgun-dns.yaml blueprint with Sail:

$ sail blueprint mailgun-dns.yaml
> Domain Name: example.org
> DKIM Record Name: mx._domainkey
> DKIM Record Value: k=rsa; p=...
# Applying blueprint: mailgun-dns.yaml

Note: the DKIM record name should not include the full host/domain, but only the DNS record name. So if the full host name is mx._domainkey.example.org, you’ll need to enter just the mx._domainkey part (no trailing dot).

After the DNS records have been added to your DigitalOcean account, visit your Mailgun control panel and under SendingDomainsYour Domain, hit the Verify DNS Settings button.

Configure Postfix for Mailgun

After verifying the domain with Mailgun, grab your SMTP settings from your Mailgun control panel under SendingDomain SettingsSMTP Credentials. You’ll need the SMTP login and password here. You might have to reset the password if you’d like to use the default postmaster user.

Armed with these credentials, you can now run the mailgun.yaml blueprint with Sail:

$ sail blueprint mailgun.yaml
> Mailgun SMTP Login: postmaster@example.org
> Mailgun SMTP Password: ...
> From name: WordPress
> From e-mail: hi@example.org
# Applying blueprint: mailgun.yaml

The From name and From e-mail values are what’s going to be used with wp_mail() by default in WordPress, when setting the From: header for e-mail delivery. For best results, it is recommended that you use an e-mail address within the domain you’re using with Mailgun.

This blueprint will also create a mu-plugin on your production site, which sets the from name and e-mail. Be careful not to overwrite/delete this plugin when deploying changes to your production site. It is always recommended to run a `sail download` after any blueprint, to make sure your local copy is completely in sync.

Send a Test E-mail

You can send a test e-mail using Sail, to make sure it reaches your inbox:

$ sail wp shell
wp> wp_mail( 'your@email.address', 'Test from Sail', 'The body' );
=> bool(true)

If it has not reached your inbox, there are a few places you could look at for more information:

  1. Postfix logs using sail logs --postfix
  2. Mail logs in the Mailgun dashboard
  3. Your spam folder, of course

If you’re having problems with e-mail delivery in Sail for WordPress, please double-check to make sure your DNS records are correct, your SMTP credentials are okay. If they are and you’re still experiencing trouble, feel free to reach out Sail support or Mailgun support for help.

Mail Performance

I’ll be doing a mail performance test of this configuration, and possibly fine-tuning it during a live stream via Koddr.io on September 28th at 17:00 UTC. Let’s see what it takes to send 10,000 e-mails from a five-dollar DigitalOcean droplet.

If you’d like to be notified (and also support us in general), you can:

See you soon!

Fun with Blueprints in Sail CLI for WordPress

Blueprints allow Sail users to define an environment, where their WordPress applications will be provisioned.

Currently blueprints support plugins (wp.org or custom), themes, options and wp-config.php constants. In future updates we’ll add support for Redis/Memcached setups, Mail, security (fail2ban, etc.) and much more. Possibly some DigitalOcean features too, like floating IPs and volumes.

Here’s a demo overview of how blueprints work:

More information in the GitHub repo.

Push-to-Deploy with Sail and GitHub Actions

Sail supports deploying WordPress out of the box, without the need of Git or any other source code management tools. This is great for solo-projects, or simple applications with very small teams.

With larger teams and more complex WordPress applications, you’ll want a more robust workflow, including pull requests, code reviews, etc.

GitHub Actions is one of the best CI/CD tools on the market, and personally my favorite. In this short tutorial, I’ll demonstrate how to use Sail with GitHub Actions to quickly deploy your WordPress applications to the DigitalOcean cloud.

Create a new GitHub repository

Let’s start by creating a new repository on GitHub and cloning it to our new project directory.

$ mkdir project && cd project
$ git clone git@github.com:username/repo.git .

Add a .gitignore file in the root of our project, to make sure things, such as uploads, backups or secrets don’t end up in the repository. Here’s the bare minimum I recommend:

.*
wp-content/uploads
wp-content/upgrade
wp-content/cache

Then add the .gitignore file to the repository and commit our changes:

$ git add .gitignore
$ git commit -m "Adding a .gitignore file"

Create a new Sail project

If you’re new to Sail, it’s a fresh CLI tool I wrote to provision and deploy WordPress to DigitalOcean. It takes a couple of minutes to install via Homebrew on Linux, MacOS and Windows.

In our project directory, start a new Sail application with sail init:

$ sail init

This will provision and deploy a WordPress instance to our DigitalOcean account, as well as pull all the application files from the new production server to our local working copy. We can commit these files to our Git repository:

$ git add .
$ git commit -m "Adding a fresh WordPress application"

Encrypt Sail secrets

Sail stores some secrets about your project in a hidden .sail directory, in the root of your project. You should never commit this directory unencrypted to your Git repository, as it contains SSH keys with root access to the production server, and other sensitive information.

However, we’ll need these secrets for GitHub Actions to deploy our Sail project, so we’ll generate a passphrase and encrypt the entire directory with GPG:

$ tar czf - .sail | gpg -c > .sail.tar.gz.gpg

You’ll be prompted to enter a new passphrase twice. Make sure it’s long and secure. We can then commit the encrypted file to the Git repository, and push all our changes to GitHub.

$ git add .sail.tar.gz.gpg
$ git commit -m "Add Sail secrets"
$ git push

We’ll also need to share the passphrase we used with GitHub, to make sure our Workflow can decrypt our file and read the contents. On the GitHub repository page open the Settings tab, select Secrets from the menu on the left, and then hit the New repository secret button in the right hand corner.

Use SAIL_PASSPHRASE as the secret name, and the passphrase you entered earlier as the value.

Create a GitHub Workflow

Create a new .github/workflows/deploy.yml file in your repository:

$ mkdir -p .github/workflows
$ touch .github/workflows/deploy.yml

Open the file in your favorite code editor, and paste the following contents:

name: deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: kovshenin/sail-deploy@v1
      with:
        passphrase: ${{ secrets.SAIL_PASSPHRASE }}
        encrypted_filename: '.sail.tar.gz.gpg'

The first line is just the name of the workflow, we’ll call it “deploy.”

The next few lines (the “on” section) determine the branches and actions, on which the workflow is going to run. In our case, we’re going to run this workflow whenever something is pushed to the branch called main.

Next is the jobs section, which describes exactly what to do. In our case, we’re going to run some steps, using the ubuntu-latest GitHub image. The steps themselves reference external reusable actions. The first one, actions/checkout clones the Git repository and does a checkout of the main branch.

The second is a custom action I wrote called sail-deploy, which essentially installs Sail, decrypts and extracts the Sail secrets archive using the provided passphrase, and then runs sail deploy.

Add and commit the file to your Git repository, and push the change to GitHub:

$ git add .github/workflows/deploy.yml
$ git commit -m "Adding the deploy workflow"
$ git push

Testing

To test your deployment, simply make a change to your repository and push it to your main branch. For example, create a new test.php file:

$ echo "<?php echo time();" > test.php
$ git add test.php
$ git commit -m "Testing deploys"
$ git push

Then go to your GitHub repository page, hit the Actions tab and watch the deploy run. When successful, you should be able to visit the test.php page of your site and get a timestamp.

Now, whenever you push changes to the main branch of your GitHub repository, a GitHub Action will use Sail to deploy those changes to your production environment, in an atomic way.

Of course you can continue to use Sail in your local working directory for ad-hoc deploys, quick rollbacks, domain management, backups and more.

If you have trouble setting this up for your project, or have questions or any other feedback, feel free to leave a comment below, or message me on Twitter and I’ll be happy to help you out. If you’re working on an existing project, don’t forget to check out these Sail migration tips on GitHub.

Happy sailing!

Sail: Deploy WordPress to DigitalOcean

Sail is a free and open source CLI tool to provision and deploy WordPress applications to the DigitalOcean cloud. Here’s a quick video demo of how it works:

I’m a DIY guy when it comes to WordPress hosting, so I like to get my hands dirty with servers, code, configuration and everything else. I’ve been using virtual servers at DigitalOcean for small WordPress projects for a very long time, and it’s great, and also very affordable.

However, it’s a bit annoying to do routine maintenance on existing servers, or provision and configure new servers for newer projects so, like most developers, I wrote a bunch of scripts, and used them for many many years.

Over the last couple months I’ve decided to clean up (rewrite) all those scripts and package them into one easy to use CLI tool, which I called sail. It’s open source on GitHub, and available for Linux, MacOS and Windows through Homebrew and PyPI.

The Competition

Sure, there are plenty of existing products and services for managing WordPress on DigitalOcean and other cloud providers, and trust me, I tried them all. Every single one of them beasts. Here are some of the problems I had:

  • A lot of them don’t provide vanilla WordPress, they bundle stuff from their partners which I don’t want
  • Most of them lack any sort of deployment tools, so I have to set things up on my own
  • Many of them won’t give me root access to the server I’m paying for, and some will not even let me use my own DigitalOcean account to run the VMs
  • Most of them are web GUIs, while I always prefer the command line for such things
  • Some of them charge me double the droplet price for features and services which I’ll never use

Sail is free and open source, and it allows you to:

  • Quickly provision a clean WordPress site to your DigitalOcean account
  • Deploy code changes and rollback in quick atomic operations
  • Add domains and free SSL certificates through Let’s Encrypt
  • Create and restore complete file and database backups
  • Quickly access server logs, production SSH, WP-CLI and MySQL shells
  • With full root access, and all from the command line

Roadmap

While my focus right now is to complete and polish all the core Sail features, I do have some more exciting things on the roadmap for the next few releases. This is not a promise, but rather a taste of what’s coming next:

  • Blueprints, which will allow you to specify additional plugins, themes, settings and server software to launch with your Sail project
  • Staging, and all the pushing/syncing to and from and between
  • Profiling, because every millisecond counts

Download

The easiest way to get Sail is from Homebrew or PyPI. It run on Linux, MacOS and Windows (via WSL). Give it a spin, I think you’ll love it. And if you don’t, let me know why in the comments below.

Rsync’s link-dest: Not Great for Deployments

TIL: rsync’s --link-dest is pretty bad for deploying code to production servers, unless you can get some fancy copy-on-write going on.

Rsync is probably the best utility to transfer large numbers of files from one location to another, quickly and effectively. The --link-dest argument allows you to hard-link files from a different destination if they haven’t changed, saving both time on transfer, as well as disk space.

It’s perfect for backups, and seemed to me like it would be a good idea for code deployments as well. But I was wrong.

Deploying to production means you have a particular copy lying around, that is not unlikely to change especially in single-server setups, where user actions, such as a WordPress core or plugin update, can lead to changes on the filesystem.

So if you change a file that happened to be hard-linked to other releases lying around for a potential emergency rollback, then you’ve effectively just borked them all :(

$ mkdir -p releases/1
$ echo good > releases/1/wp-config.php

$ rsync -rt --link-dest=../1/ releases/1/ releases/2/
$ rsync -rt --link-dest=../2/ releases/2/ releases/3/
$ rsync -rt --link-dest=../3/ releases/3/ releases/4/

$ ln -sfn releases/4 latest
$ echo bad > latest/wp-config.php

# I screwed up, I'm going to roll back to release 1.
# Which I know was good... Right?
$ cat releases/1/wp-config.php
bad

Next best option is --copy-dest just to speed up transfer, but not preserve disk space.

LIVE: Creating a Caching Plugin for WordPress from Scratch

Join me live as I write a page caching plugin for WordPress from scratch. For educational purposes of course, as part of our advanced WordPress training program over at Koddr.io. I’ll be tearing apart some existing caching plugins to find out how they work, then build my own using similar concepts.

I’ll be doing a deep dive into WordPress’ advanced-cache.php drop-in and covering everything you’ll ever need to know about it.

Bring your favorite beverage and join me on YouTube or Twitch.tv.