pgstef's blog

SELECT * FROM pgstef

Home About me Talks PITR tools View on GitHub
2022-02-23

pgBackRest preview - Bundle files in the repository during backup

A nice new feature has been added on 14 Feb 2022: Bundle files in the repository during backup. Details can be found in commits 34d6495 and efc09db.

This feature combines smaller files during backup to reduce the number of files written to the repository (enabled with --bundle). Files are batched up to --bundle-size and then compressed/encrypted individually and stored sequentially in the bundle. --bundle-limit limits which files can be added to bundles. Files larger than this size will be stored separately. On backup resume, the bundles are removed and any remaining file is eligible to be resumed.

IMPORTANT NOTE: this feature is still experimental and will not be released in pgBackRest 2.38.

Read More
2022-02-21

pgBackRest and TLS connections

The TLS server is an alternative to using SSH for protocol connections to remote hosts. It has been pushed in the 2.37 release on 3 Jan 2022.

In this demo setup, the repository host will be called backup-srv and the 3 PostgreSQL nodes in Streaming Replication: pg1-srv, pg2-srv, pg3-srv. All the nodes will be running on Rocky Linux 8.

If you’re familiar with Vagrant, here’s a simple Vagrantfile to initiate 4 virtual machines using those names:

# Vagrantfile
Vagrant.configure(2) do |config|
    config.vm.box = 'rockylinux/8'
    config.vm.provider 'libvirt' do |lv|
        lv.cpus = 1
        lv.memory = 1024
    end

    # share the default vagrant folder
    config.vm.synced_folder ".", "/vagrant", type: "nfs"

    nodes  = 'backup-srv', 'pg1-srv', 'pg2-srv', 'pg3-srv'
    nodes.each do |node|
        config.vm.define node do |conf|
            conf.vm.hostname = node
        end
    end
end
Read More
2021-06-03

pgBackRest and SUSE

I got the question the other day from a friend: “does pgBackRest work on SUSE?”. Having to admit I never really used SUSE, and not knowing what to answer, I decided to give it a try. Let’s see in this short post how far we can go.

Read More
2021-02-10

check_pgbackrest 2.0 has been released

check_pgbackrest is designed to monitor pgBackRest backups, relying on the status information given by the info command.

The biggest change in this new release is that the tool will now only support pgBackRest 2.32 and above in order to only use its internal commands. This remove Perl dependencies no-longer needed to reach repository hosts or S3 compatible object stores. This also brings Azure compatible object stores support. The repo-* arguments have then been deprecated.

pgBackRest 2.32 has been released this week and brings official support for the repository commands: repo-ls and repo-get.

Let’s find out in this post how nice this feature is.

Read More
2021-02-01

Combining Barman and EDB Postgres Advanced Server

Barman allows you to implement disaster recovery solutions for databases with high requirements of business continuity.

Traditionally, Barman has always operated remotely via SSH, taking advantage of rsync for physical backup operations. Version 2.0 introduces native support for Streaming Replication backup operations, via pg_basebackup.

Choosing one of these two methods is a decision you will need to make. The official documentation deeply describes how to implement it and covers a lot of important general considerations.

In this post, we’ll briefly see how to setup a dedicated Barman server to backup an Advanced Server 3-nodes cluster, using the SSH method.

Read More
2020-09-04

Combining pgBackRest and Streaming Replication, PG13 update

pgBackRest is a well-known powerful backup and restore tool. It offers a lot of possibilities.

While pg_basebackup is commonly used to setup the initial database copy for the Streaming Replication, it could be interesting to reuse a previous database backup (eg. taken with pgBackRest) to perform this initial copy.

This content updates one of my old posts, using PostgreSQL 13 and the latest pgBackRest version.

Read More
2020-05-25

pgBackRest preview - A tour of retention policy options

pgBackRest is a well-known powerful backup and restore tool. Old backups and archives are removed by the expire command based upon the defined retention policy.

Since the latest version published last month, new features regarding retention have been committed. We’ll here first overview those changes and then make a tour of the retention policy options that should be available in the next release.

Read More
2020-05-19

pgBackRest preview - Info command and backup/expire running status

pgBackRest is a well-known powerful backup and restore tool. The 2.26 version has been released on Apr 20, 2020. New features have been developed since then.

Today, let’s have a look at: add backup/expire running status to the info command.

This is implemented by checking for a backup lock on the host where info is running so there are a few limitations:

* It is not currently possible to know which command is running: backup, expire, or stanza-*. 
The stanza commands are very unlikely to be running so it's pretty safe to guess backup/expire. 
Command information may be added to the lock file to improve the accuracy of the reported command.

* If the info command is run on a host that is not participating in the backup, e.g. a standby, then there will be no backup lock. 
This seems like a minor limitation since running info on the repo or primary host is preferred.
Read More