Backup on Renne Rocha

Using rclone on Backblaze without spending all your money

blog@rocha.dev.br (Renne Rocha) — Thu, 14 Aug 2025 00:00:00 +0000

Backblaze is an object storage company that offers a S3 compatible cloud object storage with very competitive prices. With prices starting at USD6 / TB / month (or less, as you pay proportionally to the amount of data stored), it was a perfect choice to store live data and backups of my projects and self-hosted services.

One of the services that I use is Immich, a self-hosted photo and video management solution that allows me to have fully control on my photos (even in mobile). It doesn’t allows me to store my photos directly in a object storage as Backblaze, but I decided that it would be a good idea to keep a backup of the files of the server (photos, videos, and some other data that I need to keep safe).

One tool that simplifies the process is rclone, a tool that make it easier to sync data in cloud providers (their maintainers call it as “rsync for cloud storage”). I followed of the instructions they provide to configure Backblaze and then I was able to run the following command:

rclone sync /home/immich/library/ b2:my-immich-backup-bucket

This command syncs automatically only new and changed files (e.g. updated photos metadata), reducing the time needed to backup my photos and keeping them updated, uploading to my Backblaze bucket.

There are many other options that I can use with rclone, and scenarios where this backup scheme doesn’t cover. However, as a starting point to give me a bit more of confidence that I will not lose my photos if my VPS provider suddenly decides to stop providing me services, it is good enough.

I configured crontab to run it hourly, and then I start to get my files synchronized.

So I got the first charge after enabling this scheme! Usually I was paying less than USD2/month, but the first month I got a USD10 bill, then USD60, then USD70. I wasn´t storing terabytes of data. Looking at my billing report I noticed that now I have a not negligible amount of Class C (Charged) transactions, that are responsible for me to pay that much.

We interact with Backblaze using API calls to manage our storage, so we can upload and download files, get information about them and the buckets, and many other actions. Each call is grouped in a different class of transaction (A, B and C), and they are charged differently. Class A transactions are always free, but Class B and C transactions have a daily limit of 2,500 calls each and they charge for extra calls.

When rclone executes, it gets information from the remote location about the files already there to decide if the local files need to be uploaded or not. This is done by the Class C API call b2_list_file_names that returns a list of filenames in chunks of 1.000 files each call.

But when we have our files in nested directories, a call is made for each of them. Immich organizes the uploaded files in hundreds of nested directories, and because of that, every time my cron was executed, thousands of b2_list_file_names calls were made. Given that I was running it hourly, I reached 17M calls in a month easily (around 500k a day). As I have only 2.5K free calls, I found the reason of the amount I was charged.

The solution is to add --fast-list parameter to the rclone command, that requires fewer transactions for highly recursive operations as we have in this scenario. This parameter is mentioned in rclone documentation that explains how to setup it with Backblaze, but I didn’t consider that this is basically a mandatory parameter based on how things are charged there!

So I updated my rclone command to:

rclone sync /home/immich/library/ b2:my-immich-backup-bucket --fast-list

I reactivated cron again and after almost a day, instead of 500K calls, less than 1K calls were made (which is within the free amount I am entitled to). Probably when I have the double of the number of files I have right now, the daily quota I have will not be enough, but then I will know how to deal with it (possibly with a better backup strategy).

Creating backups for fly.io Volumes

blog@rocha.dev.br (Renne Rocha) — Wed, 30 Apr 2025 00:00:00 +0000

I have a few applications running on fly.io, and some of them need to keep data in the file system persistently (more precisely, an SQLite database file and user-submitted data) so that it is not lost after a redeploy or when the Fly Machine running my application is restarted.

To achieve that, I use Fly Volumes which are local persistent storage for Fly Machines, mounted in my server just like a regular directory. This setup works fine, but I began considering how to back up the data stored there.

Volume snapshots are created automatically on a daily basis and retained for 5 days by default. However, there doesn’t seem to be an easy (or well-documented) way to implement a custom backup policy. I wanted the ability to copy the entire directory’s content using tools like rsync or upload it to an S3 bucket on my own schedule.

I explored solutions involving cron jobs running inside my Fly Machine, but they became overly complicated. These approaches required modifying my Dockerfile to install additional applications, and I wasn’t sure how to manage the schedule effectively, especially since I configured my machines to auto-stop to save resources.

Direct SSH connections requires me to use flyctl CLI and it wasn’t clear to me how to handle authentication in this case. After some research, I found that I can use access tokens to connect to the machines using SSH allowing me to send commands there in an automated way.

Generating your access token

First step is to create an access token that allows me to send commands to my machine without requiring any manual form of authentication. This can be done using the following command:

`1`	`fly tokens create ssh -n my-token-name`

Check the command documentation for more options. The output of this command will be as the following, where <TOKEN_CONTENT_STRING> will be a very long string that you need to store and don’t share it publicly.

`1`	`FlyV1 <TOKEN_CONTENT_STRING>`

Add the token to an environment var in the machine you will run the backup script:

`1`	`export FLY_SSH_TOKEN=<TOKEN_CONTENT_STRING>`

Data location

The volume is mounted in /data directory, defined in our application fly.toml file:

[[mounts]]
  source = 'app_data'
  destination = '/data'

Creating a backup script

Given that we have a token, we are now able to execute SSH commands remotely on our Fly Machine. In our scenario, I am compacting the whole content of /data/ directory (where all the data that I want to backup is located) generating a tarball, then I download it locally.

I could create a custom script and copy it to the remote machine if I want to perform more complex tasks, or you can extend/modify this script to perform other tasks (e.g. download the tarball and upload to a S3 bucket).

#!/bin/bash
# backup.sh

# Need to start container in fly.io if it was stopped by inactivity
curl -s -o /dev/null https://your-app.fly.dev/

filename="data_backup_$(date +%F).tar.gz"
fly ssh console -C 'tar cvz /data' -t $FLY_SSH_TOKEN > $filename

Run it periodically

Now you can add backup.sh to your crontab schedule, or even adapt the procedure described here to be executed in other environments, like defining a GitHub Action or another way to schedule jobs.

I know this is not the most complete way to implement a backup policy, but it is working for my current projects. In the future, as I improve my scripts, I will possibly update this post to make it more complete.