Skip to content

ADR 001: rclone, not aws s3 cp, for R2 multipart uploads on macOS

Status: accepted Date: 2026-05-04 Supersedes: —

Context

When wiring R2 backup into ~/garmin-warehouse/scripts/daily_sync.sh, the natural choice was AWS CLI (S3-compatible) with --endpoint-url pointing at the R2 endpoint. That works for files <100MB.

For the GarminDB activities database (~1.1GB), the AWS CLI consistently failed mid-upload with:

SSL: SSLV3_ALERT_BAD_RECORD_MAC

at random partNumber= values (32, 7, 45 — different each run). Tried:

  • Tuning chunk size (--cli-write-timeout, s3.multipart_chunksize)
  • Forcing single-part with --cli-binary-format raw-in-base64-out and size hacks
  • Different AWS CLI versions
  • Direct boto3 client with custom Session

All failed at the SSL layer. Root cause: macOS LibreSSL + Python's ssl module + R2's TLS termination has a known incompatibility with multipart uploads.

Decision

Use rclone for R2, not AWS CLI. rclone has its own R2-aware backend with proper SSL handling.

Required config in ~/.config/rclone/rclone.conf:

[r2]
type = s3
provider = Cloudflare
access_key_id = ${R2_ACCESS_KEY_ID}
secret_access_key = ${R2_SECRET_ACCESS_KEY}
endpoint = ${R2_ENDPOINT}
no_check_bucket = true   # CRITICAL: token can't CreateBucket

The no_check_bucket = true flag is essential — without it, every operation tries to verify bucket existence via a HEAD on the bucket root, which 401s for our bucket-scoped token.

daily_sync.sh uses aws s3 cp for files <100MB (kb.duckdb, state files) where it works fine, and rclone for the GarminDB upload. This is fine for now; eventual goal is rclone for everything.

Consequences

Good: - 1.1GB upload completes in ~2m42s, reliably - Same R2 token works for both tools - rclone is in homebrew (brew install rclone)

Tradeoffs: - Two tools to maintain config for - rclone syntax is different from aws s3 (rclone copy src dest rather than aws s3 cp src dest) - New users need both tools installed

Alternatives considered

  1. Python boto3 directly — same SSL issue, it's the underlying layer
  2. curl with R2's S3 API directly — would work, but multipart handshake is annoying to write by hand
  3. Different upload tool (aws-vault, s5cmd, etc.) — rclone won on existing familiarity + active maintenance
  4. Disable multipart entirely (force single-part) — R2 doesn't accept single-part >5GB anyway, and 1.1GB single-part is slow
  5. Wait for the SSL issue to be fixed upstream — not worth blocking on

Notes

This is a macOS-specific issue. Linux + AWS CLI + R2 multipart works fine. If daily_sync.sh ever runs on Linux (e.g. moved to a NAS or cloud VM), AWS CLI could replace rclone there.

References