Medios-Macina/docs/zerotier.md

ZeroTier integration (store sharing)

This document describes how Medios-Macina integrates with ZeroTier to share storage backends between machines on a private virtual network.

Goals

• Allow you to expose stores (folder-based, remote storage server, Hydrus client) to other members of your ZeroTier network.
• Keep the CLI experience identical: remote stores appear as normal -store backends.
• Use secure authentication (API keys / per-store tokens) and limit exposure to private network.

Prerequisites

• Each machine must run zerotier-one and be a member of your ZeroTier network.
• The Medios-Macina instance on each machine should run the remote_storage_server.py or a Hydrus client instance you want to expose.
• The remote storage server requires Flask and Flask-CORS to run (install with: pip install flask flask-cors).

Auto-install behavior

• When a zerotier section is present in config.conf or a store=zerotier instance is configured, the CLI will attempt to auto-install the required packages (flask, flask-cors, and werkzeug) on startup unless you disable it with auto_install = false in the zerotier config block. This mirrors the behavior for other optional features (e.g., Soulseek).
• On your controller/management machine, authorize members via ZeroTier Central.

Configuration (conceptual)

You can configure networks and Zerotier-backed stores in your config.conf. Here are example snippets and recommendations.

Top-level ZeroTier networks (recommended)

Use a zerotier section to list networks your instance is willing to use/auto-join:

[zerotier]
# Example config (implementation treats this as a dict via the loader)
# networks:
#   home:
#     network_id: 8056c2e21c000001
#     api_key: my-zt-central-token  ; optional, only needed for automating member authorization
#     auto_join: true
#     prefer_hosts: ["192.168.86.42"]  ; optional peer IP inside the ZT network

Store config (ZeroTier store instances)

Add a store=zerotier block so the Store registry can create a ZeroTier store instance:

[store=zerotier]
my-remote = { "NAME": "my-remote", "NETWORK_ID": "8056c2e21c000001", "SERVICE": "remote", "PORT": 999, "API_KEY": "myremotekey" }
hydrus-remote = { "NAME": "hydrus-remote", "NETWORK_ID": "8056c2e21c000001", "SERVICE": "hydrus", "PORT": 45869, "API_KEY": "hydrus-access-key" }

• SERVICE can be remote (our remote_storage_server), or hydrus (Hydrus API).
• HOST is optional; if present, discovery is skipped and the host:port is used.
• API_KEY will be sent as X-API-Key (and Hydrus access keys, when relevant).

Operation & discovery

• The local ZeroTier store wrapper will attempt to discover peers on the configured ZeroTier network by inspecting assigned addresses on this node and probing common service endpoints (e.g., /health, /api_version).
• For hydrus service types we look for Hydrus-style /api_version responses.
• For remote service types we look for our remote_storage_server /health endpoint.

Security notes

• Your ZeroTier network provides a private IP layer, but the exposed services should still require authentication (API keys) and enforce scope (read/write).
• If you plan to expose stores to other users, consider per-store API keys with roles (read-only, write, admin) and monitor/audit access.

Next steps / prototyping

• The first prototype in this repo adds API/zerotier.py (discovery + join helpers) and Store/ZeroTier.py (a store wrapper that proxies to hydrus or remote endpoints).
• Upload support (server-side POST /files/upload) is now implemented allowing authenticated multipart uploads; the ZeroTier store wrapper supports add_file() and the add-file cmdlet can be used with a configured ZeroTier store for end-to-end uploads.

Example: upload via the helper script (discovers a remote on the network and uploads the file):

python .\scripts\zerotier_setup.py --upload 8056c2e21c000001 --file "C:\path\to\file.mp4" --api-key myremotekey --tag tag1 --tag tag2

Or using curl directly against a discovered ZeroTier peer's IP:

curl -X POST -H "X-API-Key: myremotekey" -F "file=@/path/to/file.mp4" -F "tag=tag1" http://<zerotier-ip>:999/files/upload

If you'd like I can:

• Add an example scripts/zt-join.py helper that uses the API wrapper to join a network;
• Add a presigned-upload + multipart upload flow to scripts/remote_storage_server.py so ZeroTier stores can support add-file uploads directly.

Tell me which of the above you want next (upload support, auto-join helper, or presigned flow) and I'll proceed.