Medios-Macina/docs/zerotier.md

# ZeroTier integration (store sharing)

This document describes how Medios-Macina integrates with ZeroTier to share
storage backends between machines on a private virtual network.

Goals
- Allow you to expose stores (folder-based, remote storage server, Hydrus client)
  to other members of your ZeroTier network.
- Keep the CLI experience identical: remote stores appear as normal `-store` backends.
- Use secure authentication (API keys / per-store tokens) and limit exposure to private network.

Prerequisites
- Each machine must run `zerotier-one` and be a member of your ZeroTier network.
- The Medios-Macina instance on each machine should run the `remote_storage_server.py`
  or a Hydrus client instance you want to expose.
- The remote storage server requires Flask and Flask-CORS to run (install with: `pip install flask flask-cors`).

Auto-install behavior
- When a `zerotier` section is present in `config.conf` **or** a `store=zerotier` instance is configured, the CLI will attempt to auto-install the required packages (`flask`, `flask-cors`, and `werkzeug`) on startup unless you disable it with `auto_install = false` in the `zerotier` config block. This mirrors the behavior for other optional features (e.g., Soulseek).
- On your controller/management machine, authorize members via ZeroTier Central.

Configuration (conceptual)

You can configure networks and Zerotier-backed stores in your `config.conf`. Here
are example snippets and recommendations.

## Top-level ZeroTier networks (recommended)

Use a `zerotier` section to list networks your instance is willing to use/auto-join:

```ini
[zerotier]
# Example config (implementation treats this as a dict via the loader)
# networks:
#   home:
#     network_id: 8056c2e21c000001
#     api_key: my-zt-central-token  ; optional, only needed for automating member authorization
#     auto_join: true
#     prefer_hosts: ["192.168.86.42"]  ; optional peer IP inside the ZT network
```

## Store config (ZeroTier store instances)

Add a `store=zerotier` block so the Store registry can create a ZeroTier store instance:

```ini
[store=zerotier]
my-remote = { "NAME": "my-remote", "NETWORK_ID": "8056c2e21c000001", "SERVICE": "remote", "PORT": 999, "API_KEY": "myremotekey" }
hydrus-remote = { "NAME": "hydrus-remote", "NETWORK_ID": "8056c2e21c000001", "SERVICE": "hydrus", "PORT": 45869, "API_KEY": "hydrus-access-key" }
```

- `SERVICE` can be `remote` (our `remote_storage_server`), or `hydrus` (Hydrus API).
- `HOST` is optional; if present, discovery is skipped and the host:port is used.
- `API_KEY` will be sent as `X-API-Key` (and Hydrus access keys, when relevant).

Operation & discovery
- The local ZeroTier store wrapper will attempt to discover peers on the configured
  ZeroTier network by inspecting assigned addresses on this node and probing common
  service endpoints (e.g., `/health`, `/api_version`).
- For `hydrus` service types we look for Hydrus-style `/api_version` responses.
- For `remote` service types we look for our `remote_storage_server` `/health` endpoint.

Security notes
- Your ZeroTier network provides a private IP layer, but the exposed services
  should still require authentication (API keys) and enforce scope (read/write).
- If you plan to expose stores to other users, consider per-store API keys with
  roles (read-only, write, admin) and monitor/audit access.

Next steps / prototyping
- The first prototype in this repo adds `API/zerotier.py` (discovery + join helpers)
  and `Store/ZeroTier.py` (a store wrapper that proxies to `hydrus` or `remote` endpoints).
- Upload support (server-side `POST /files/upload`) is now implemented allowing authenticated multipart uploads; the ZeroTier store wrapper supports `add_file()` and the `add-file` cmdlet can be used with a configured ZeroTier store for end-to-end uploads.

Example: upload via the helper script (discovers a remote on the network and uploads the file):

```powershell
python .\scripts\zerotier_setup.py --upload 8056c2e21c000001 --file "C:\path\to\file.mp4" --api-key myremotekey --tag tag1 --tag tag2
```

Or using curl directly against a discovered ZeroTier peer's IP:

```powershell
curl -X POST -H "X-API-Key: myremotekey" -F "file=@/path/to/file.mp4" -F "tag=tag1" http://<zerotier-ip>:999/files/upload
```

If you'd like I can:
- Add an example `scripts/zt-join.py` helper that uses the API wrapper to join a network;
- Add a presigned-upload + multipart upload flow to `scripts/remote_storage_server.py` so
  ZeroTier stores can support `add-file` uploads directly.

Tell me which of the above you want next (upload support, auto-join helper, or presigned flow) and I'll proceed.