f
This commit is contained in:
@@ -1,91 +0,0 @@
|
||||
# ZeroTier integration (store sharing)
|
||||
|
||||
This document describes how Medios-Macina integrates with ZeroTier to share
|
||||
storage backends between machines on a private virtual network.
|
||||
|
||||
Goals
|
||||
- Allow you to expose stores (folder-based, remote storage server, Hydrus client)
|
||||
to other members of your ZeroTier network.
|
||||
- Keep the CLI experience identical: remote stores appear as normal `-store` backends.
|
||||
- Use secure authentication (API keys / per-store tokens) and limit exposure to private network.
|
||||
|
||||
Prerequisites
|
||||
- Each machine must run `zerotier-one` and be a member of your ZeroTier network.
|
||||
- The Medios-Macina instance on each machine should run the `remote_storage_server.py`
|
||||
or a Hydrus client instance you want to expose.
|
||||
- The remote storage server requires Flask and Flask-CORS to run (install with: `pip install flask flask-cors`).
|
||||
|
||||
Auto-install behavior
|
||||
- When a `zerotier` section is present in `config.conf` **or** a `store=zerotier` instance is configured, the CLI will attempt to auto-install the required packages (`flask`, `flask-cors`, and `werkzeug`) on startup unless you disable it with `auto_install = false` in the `zerotier` config block. This mirrors the behavior for other optional features (e.g., Soulseek).
|
||||
- On your controller/management machine, authorize members via ZeroTier Central.
|
||||
|
||||
Configuration (conceptual)
|
||||
|
||||
You can configure networks and Zerotier-backed stores in your `config.conf`. Here
|
||||
are example snippets and recommendations.
|
||||
|
||||
## Top-level ZeroTier networks (recommended)
|
||||
|
||||
Use a `zerotier` section to list networks your instance is willing to use/auto-join:
|
||||
|
||||
```ini
|
||||
[zerotier]
|
||||
# Example config (implementation treats this as a dict via the loader)
|
||||
# networks:
|
||||
# home:
|
||||
# network_id: 8056c2e21c000001
|
||||
# api_key: my-zt-central-token ; optional, only needed for automating member authorization
|
||||
# auto_join: true
|
||||
# prefer_hosts: ["192.168.86.42"] ; optional peer IP inside the ZT network
|
||||
```
|
||||
|
||||
## Store config (ZeroTier store instances)
|
||||
|
||||
Add a `store=zerotier` block so the Store registry can create a ZeroTier store instance:
|
||||
|
||||
```ini
|
||||
[store=zerotier]
|
||||
my-remote = { "NAME": "my-remote", "NETWORK_ID": "8056c2e21c000001", "SERVICE": "remote", "PORT": 999, "API_KEY": "myremotekey" }
|
||||
hydrus-remote = { "NAME": "hydrus-remote", "NETWORK_ID": "8056c2e21c000001", "SERVICE": "hydrus", "PORT": 45869, "API_KEY": "hydrus-access-key" }
|
||||
```
|
||||
|
||||
- `SERVICE` can be `remote` (our `remote_storage_server`), or `hydrus` (Hydrus API).
|
||||
- `HOST` is optional; if present, discovery is skipped and the host:port is used.
|
||||
- `API_KEY` will be sent as `X-API-Key` (and Hydrus access keys, when relevant).
|
||||
|
||||
Operation & discovery
|
||||
- The local ZeroTier store wrapper will attempt to discover peers on the configured
|
||||
ZeroTier network by inspecting assigned addresses on this node and probing common
|
||||
service endpoints (e.g., `/health`, `/api_version`).
|
||||
- For `hydrus` service types we look for Hydrus-style `/api_version` responses.
|
||||
- For `remote` service types we look for our `remote_storage_server` `/health` endpoint.
|
||||
|
||||
Security notes
|
||||
- Your ZeroTier network provides a private IP layer, but the exposed services
|
||||
should still require authentication (API keys) and enforce scope (read/write).
|
||||
- If you plan to expose stores to other users, consider per-store API keys with
|
||||
roles (read-only, write, admin) and monitor/audit access.
|
||||
|
||||
Next steps / prototyping
|
||||
- The first prototype in this repo adds `API/zerotier.py` (discovery + join helpers)
|
||||
and `Store/ZeroTier.py` (a store wrapper that proxies to `hydrus` or `remote` endpoints).
|
||||
- Upload support (server-side `POST /files/upload`) is now implemented allowing authenticated multipart uploads; the ZeroTier store wrapper supports `add_file()` and the `add-file` cmdlet can be used with a configured ZeroTier store for end-to-end uploads.
|
||||
|
||||
Example: upload via the helper script (discovers a remote on the network and uploads the file):
|
||||
|
||||
```powershell
|
||||
python .\scripts\zerotier_setup.py --upload 8056c2e21c000001 --file "C:\path\to\file.mp4" --api-key myremotekey --tag tag1 --tag tag2
|
||||
```
|
||||
|
||||
Or using curl directly against a discovered ZeroTier peer's IP:
|
||||
|
||||
```powershell
|
||||
curl -X POST -H "X-API-Key: myremotekey" -F "file=@/path/to/file.mp4" -F "tag=tag1" http://<zerotier-ip>:999/files/upload
|
||||
```
|
||||
|
||||
If you'd like I can:
|
||||
- Add an example `scripts/zt-join.py` helper that uses the API wrapper to join a network;
|
||||
- Add a presigned-upload + multipart upload flow to `scripts/remote_storage_server.py` so
|
||||
ZeroTier stores can support `add-file` uploads directly.
|
||||
|
||||
Tell me which of the above you want next (upload support, auto-join helper, or presigned flow) and I'll proceed.
|
||||
Reference in New Issue
Block a user