goyimnose/Medios-Macina
1
0
Fork 0
Code Issues Pull Requests Packages Releases 1 Wiki Activity

updating and refining plugin system refactor

Browse Source
This commit is contained in:
Nose
2026-04-28 22:20:54 -07:00
parent 8685fbb723
commit 323c24f4f4
33 changed files with 4287 additions and 3312 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ftp_plugin_tutorial.md
+33 -23
View File
@@ -2,11 +2,11 @@
This walkthrough adds a real bundled `ftp` plugin so users can:
- run `search-file -plugin ftp ...`
- run `search-file -plugin ftp -instance <name> ...`
- browse remote folders as result tables
- select file rows to `download-file`
- pipe selected file rows into `add-file`
- upload local files with `add-file -plugin ftp`
- upload local files with `add-file -plugin ftp -instance <name>`
The implementation lives in [plugins/ftp/__init__.py](plugins/ftp/__init__.py).
@@ -20,14 +20,14 @@ The FTP plugin demonstrates the main provider hooks that matter for a storage-st
- `selector()` turns folder rows into a follow-up table when the user runs `@N`.
- `download()` and `download_url()` fetch FTP files into `download-file` output paths.
- `resolve_pipe_result_download()` lets `@N | add-file -store ...` materialize a remote FTP file first.
- `upload()` lets `add-file -plugin ftp -path ...` push a local file to the configured FTP server.
- `upload()` lets `add-file -plugin ftp -instance <name> -path ...` push a local file to the configured FTP server.
## Example Config
Add an FTP provider block to your config:
Add one or more named FTP provider instances to your config:
```toml
[provider.ftp]
[provider.ftp.work]
host = "ftp.example.com"
port = 21
username = "demo"
@@ -37,11 +37,20 @@ tls = false
passive = true
timeout = 20
search_depth = 1
[provider.ftp.archive]
host = "archive.example.com"
port = 2121
username = "archive-bot"
password = "secret"
base_path = "/dropbox"
tls = true
```
Notes:
- `host` is the only required field for the plugin to validate.
- `work` and `archive` are instance names; use them with `-instance work` or `-instance archive`.
- `host` is the only required field for each instance to validate.
- `username` defaults to `anonymous` and `password` defaults to `anonymous@`.
- `base_path` is both the default search root and the upload target directory.
- `search_depth` controls how many folder levels `search-file -plugin ftp` scans by default.
@@ -51,25 +60,25 @@ Notes:
Basic listing from the configured base path:
```powershell
search-file -plugin ftp "*"
search-file -plugin ftp -instance work "*"
```
Search by filename fragment:
```powershell
search-file -plugin ftp "invoice"
search-file -plugin ftp -instance work "invoice"
```
Search a different subtree and recurse deeper:
```powershell
search-file -plugin ftp "path:/pub depth:2 invoice"
search-file -plugin ftp -instance work "path:/pub depth:2 invoice"
```
Filter to folders only:
```powershell
search-file -plugin ftp "path:/pub type:folder *"
search-file -plugin ftp -instance work "path:/pub type:folder *"
```
The plugin returns rows with explicit columns for name, type, directory, size, and modification time.
@@ -79,20 +88,20 @@ The plugin returns rows with explicit columns for name, type, directory, size, a
Folder rows are navigation rows. If the selected row is a directory, plain `@N` opens a new FTP table for that directory:
```powershell
search-file -plugin ftp "*"
search-file -plugin ftp -instance work "*"
@2
```
File rows carry an explicit row action:
```powershell
download-file -plugin ftp -url ftp://ftp.example.com/incoming/report.pdf
download-file -plugin ftp -instance work -url ftp://ftp.example.com/incoming/report.pdf
```
That means plain `@N` on a file row downloads it immediately:
```powershell
search-file -plugin ftp "report"
search-file -plugin ftp -instance work "report"
@1
```
@@ -101,14 +110,14 @@ search-file -plugin ftp "report"
If you want the downloaded file in a specific local directory:
```powershell
search-file -plugin ftp "report"
search-file -plugin ftp -instance work "report"
@1 | download-file -path C:\Downloads
```
If you want to ingest the selected FTP file into a configured store backend:
```powershell
search-file -plugin ftp "report"
search-file -plugin ftp -instance work "report"
@1 | add-file -store tutorial
```
@@ -117,23 +126,24 @@ Why this works:
- the file row advertises a `download-file` row action
- the pipeline auto-inserts that download before `add-file`
- the FTP plugin also implements `resolve_pipe_result_download()` so provider-owned FTP rows can be materialized for ingestion
- file rows also carry the chosen `instance`, so selection replay and `@N | add-file ...` keep the same FTP target
## Upload Flow
Uploading uses the same provider name, but through `add-file -plugin ftp`:
Uploading uses the same provider name, but through `add-file -plugin ftp -instance <name>`:
```powershell
add-file -plugin ftp -path C:\Media\report.pdf
add-file -plugin ftp -instance archive -path C:\Media\report.pdf
```
That sends the file to the configured FTP `base_path` and returns the FTP URL as the uploaded result.
That sends the file to the selected instance's FTP `base_path` and returns the FTP URL as the uploaded result.
## Why The Row Metadata Matters
The critical part of this plugin is the file-row metadata:
- file rows emit `_selection_args` as `['-url', '<ftp-url>']`
- file rows emit `_selection_action` as `['download-file', '-plugin', 'ftp', '-url', '<ftp-url>']`
- file rows emit `_selection_args` as `['-instance', '<name>', '-url', '<ftp-url>']`
- file rows emit `_selection_action` as `['download-file', '-plugin', 'ftp', '-instance', '<name>', '-url', '<ftp-url>']`
- folder rows do not emit a download action, so `selector()` can own drill-in behavior instead
That split is what keeps these two user experiences compatible:
@@ -155,10 +165,10 @@ The code is intentionally small and uses only Python stdlib pieces:
## Recommended Commands To Demo The Walkthrough
```powershell
search-file -plugin ftp "*"
search-file -plugin ftp "path:/incoming depth:2 *.pdf"
search-file -plugin ftp -instance work "*"
search-file -plugin ftp -instance work "path:/incoming depth:2 *.pdf"
@1
@1 | download-file -path C:\Downloads
@1 | add-file -store tutorial
add-file -plugin ftp -path C:\Media\report.pdf
add-file -plugin ftp -instance archive -path C:\Media\report.pdf
```
docs/scp_plugin_tutorial.md
+28 -18
View File
@@ -11,16 +11,16 @@ The implementation lives in [plugins/scp/__init__.py](plugins/scp/__init__.py).
The SCP plugin mirrors the FTP walkthrough, but on top of SSH:
- `search-file -plugin scp ...` lists remote files and folders over SFTP.
- `search-file -plugin scp -instance <name> ...` lists remote files and folders over SFTP.
- plain `@N` on a folder drills into that directory.
- plain `@N` on a file runs `download-file -plugin scp -url ...`.
- plain `@N` on a file runs `download-file -plugin scp -instance <name> -url ...`.
- `@N | add-file -store ...` downloads first, then ingests the local temp file.
- `add-file -plugin scp -path ...` uploads a local file to the configured remote path.
- `add-file -plugin scp -instance <name> -path ...` uploads a local file to the configured remote path.
## Example Config
```toml
[provider.scp]
[provider.scp.work]
host = "ssh.example.com"
port = 22
username = "deploy"
@@ -31,11 +31,20 @@ timeout = 20
search_depth = 1
allow_agent = true
look_for_keys = true
[provider.scp.archive]
host = "ssh-archive.example.com"
port = 2222
username = "archive"
key_path = "C:/Users/Admin/.ssh/archive_ed25519"
base_path = "/srv/archive"
timeout = 20
```
Notes:
- `host` and `username` are required for the plugin to validate.
- `work` and `archive` are instance names; use them with `-instance work` or `-instance archive`.
- `host` and `username` are required for each instance to validate.
- You can use password auth, key auth, or both.
- `base_path` is both the default search root and the default upload directory.
@@ -44,25 +53,25 @@ Notes:
List the configured base path:
```powershell
search-file -plugin scp "*"
search-file -plugin scp -instance work "*"
```
Search by filename:
```powershell
search-file -plugin scp "invoice"
search-file -plugin scp -instance work "invoice"
```
Search another subtree with deeper recursion:
```powershell
search-file -plugin scp "path:/srv/files/releases depth:2 *.zip"
search-file -plugin scp -instance work "path:/srv/files/releases depth:2 *.zip"
```
Show only folders:
```powershell
search-file -plugin scp "path:/srv/files type:folder *"
search-file -plugin scp -instance work "path:/srv/files type:folder *"
```
## Selection Flow
@@ -70,21 +79,21 @@ search-file -plugin scp "path:/srv/files type:folder *"
Folder rows are navigation rows:
```powershell
search-file -plugin scp "*"
search-file -plugin scp -instance work "*"
@2
```
File rows carry an explicit row action, so terminal selection downloads directly:
```powershell
search-file -plugin scp "report"
search-file -plugin scp -instance work "report"
@1
```
That expands to the equivalent of:
```powershell
download-file -plugin scp -url scp://ssh.example.com/srv/files/report.pdf
download-file -plugin scp -instance work -url scp://ssh.example.com/srv/files/report.pdf
```
## Download And Add-File Flow
@@ -92,14 +101,14 @@ download-file -plugin scp -url scp://ssh.example.com/srv/files/report.pdf
Download into a local folder:
```powershell
search-file -plugin scp "report"
search-file -plugin scp -instance work "report"
@1 | download-file -path C:\Downloads
```
Ingest a selected remote file into a configured store backend:
```powershell
search-file -plugin scp "report"
search-file -plugin scp -instance work "report"
@1 | add-file -store tutorial
```
@@ -108,13 +117,14 @@ Why this works:
- file rows advertise `_selection_action` for `download-file`
- `add-file` selection replay inserts that provider download stage before ingest
- the plugin also implements `resolve_pipe_result_download()` for provider-owned SCP rows
- file rows also carry the chosen `instance`, so replay stays bound to the same SSH target
## Upload Flow
Upload a local file to the configured remote `base_path`:
```powershell
add-file -plugin scp -path C:\Media\report.pdf
add-file -plugin scp -instance archive -path C:\Media\report.pdf
```
## Implementation Notes
@@ -127,10 +137,10 @@ The plugin uses SFTP for directory listing because SCP itself is a transfer prot
## Recommended Demo Commands
```powershell
search-file -plugin scp "*"
search-file -plugin scp "path:/srv/files depth:2 *.zip"
search-file -plugin scp -instance work "*"
search-file -plugin scp -instance work "path:/srv/files depth:2 *.zip"
@1
@1 | download-file -path C:\Downloads
@1 | add-file -store tutorial
add-file -plugin scp -path C:\Media\report.pdf
add-file -plugin scp -instance archive -path C:\Media\report.pdf
```