updated plugin refactor and added FTP and SCP plugins , also hydrusnetwork plugin migration

docs/ftp_plugin_tutorial.md

@@ -0,0 +1,164 @@
+# FTP Plugin Walkthrough
+
+This walkthrough adds a real bundled `ftp` plugin so users can:
+
+- run `search-file -plugin ftp ...`
+- browse remote folders as result tables
+- select file rows to `download-file`
+- pipe selected file rows into `add-file`
+- upload local files with `add-file -plugin ftp`
+
+The implementation lives in [plugins/ftp/__init__.py](plugins/ftp/__init__.py).
+
+## What The Plugin Does
+
+The FTP plugin demonstrates the main provider hooks that matter for a storage-style integration:
+
+- `config_schema()` exposes host, credentials, base path, TLS, and search depth.
+- `extract_query_arguments()` supports inline query fields like `path:` and `depth:`.
+- `search()` walks an FTP directory tree and returns `SearchResult` rows.
+- `selector()` turns folder rows into a follow-up table when the user runs `@N`.
+- `download()` and `download_url()` fetch FTP files into `download-file` output paths.
+- `resolve_pipe_result_download()` lets `@N | add-file -store ...` materialize a remote FTP file first.
+- `upload()` lets `add-file -plugin ftp -path ...` push a local file to the configured FTP server.
+
+## Example Config
+
+Add an FTP provider block to your config:
+
+```toml
+[provider.ftp]
+host = "ftp.example.com"
+port = 21
+username = "demo"
+password = "secret"
+base_path = "/incoming"
+tls = false
+passive = true
+timeout = 20
+search_depth = 1
+```
+
+Notes:
+
+- `host` is the only required field for the plugin to validate.
+- `username` defaults to `anonymous` and `password` defaults to `anonymous@`.
+- `base_path` is both the default search root and the upload target directory.
+- `search_depth` controls how many folder levels `search-file -plugin ftp` scans by default.
+
+## Search Flow
+
+Basic listing from the configured base path:
+
+```powershell
+search-file -plugin ftp "*"
+```
+
+Search by filename fragment:
+
+```powershell
+search-file -plugin ftp "invoice"
+```
+
+Search a different subtree and recurse deeper:
+
+```powershell
+search-file -plugin ftp "path:/pub depth:2 invoice"
+```
+
+Filter to folders only:
+
+```powershell
+search-file -plugin ftp "path:/pub type:folder *"
+```
+
+The plugin returns rows with explicit columns for name, type, directory, size, and modification time.
+
+## Selection Flow
+
+Folder rows are navigation rows. If the selected row is a directory, plain `@N` opens a new FTP table for that directory:
+
+```powershell
+search-file -plugin ftp "*"
+@2
+```
+
+File rows carry an explicit row action:
+
+```powershell
+download-file -plugin ftp -url ftp://ftp.example.com/incoming/report.pdf
+```
+
+That means plain `@N` on a file row downloads it immediately:
+
+```powershell
+search-file -plugin ftp "report"
+@1
+```
+
+## Download And Add-File Flow
+
+If you want the downloaded file in a specific local directory:
+
+```powershell
+search-file -plugin ftp "report"
+@1 | download-file -path C:\Downloads
+```
+
+If you want to ingest the selected FTP file into a configured store backend:
+
+```powershell
+search-file -plugin ftp "report"
+@1 | add-file -store tutorial
+```
+
+Why this works:
+
+- the file row advertises a `download-file` row action
+- the pipeline auto-inserts that download before `add-file`
+- the FTP plugin also implements `resolve_pipe_result_download()` so provider-owned FTP rows can be materialized for ingestion
+
+## Upload Flow
+
+Uploading uses the same provider name, but through `add-file -plugin ftp`:
+
+```powershell
+add-file -plugin ftp -path C:\Media\report.pdf
+```
+
+That sends the file to the configured FTP `base_path` and returns the FTP URL as the uploaded result.
+
+## Why The Row Metadata Matters
+
+The critical part of this plugin is the file-row metadata:
+
+- file rows emit `_selection_args` as `['-url', '<ftp-url>']`
+- file rows emit `_selection_action` as `['download-file', '-plugin', 'ftp', '-url', '<ftp-url>']`
+- folder rows do not emit a download action, so `selector()` can own drill-in behavior instead
+
+That split is what keeps these two user experiences compatible:
+
+- `@N` on a folder opens a new table
+- `@N` on a file downloads the file
+- `@N | add-file -store ...` first downloads, then ingests
+
+## Implementation Notes
+
+The plugin prefers `MLSD` for directory listings and falls back to `NLST` plus directory probes when the server does not support machine-readable listings.
+
+The code is intentionally small and uses only Python stdlib pieces:
+
+- `ftplib` for FTP and FTPS
+- `fnmatch` for wildcard-style search tokens
+- `tempfile` for `add-file` handoff downloads
+
+## Recommended Commands To Demo The Walkthrough
+
+```powershell
+search-file -plugin ftp "*"
+search-file -plugin ftp "path:/incoming depth:2 *.pdf"
+@1
+@1 | download-file -path C:\Downloads
+@1 | add-file -store tutorial
+add-file -plugin ftp -path C:\Media\report.pdf
+```