huge refactor of the entire codebase, with the goal of improving maintainability, readability, and extensibility. This commit includes changes to almost every file in the project, including:
This commit is contained in:
+12
-12
@@ -1,13 +1,13 @@
|
||||
# Provider authoring: ResultTable & provider adapters ✅
|
||||
# Plugin authoring: ResultTable & plugin adapters
|
||||
|
||||
This short guide explains how to write providers that integrate with the *strict* ResultTable API: adapters must yield `ResultModel` instances and providers register via `SYS.result_table_adapters.register_provider` with a column specification and a `selection_fn`.
|
||||
This short guide explains how to write plugins that integrate with the *strict* ResultTable API: adapters must yield `ResultModel` instances and plugins register via `SYS.result_table_adapters.register_plugin` with a column specification and a `selection_fn`.
|
||||
|
||||
---
|
||||
|
||||
## Quick summary
|
||||
|
||||
- Providers register a *provider adapter* (callable that yields `ResultModel`).
|
||||
- Providers must also provide `columns` (static list or factory) and a `selection_fn` that returns CLI args for a selected row.
|
||||
- Plugins register a *plugin adapter* (callable that yields `ResultModel`).
|
||||
- Plugins must also provide `columns` (static list or factory) and a `selection_fn` that returns CLI args for a selected row.
|
||||
- For simple HTML table/list scraping, prefer `TableProviderMixin` from `SYS.provider_helpers` to fetch and extract rows using `SYS.html_table.extract_records`.
|
||||
|
||||
## Runtime dependency policy
|
||||
@@ -21,11 +21,11 @@ This short guide explains how to write providers that integrate with the *strict
|
||||
## Minimal provider template (copy/paste)
|
||||
|
||||
```py
|
||||
# Provider/my_provider.py
|
||||
# plugins/my_plugin.py
|
||||
from typing import Any, Dict, Iterable, List
|
||||
|
||||
from SYS.result_table_api import ResultModel, ColumnSpec, title_column, metadata_column
|
||||
from SYS.result_table_adapters import register_provider
|
||||
from SYS.result_table_adapters import register_plugin
|
||||
|
||||
# Example adapter: convert provider-specific items into ResultModel instances
|
||||
SAMPLE_ITEMS = [
|
||||
@@ -59,8 +59,8 @@ def selection_fn(row: ResultModel) -> List[str]:
|
||||
return ["-path", row.path]
|
||||
return ["-title", row.title or ""]
|
||||
|
||||
# Register provider (done at import time)
|
||||
register_provider("myprovider", adapter, columns=columns_factory, selection_fn=selection_fn)
|
||||
# Register plugin (done at import time)
|
||||
register_plugin("myprovider", adapter, columns=columns_factory, selection_fn=selection_fn)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -84,7 +84,7 @@ class MyTableProvider(TableProviderMixin, Provider):
|
||||
return self.search_table_from_url(url, limit=limit)
|
||||
```
|
||||
|
||||
`TableProviderMixin.search_table_from_url` returns `ProviderCore.base.SearchResult` entries. If you want to integrate this provider with the strict `ResultTable` registry, add a small adapter that converts `SearchResult` -> `ResultModel` and register it using `register_provider` (see `Provider/vimm.py` for a real example).
|
||||
`TableProviderMixin.search_table_from_url` returns `ProviderCore.base.SearchResult` entries. If you want to integrate this plugin with the strict `ResultTable` registry, add a small adapter that converts `SearchResult` -> `ResultModel` and register it using `register_plugin` (see `Provider/vimm.py` for a real example).
|
||||
|
||||
---
|
||||
|
||||
@@ -93,7 +93,7 @@ class MyTableProvider(TableProviderMixin, Provider):
|
||||
- `columns` may be a static `List[ColumnSpec]` or a factory `def cols(rows: List[ResultModel]) -> List[ColumnSpec]` that inspects sample rows.
|
||||
- `selection_fn` must accept a `ResultModel` and return a `List[str]` representing CLI args (e.g., `['-path', row.path]`). These args are used by `select` and `@N` expansion.
|
||||
|
||||
**Tip:** for providers that produce downloadable file rows prefer returning explicit URL args (e.g., `['-url', row.path]`) so the selected URL is clearly identified by downstream downloaders and to avoid ambiguous parsing when provider hints (like `-provider`) are present.
|
||||
**Tip:** for plugins that produce downloadable file rows prefer returning explicit URL args (e.g., `['-url', row.path]`) so the selected URL is clearly identified by downstream downloaders and to avoid ambiguous parsing when plugin hints (like `-plugin`) are present.
|
||||
- Ensure your `ResultModel.source` is set (either in the model or rely on the provider name set by `serialize_row`).
|
||||
|
||||
---
|
||||
@@ -107,7 +107,7 @@ class MyTableProvider(TableProviderMixin, Provider):
|
||||
## Testing & examples
|
||||
|
||||
- Write `tests/test_provider_<name>.py` that imports your provider and verifies `provider.build_table(...)` produces a `ResultTable` (has `.rows` and `.columns`) and that `serialize_rows()` yields dicts with `_selection_args`, `_selection_action` when applicable, and `source`.
|
||||
- When you need to guarantee a specific CLI stage sequence (e.g., `download-file -url <path> -provider <name>`), call `table.set_row_selection_action(row_index, tokens)` so the serialized payload emits `_selection_action` and the CLI can run the row exactly as intended.
|
||||
- When you need to guarantee a specific CLI stage sequence (e.g., `download-file -url <path> -plugin <name>`), call `table.set_row_selection_action(row_index, tokens)` so the serialized payload emits `_selection_action` and the CLI can run the row exactly as intended.
|
||||
- For table providers you can test `search_table_from_url` using a local HTML fixture or by mocking `HTTPClient` to return a small sample page.
|
||||
- If you rely on pandas, add a test that monkeypatches `sys.modules['pandas']` to a simple shim to validate the pandas path.
|
||||
|
||||
@@ -119,7 +119,7 @@ from Provider import example_provider
|
||||
|
||||
|
||||
def test_example_provider_registration():
|
||||
provider = get_provider("example")
|
||||
plugin = get_plugin("example")
|
||||
rows = list(provider.adapter(example_provider.SAMPLE_ITEMS))
|
||||
assert rows and rows[0].title
|
||||
cols = provider.get_columns(rows)
|
||||
|
||||
+20
-21
@@ -1,23 +1,23 @@
|
||||
# Provider Development Guide
|
||||
# Plugin Development Guide
|
||||
|
||||
## 🎯 Purpose
|
||||
This guide describes how to write, test, and register a provider so the application can discover and use it as a pluggable component.
|
||||
This guide describes how to write, test, and register a plugin so the application can discover and use it as a pluggable component.
|
||||
|
||||
> Keep provider code small, focused, and well-tested. Use existing providers as examples.
|
||||
> Keep plugin code small, focused, and well-tested. Built-in plugins live in `Provider/` and external drop-in plugins live under `plugins/`.
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Anatomy of a Provider
|
||||
A provider is a Python class that extends `ProviderCore.base.Provider` and implements a few key methods and attributes.
|
||||
## 🔧 Anatomy of a Plugin
|
||||
A plugin is a Python class that extends `ProviderCore.base.Provider` and implements a few key methods and attributes.
|
||||
|
||||
Minimum expectations:
|
||||
- `class MyProvider(Provider):` — subclass the base provider
|
||||
- `class MyPlugin(Provider):` — subclass the base plugin class
|
||||
- `URL` / `URL_DOMAINS` or `url_patterns()` — to let the registry route URLs
|
||||
- `validate(self) -> bool` — return True when provider is configured and usable
|
||||
- `validate(self) -> bool` — return True when the plugin is configured and usable
|
||||
- `search(self, query, limit=50, filters=None, **kwargs)` — return a list of `SearchResult`
|
||||
|
||||
Optional but common:
|
||||
- `download(self, result: SearchResult, output_dir: Path) -> Optional[Path]` — download a provider result
|
||||
- `download(self, result: SearchResult, output_dir: Path) -> Optional[Path]` — download a plugin result
|
||||
- `selector(self, selected_items, *, ctx, stage_is_last=True, **kwargs) -> bool` — handle `@N` selections
|
||||
- `download_url(self, url, output_dir, progress_cb=None)` — direct URL-handling helper
|
||||
|
||||
@@ -71,8 +71,8 @@ class HelloProvider(Provider):
|
||||
---
|
||||
|
||||
## ⬇️ Implementing download() and download_url()
|
||||
- Prefer provider `download(self, result, output_dir)` for piped provider items.
|
||||
- For provider-provided URLs, implement `download_url` to allow `download-file` to route downloads through providers.
|
||||
- Prefer plugin `download(self, result, output_dir)` for piped plugin items.
|
||||
- For plugin-provided URLs, implement `download_url` to allow `download-file` to route downloads through plugins.
|
||||
- Use the repo `_download_direct_file` helper for HTTP downloads when possible.
|
||||
|
||||
Example download():
|
||||
@@ -90,12 +90,12 @@ def download(self, result: SearchResult, output_dir: Path) -> Optional[Path]:
|
||||
---
|
||||
|
||||
## 🧭 URL routing
|
||||
Providers can declare:
|
||||
Plugins can declare:
|
||||
- `URL = ("magnet:",)` or similar prefix list
|
||||
- `URL_DOMAINS = ("example.com",)` to match hosts
|
||||
- Or override `@classmethod def url_patterns(cls):` to combine static and dynamic patterns
|
||||
|
||||
The registry uses these to match `download-file <url>` or to pick which provider should handle the URL.
|
||||
The registry uses these to match `download-file <url>` or to pick which plugin should handle the URL.
|
||||
|
||||
---
|
||||
|
||||
@@ -106,8 +106,8 @@ The registry uses these to match `download-file <url>` or to pick which provider
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing providers
|
||||
- Keep tests small and local. Create `tests/test_provider_<name>.py`.
|
||||
## 🧪 Testing plugins
|
||||
- Keep tests small and local. Create `tests/test_provider_<name>.py` or another tracked test target.
|
||||
- Test `search()` with mock HTTP responses (use `requests-mock` or similar).
|
||||
- Test `download()` using a temp directory and a small file server or by mocking `_download_direct_file`.
|
||||
- Test `selector()` by constructing a fake result and `ctx` object.
|
||||
@@ -125,10 +125,9 @@ pytest -q
|
||||
---
|
||||
|
||||
## 📦 Registration & packaging
|
||||
- Add your provider module under `Provider/` and ensure it is imported by module package initialization. Common approach:
|
||||
- Place file `Provider/myprovider.py`
|
||||
- Ensure `Provider/__init__.py` imports the module (or the registry auto-discovers by package import)
|
||||
- If the project has a central provider registry, add lookup helpers there (e.g., `ProviderCore/registry.py`). Usually providers register themselves at import time.
|
||||
- Built-in plugins live under `Provider/` and are auto-discovered from that package.
|
||||
- External user plugins can be dropped into `plugins/` or any directory listed in `MM_PLUGIN_PATH` / `MEDEIA_PLUGIN_PATH`.
|
||||
- Plugin authors should import from `ProviderCore.*`.
|
||||
|
||||
---
|
||||
|
||||
@@ -147,19 +146,19 @@ pytest -q
|
||||
- [ ] Provide `URL` / `URL_DOMAINS` or `url_patterns()` for routing
|
||||
- [ ] Add `download()` or `download_url()` for piped/passed URL downloads
|
||||
- [ ] Add tests under `tests/`
|
||||
- [ ] Add module to `Provider/` package and ensure import/registration
|
||||
- [ ] Add the plugin module to `Provider/` for built-ins, or drop it into `plugins/` for plug-and-play user installs
|
||||
|
||||
---
|
||||
|
||||
## 🔗 Further reading
|
||||
- See existing providers in `Provider/` for patterns and edge cases.
|
||||
- See existing built-in plugins in `Provider/` for patterns and edge cases.
|
||||
- Check `API/` helpers for HTTP and debrid clients.
|
||||
|
||||
|
||||
---
|
||||
|
||||
If you'd like, I can:
|
||||
- Add an example provider file under `Provider/` as a template (see `Provider/hello_provider.py`), and
|
||||
- Add an example plugin file under `Provider/` as a template (see `Provider/hello_provider.py`), and
|
||||
- Create unit tests for it (see `tests/test_provider_hello.py`).
|
||||
|
||||
I have added a minimal example provider and tests in this repository; use them as a starting point for new providers.
|
||||
|
||||
@@ -40,7 +40,7 @@ from SYS.result_table import ResultTable
|
||||
table = ResultTable("Provider: X result").set_preserve_order(True)
|
||||
table.set_table("provider_name")
|
||||
table.set_table_metadata({"provider":"provider_name","view":"folders"})
|
||||
table.set_source_command("search-file", ["-provider","provider_name","query"])
|
||||
table.set_source_command("search-file", ["-plugin","provider_name","query"])
|
||||
|
||||
for r in results:
|
||||
table.add_result(r) # r can be a SearchResult, dict, or PipeObject
|
||||
@@ -82,13 +82,13 @@ Example commands:
|
||||
|
||||
```
|
||||
# List magnets in your account
|
||||
search-file -provider alldebrid "*"
|
||||
search-file -plugin alldebrid "*"
|
||||
|
||||
# Open magnet id 123 and list its files
|
||||
search-file -provider alldebrid -open 123 "*"
|
||||
search-file -plugin alldebrid -open 123 "*"
|
||||
|
||||
# Or expand via @ selection (selector handles drilling):
|
||||
search-file -provider alldebrid "*"
|
||||
search-file -plugin alldebrid "*"
|
||||
@3 # selector will open the magnet referenced by row #3 and show the file table
|
||||
```
|
||||
|
||||
@@ -147,7 +147,7 @@ Selection & download flows
|
||||
|
||||
```
|
||||
# Expand magnet and add first file to local directory
|
||||
search-file -provider alldebrid "*"
|
||||
search-file -plugin alldebrid "*"
|
||||
@3 # view files
|
||||
@1 | add-file -path C:\mydir
|
||||
```
|
||||
@@ -167,7 +167,7 @@ Example usage:
|
||||
|
||||
```
|
||||
# Search for an artist
|
||||
search-file -provider bandcamp "artist:radiohead"
|
||||
search-file -plugin bandcamp "artist:radiohead"
|
||||
|
||||
# Select an artist row to expand into releases
|
||||
@1
|
||||
|
||||
@@ -1,22 +1,22 @@
|
||||
Selector & provider-table usage
|
||||
Selector & plugin-table usage
|
||||
|
||||
This project provides a small provider/table/selector flow that allows providers
|
||||
This project provides a small plugin/table/selector flow that allows plugins
|
||||
and cmdlets to interact via a simple, pipable API.
|
||||
|
||||
Key ideas
|
||||
- `provider-table` renders a provider result set and *emits* pipeline-friendly dicts for each row. Each emitted item includes `_selection_args`, a list of args the provider suggests for selecting that row (e.g., `['-path', '/tmp/file']`).
|
||||
- `plugin-table` renders a plugin result set and *emits* pipeline-friendly dicts for each row. Each emitted item includes `_selection_args`, a list of args the plugin suggests for selecting that row (e.g., `['-path', '/tmp/file']`).
|
||||
- Use the `@N` syntax to select an item from a table and chain it to the next cmdlet.
|
||||
|
||||
Example:
|
||||
|
||||
provider-table -provider example -sample | @1 | add-file -store default
|
||||
plugin-table -plugin example -sample | @1 | add-file -store default
|
||||
|
||||
What providers must implement
|
||||
What plugins must implement
|
||||
- An adapter that yields `ResultModel` objects (breaking API).
|
||||
- Optionally supply a `columns` factory and `selection_fn` (see `Provider/example_provider.py`).
|
||||
|
||||
Implementation notes
|
||||
- `provider-table` emits dicts like `{ 'title': ..., 'path': ..., 'metadata': ..., '_selection_args': [...] }`.
|
||||
- Selection syntax (`@1`) will prefer `_selection_args` if present; otherwise it will fall back to provider selection logic or sensible defaults (`-path` or `-title`).
|
||||
- `plugin-table` emits dicts like `{ 'title': ..., 'path': ..., 'metadata': ..., '_selection_args': [...] }`.
|
||||
- Selection syntax (`@1`) will prefer `_selection_args` if present; otherwise it will fall back to plugin selection logic or sensible defaults (`-path` or `-title`).
|
||||
|
||||
This design keeps the selector-focused UX small and predictable while enabling full cmdlet interoperability via piping and `-run-cmd`.
|
||||
|
||||
Reference in New Issue
Block a user