Source code

Download source archive ↓

README.md

1# Portable Galaxy adapters
2
3Each adapter exposes one named operation from a pinned Galaxy tool through the XStream Python SDK. It takes named Arrow `bytes` streams and emits a `result` bytes stream. Each source archive includes its upstream XML, any supporting source used for the adaptation, a hash manifest, an example step and test fixtures. The examples in the repository can be composed with `item_source` and `client_sink`.
4
5The adapter contract checks input ports and parameters before processing. It bounds each input at 4 MiB, each text input at 25,000 lines and the result at 8 MiB. These adapters process complete bounded inputs in memory. The preview consumes only complete samples of up to 4 KiB per port. Formats, defaults and supported subsets are listed for each operator in the catalog.
6
7## Verify and publish
8
9Run these commands from this directory. The public repository must already provide `python-arrow@2.0`.
10
11```sh
12python3 publish.py --url https://repo.conox.cn --password-file /secure/admin-password \
13  --output /tmp/galaxy-portable-results --family datamash
14```
15
16The publisher builds through the repository's submission API. It tests the isolated build using chunked Arrow inputs, structure inference, bounded previews and invalid parameters before approval. It then compares the public artifact to the tested bytes. An interrupted run can resume from pending submissions and public versions. Keep the results directory private. A file named by `--password-file`, or an authorized SSH host passed as `--admin-ssh`, supplies the password without adding it to an archive or the command line.
17
18## Supported operations
19
20The 26 Datamash steps support one named metric each, with optional grouping by one TSV column. They use lexical ordering, emit 12 significant digits, and report undefined statistics as `NA`. The median absolute deviation uses the 1.4826 scale factor. `galaxy_datamash_reverse` reverses the column order within each row.
21
22The 49 ImageJ and ImageMagick steps accept static, noninterlaced 8-bit grayscale PNG up to 512 × 512 pixels. They cover point arithmetic, rotations, cropping, resizing, neighborhood filters, thresholds, contrast, seeded noise, montage, blending and image comparison. PNG outputs preserve the two-dimensional raster. Comparison metrics return TSV. Kernel shapes, rounding and other subset choices are documented per operator.
23
24The 17 text and format steps cover JSON Lines collections and projections, TSV grouping and three-way joins, BIOM 1.0 tables, simple unweighted graph interchange and a single-sheet CSV-to-XLSX conversion. BIOM output preserves axis metadata and a dense or sparse matrix. XLSX cells are emitted as literal text.
25
26## Run the examples
27
28Use the included source archive's `fixtures/` files as inputs. If using the included `job.yaml` unchanged, place them at the root of a dataset named `galaxy` in the local XStream engine. The public audit script can fetch the published source and package and run all configured examples using the standalone engine.
29
30```sh
31python3 audit.py --artifact-tests /tmp/galaxy-portable-results \
32  --output /tmp/galaxy-portable-audit.json \
33  --runner /path/to/xstream-runner --work /tmp/galaxy-portable-jobs
34```