piflow_filter
Data transformation · SQLFilter Arrow batches by a SQLite SQL predicate (not Spark SQL). Params: condition, e.g. quality = 0 AND temperature >= 12. Scalar columns only.
Use this operator
operator: piflow_filter@2.0Configure remote_operator_library_urls with this repository’s origin. XStream verifies and caches artifacts by digest, sharing them across operators.
Named ports
- sourceInputrecords
Input records to process.
Formats: Any compatible format - resultOutputrecords
Keep records that satisfy the configured SQLite predicate.
Output structure and samples are provided by operator inference.
Parameters
- conditionstringRequired
SQLite SQL predicate applied to each record batch, e.g. quality = 0.
- How to fill
- Use column names from the input; quote string literals and CAST string columns for numeric comparisons.
- Example value
- quality = '0'
Operator usage example
Connect the input ports to upstream steps and adjust the parameters for your data.
id: process
operator: piflow_filter@2.0
inputs:
source: read_source
params:
condition: quality = '0'
Manual download
curl -fLo 'operator.jar' 'https://repo.conox.cn/v1/blobs/sha256/c8039cc7bf98d81ce0e60ca5c1e88a5fa9e0b6db07f97d86615f6d423e86a674/operator.jar'
sha256sum -- 'operator.jar'
curl -fLo 'environment.tar' 'https://repo.conox.cn/v1/blobs/sha256/9772a8410a7f0f615ad66d49d5aef4466cf95ed1c082c9b9f9c83742aa963598/environment.tar'
sha256sum -- 'environment.tar'Compare checksums with the manifest. Download execution and build environments from the runtime catalog ↗.
Source and licensing
See Source archive under Artifact details
- Source repository
- Source repository not provided
- Source revision
- Revision not declared
- License (publisher declared)
- Apache-2.0
- License text
- License text not provided
- Dependency SBOM
- Dependency SBOM not provided
Build instructions
Build instructions not provided
Publisher-provided source, licenses and dependency information support your review. Availability and checksums do not certify safety, legal clearance, or that the executable matches the source.