OPERATOR / VERSION 2.0

piflow_filter

Data transformation · SQL

Filter Arrow batches by a SQLite SQL predicate (not Spark SQL). Params: condition, e.g. quality = 0 AND temperature >= 12. Scalar columns only.

View manifest JSON ↗

Use this operator

operator: piflow_filter@2.0

Configure remote_operator_library_urls with this repository’s origin. XStream verifies and caches artifacts by digest, sharing them across operators.

Named ports

  • sourceInputrecords

    Input records to process.

    Formats: Any compatible format
  • resultOutputrecords

    Keep records that satisfy the configured SQLite predicate.

Output structure and samples are provided by operator inference.

Parameters

  • conditionstringRequired

    SQLite SQL predicate applied to each record batch, e.g. quality = 0.

    How to fill
    Use column names from the input; quote string literals and CAST string columns for numeric comparisons.
    Example value
    quality = '0'

Operator usage example

Connect the input ports to upstream steps and adjust the parameters for your data.

id: process
operator: piflow_filter@2.0
inputs:
  source: read_source
params:
  condition: quality = '0'

Manual download

curl -fLo 'operator.jar' 'https://repo.conox.cn/v1/blobs/sha256/c8039cc7bf98d81ce0e60ca5c1e88a5fa9e0b6db07f97d86615f6d423e86a674/operator.jar'
sha256sum -- 'operator.jar'
curl -fLo 'environment.tar' 'https://repo.conox.cn/v1/blobs/sha256/9772a8410a7f0f615ad66d49d5aef4466cf95ed1c082c9b9f9c83742aa963598/environment.tar'
sha256sum -- 'environment.tar'

Compare checksums with the manifest. Download execution and build environments from the runtime catalog ↗.

Source and licensing

See Source archive under Artifact details

Source repository
Source repository not provided
Source revision
Revision not declared
License (publisher declared)
Apache-2.0
License text
License text not provided
Dependency SBOM
Dependency SBOM not provided

Build instructions

Build instructions not provided

Publisher-provided source, licenses and dependency information support your review. Availability and checksums do not certify safety, legal clearance, or that the executable matches the source.