Operator catalog

More tags45

All operators

Showing 81–93 of 93
OperatorVersionsExecutorTrustDownloadsActions
piflow_add_uuid_stopAuthor dacp-examplesPiFlow · Data processing

Prepend a distinct 32-character hexadecimal UUID per row. Params: column (new column name).

python9.2 KiB6
piflow_column_checkAuthor dacp-examplesPiFlow · Data processing

Validate full ordered column names against a reference and return left input. Params: none. Inputs: left, right. Left-input buffering limit: 128 MiB. Does not check physical units.

python8.6 KiB6
piflow_complement_by_memcacheAuthor dacp-examplesPiFlow · Data processing

Replace selected columns on a lookup key hit. Rows without a matching key keep their original values.

python35.7 KiB4
piflow_consistency_checkAuthor dacp-examplesPiFlow · Data processing

Left-join reference keys; prepend flagField=0 for matches, 1 otherwise. Params: flagField, correlationColumn (comma-separated). Duplicate reference keys retain join multiplicity. 128 MiB input limit each.

python9.7 KiB6
piflow_convert_schemaAuthor dacp-examplesPiFlow · Data processing

Rename columns; does not cast data types. Params: schema, e.g. station_id->station,temperature->air_c.

python9.1 KiB6
piflow_data_sortAuthor dacp-examplesPiFlow · Data processing

Globally sort a bounded input on one or more fields. Ascending order places nulls first, and descending places them last.

python35.7 KiB4
piflow_distinctAuthor dacp-examplesPiFlow · Data processing

Deduplicate records across Arrow batches; retains the first row per key. Params: optional columnNames; empty means all columns. Key-state limit: 128 MiB.

python9.0 KiB6
piflow_drop_fieldAuthor dacp-examplesPiFlow · Data processing

Remove record columns incrementally. Params: columnNames (comma-separated string); absent names ignored.

python9.2 KiB6
piflow_mock_dataAuthor dacp-examplesPiFlow · Data processing

Random Arrow records. source is a trigger stream consumed and discarded (matching upstream unused default input); output independent of input. Params schema e.g. name:String,age:Int,optional:Double:true; count string0..1000000. Types String/Int/Long/Float/Double/Boolean. Batches1024.

python10.3 KiB5
piflow_reichstein_checkAuthor dacp-examplesPiFlow · Data processing

EcoFlow Reichstein u-star screening implemented without Spark: six equal-count temperature groups, twenty wind bins, Pearson<0.4, median valid threshold; flag nighttime low-wind FC. Params soilTemperatureField,frictionWindSpeedField,FC strings,flagValue int32,percentFlux numeric(0,1]. Excludes null/nonfinite/flagged samples from estimation; requires120 valid observations. Fixes upper-bin bound and deterministic threshold order. Bounded128MiB.

python11.4 KiB5
piflow_select_fieldAuthor dacp-examplesPiFlow · Data processing

Select named record columns in order, incrementally by Arrow batch. Params: columnNames (comma-separated string).

python2.6 KiB8
piflow_subtractAuthor dacp-examplesPiFlow · Data processing

EXCEPT DISTINCT over whole rows: return unique left rows absent from right. Nulls compare equal for this set operation. No params. 128 MiB input limit each.

python8.9 KiB5
script_jsonl_projectAuthor xstream-script-examplesJSON · Data processing

Select top-level fields from JSONL objects without buffering the complete stream.

python9.4 KiB1