Operator catalog

More tags45

All operators

Showing 561–600 of 648
OperatorVersionsExecutorTrustDownloadsActions
piflow_email_cleanAuthor dacp-examplesPiFlow · Data cleaning

Convert full-width characters, lowercase the address and return an empty string for malformed addresses. Replace the selected string columns while preserving other fields.

python12.6 KiB4
piflow_evaluate_json_pathAuthor dacp-examplesPiFlow · JSON

Extract one field from each JSON object. Dot-separated paths traverse nested objects; missing fields return null.

python19.9 KiB3
piflow_excel_parserAuthor dacp-examplesPiFlow · Excel

Read the first sheet of each XLS or XLSX Item. Normalize first-row headers and merge string columns by name.

python699.8 KiB3
piflow_excel_readAuthor dacp-examplesPiFlow · Data conversion

XLSX byte stream to string records (no inference), dates as ISO strings, blanks null, cached formula values only. Params optional header boolean(default true),dataAddress [Sheet!]A1[:F35]. Replaces filePath with source bytes. Limits compressed/expanded128MiB and one million selected cells; materializes workbook bytes, preserves worksheet selection. No legacy XLS.

python519.3 KiB8
piflow_excel_writeAuthor dacp-examplesPiFlow · Data conversion

Records to XLSX byte stream; replaces filePath/saveMode with a new workbook output. Params optional header boolean(default true),dataAddress start cell e.g. Sheet1!A1 (no bounded write ranges). Dates become ISO text, formula-looking strings remain text. Bounded128MiB/one million rows; temporary XLSX serialization, not constant-memory streaming. Vendored openpyxl3.1.5/et_xmlfile2.0.0.

python518.5 KiB5
piflow_excel_write_multiple_sheetsAuthor dacp-examplesPiFlow · Excel

Write one worksheet for each of the left and right record inputs, and return one XLSX stream.

python538.5 KiB3
piflow_excel_write_sortAuthor dacp-examplesPiFlow · Excel

Sort records on the selected columns and emit an XLSX byte stream with data written at the requested cell.

python560.2 KiB2
piflow_execute_sql_stopAuthor dacp-examplesData transformation · SQL

Read-only SQLite SQL over a named input relation (not Spark SQL). Params: sql, viewName. Scalar inputs, max 128 MiB. Output types inferred; empty computed columns have null type.

jvm61.0 KiB6
piflow_field_type_conversionAuthor dacp-examplesPiFlow · Data conversion

Cast selected fields to double precision, 32-bit integer or fixed-precision decimal. Preserve other columns.

python35.7 KiB2
piflow_filterAuthor dacp-examplesData transformation · SQL

Filter Arrow batches by a SQLite SQL predicate (not Spark SQL). Params: condition, e.g. quality = 0 AND temperature >= 12. Scalar columns only.

jvm61.0 KiB8
piflow_flatten_xml_parserAuthor dacp-examplesPiFlow · XML

Flatten a selected XML parent and optionally emit one row per child of a nested collection.

python19.9 KiB3
piflow_folder_csv_parserAuthor dacp-examplesPiFlow · CSV

Read CSV items from one folder. Skip a first row matching the configured schema and preserve field text.

python30.5 KiB4
piflow_folder_json_parserAuthor dacp-examplesPiFlow · JSON

Read JSON objects and arrays from one folder and concatenate by field name.

python30.5 KiB3
piflow_forkAuthor dacp-examplesPiFlow · Streams

Duplicate each input batch to two fixed ports, left and right. No params. Adaptation: explicit ports replace PiFlow dynamic outports.

python8.5 KiB7
piflow_front_and_back_values_are_validAuthor dacp-examplesPiFlow · Data processing

EcoFlow same-hour interpolation using valid preceding and following days. Requires day_id,hour. Params fields and qualityControlMarkerValue integer string; fixes upstream ignored marker.

jvm190.8 KiB5
piflow_gaussian_mixture_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark Gaussian mixture model to Arrow records and append prediction.

jvm22.6 KiB4
piflow_gaussian_mixture_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark Gaussian mixture model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.2 KiB3
piflow_gbt_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark gradient-boosted trees model to Arrow records and append prediction.

jvm22.5 KiB4
piflow_gbt_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark gradient-boosted trees model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.1 KiB3
piflow_histogramAuthor dacp-examplesPiFlow · Visualization

Produce grouped histogram data as records, not an image. Params: abscissa, customizedProperties mapping columns to COUNT/SUM/AVG/MAX/MIN. SQLite aggregates, scalar columns, 128 MiB.

python10.5 KiB5
piflow_html_parserAuthor dacp-examplesPiFlow · Documents

Extract visible UTF-8 text in document order. Each logical text block becomes one record.

python19.9 KiB3
piflow_identity_number_cleanAuthor dacp-examplesPiFlow · Data cleaning

Normalize Chinese identity number text and check the final checksum character of an 18-digit number. Replace the selected string columns while preserving other fields.

python12.6 KiB4
piflow_joinAuthor dacp-examplesData transformation · Relational

Key-based inner/left/right/full_outer record join. Params: joinMode, correlationColumn. Colliding right non-key columns receive _right suffix. 128 MiB per input.

jvm52.0 KiB7
piflow_json_parserAuthor dacp-examplesPiFlow · JSON

Read one UTF-8 JSON object or a top-level array of objects as Arrow records, preserving nested fields.

python12.6 KiB4
piflow_json_saveAuthor dacp-examplesPiFlow · JSON

Serialize records as UTF-8 JSON Lines. The result is a bytes stream and each line is one JSON object.

python12.6 KiB4
piflow_json_string_parserAuthor dacp-examplesPiFlow · JSON

Parse a literal JSON object or array of objects into a records stream.

python12.6 KiB4
piflow_kmeans_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark k-means model to Arrow records and append prediction.

jvm22.5 KiB4
piflow_kmeans_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark k-means model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.2 KiB3
piflow_label_propagationAuthor dacp-examplesPiFlow · Graphs

Run synchronous label propagation using both directions of each edge. The lowest label wins a tie.

python30.6 KiB4
piflow_lda_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark online latent Dirichlet allocation model to Arrow records and append topicDistribution.

jvm22.5 KiB4
piflow_lda_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark online latent Dirichlet allocation model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.1 KiB3
piflow_line_chartAuthor dacp-examplesPiFlow · Visualization

Produce grouped line-chart data as records, not an image. Params: abscissa, customizedProperties mapping columns to COUNT/SUM/AVG/MAX/MIN. SQLite aggregates, scalar columns, 128 MiB.

python10.5 KiB5
piflow_load_graphAuthor dacp-examplesPiFlow · Graphs

Read whitespace-separated integer edges and output edge and distinct vertex records.

python30.5 KiB5
piflow_logistic_regression_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark logistic regression model to Arrow records and append prediction.

jvm22.6 KiB4
piflow_logistic_regression_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark logistic regression model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.3 KiB3
piflow_max_min_normalizationAuthor dacp-examplesPiFlow · Statistics

Scale the selected column to [0, 1]. The step materializes the input; a constant column produces nulls.

python12.7 KiB4
piflow_mdv_gAuthor dacp-examplesPiFlow · Data processing

EcoFlow MDV sliding-window interpolation at the same hour on neighboring days, excluding current day. Requires day_id,hour. Params fields,mdv_num (positive odd string),qualityControlMarkerValue (integer string).

jvm190.5 KiB5
piflow_mdv_iAuthor dacp-examplesPiFlow · Data processing

EcoFlow fixed-window MDV interpolation grouped by ceiling(day_id/mdv_num),hour. Preserves original zero contribution of flagged values to the group mean. Adds zone; fixes multiple-field replacement. Params fields,mdv_num positive string,qualityControlMarkerValue integer string.

jvm190.5 KiB5
piflow_mergeAuthor dacp-examplesData transformation · Relational

Stream left records followed by right records, retaining duplicates. Merge schemas by field name: preserve left field order, append right-only fields and fill missing values with null. Conflicting types are rejected. No params. Fixed left/right ports.

jvm52.0 KiB10
piflow_mock_dataAuthor dacp-examplesPiFlow · Data processing

Random Arrow records. source is a trigger stream consumed and discarded (matching upstream unused default input); output independent of input. Params schema e.g. name:String,age:Int,optional:Double:true; count string0..1000000. Types String/Int/Long/Float/Double/Boolean. Batches1024.

python10.3 KiB5