Operator catalog

More tags45

All operators

Showing 41–80 of 104
OperatorVersionsExecutorTrustDownloadsActions
piflow_gaussian_mixture_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark Gaussian mixture model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.2 KiB3
piflow_gbt_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark gradient-boosted trees model to Arrow records and append prediction.

jvm22.5 KiB4
piflow_gbt_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark gradient-boosted trees model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.1 KiB3
piflow_histogramAuthor dacp-examplesPiFlow · Visualization

Produce grouped histogram data as records, not an image. Params: abscissa, customizedProperties mapping columns to COUNT/SUM/AVG/MAX/MIN. SQLite aggregates, scalar columns, 128 MiB.

python10.5 KiB5
piflow_html_parserAuthor dacp-examplesPiFlow · Documents

Extract visible UTF-8 text in document order. Each logical text block becomes one record.

python19.9 KiB3
piflow_identity_number_cleanAuthor dacp-examplesPiFlow · Data cleaning

Normalize Chinese identity number text and check the final checksum character of an 18-digit number. Replace the selected string columns while preserving other fields.

python12.6 KiB4
piflow_json_parserAuthor dacp-examplesPiFlow · JSON

Read one UTF-8 JSON object or a top-level array of objects as Arrow records, preserving nested fields.

python12.6 KiB4
piflow_json_saveAuthor dacp-examplesPiFlow · JSON

Serialize records as UTF-8 JSON Lines. The result is a bytes stream and each line is one JSON object.

python12.6 KiB4
piflow_json_string_parserAuthor dacp-examplesPiFlow · JSON

Parse a literal JSON object or array of objects into a records stream.

python12.6 KiB4
piflow_kmeans_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark k-means model to Arrow records and append prediction.

jvm22.5 KiB4
piflow_kmeans_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark k-means model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.2 KiB3
piflow_label_propagationAuthor dacp-examplesPiFlow · Graphs

Run synchronous label propagation using both directions of each edge. The lowest label wins a tie.

python30.6 KiB4
piflow_lda_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark online latent Dirichlet allocation model to Arrow records and append topicDistribution.

jvm22.5 KiB4
piflow_lda_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark online latent Dirichlet allocation model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.1 KiB3
piflow_line_chartAuthor dacp-examplesPiFlow · Visualization

Produce grouped line-chart data as records, not an image. Params: abscissa, customizedProperties mapping columns to COUNT/SUM/AVG/MAX/MIN. SQLite aggregates, scalar columns, 128 MiB.

python10.5 KiB5
piflow_load_graphAuthor dacp-examplesPiFlow · Graphs

Read whitespace-separated integer edges and output edge and distinct vertex records.

python30.5 KiB5
piflow_logistic_regression_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark logistic regression model to Arrow records and append prediction.

jvm22.6 KiB4
piflow_logistic_regression_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark logistic regression model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.3 KiB3
piflow_max_min_normalizationAuthor dacp-examplesPiFlow · Statistics

Scale the selected column to [0, 1]. The step materializes the input; a constant column produces nulls.

python12.7 KiB4
piflow_mdv_gAuthor dacp-examplesPiFlow · Data processing

EcoFlow MDV sliding-window interpolation at the same hour on neighboring days, excluding current day. Requires day_id,hour. Params fields,mdv_num (positive odd string),qualityControlMarkerValue (integer string).

jvm190.5 KiB5
piflow_mdv_iAuthor dacp-examplesPiFlow · Data processing

EcoFlow fixed-window MDV interpolation grouped by ceiling(day_id/mdv_num),hour. Preserves original zero contribution of flagged values to the group mean. Adds zone; fixes multiple-field replacement. Params fields,mdv_num positive string,qualityControlMarkerValue integer string.

jvm190.5 KiB5
piflow_mock_dataAuthor dacp-examplesPiFlow · Data processing

Random Arrow records. source is a trigger stream consumed and discarded (matching upstream unused default input); output independent of input. Params schema e.g. name:String,age:Int,optional:Double:true; count string0..1000000. Types String/Int/Long/Float/Double/Boolean. Batches1024.

python10.3 KiB5
piflow_multi_folder_json_parserAuthor dacp-examplesPiFlow · JSON

Read JSON documents from two independently connected item collections and combine records by field name.

python30.6 KiB3
piflow_multilayer_perceptron_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark binary multilayer perceptron model to Arrow records and append prediction.

jvm22.6 KiB4
piflow_multilayer_perceptron_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark binary multilayer perceptron model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.3 KiB3
piflow_naive_bayes_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark multinomial naive Bayes model to Arrow records and append prediction.

jvm22.5 KiB4
piflow_naive_bayes_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark multinomial naive Bayes model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.2 KiB3
piflow_pdf_parserAuthor dacp-examplesPiFlow · Documents

Extract native text from ordinary unencrypted PDF pages. Each record contains a page number and its text layer.

python820.3 KiB4
piflow_person_id_expansionAuthor dacp-examplesPiFlow · Data cleaning

Split a document type and number pair into the corresponding identity field and append deterministic source identifiers.

python30.3 KiB3
piflow_pf_coordinate_rotationAuthor dacp-examplesPiFlow · Data processing

EcoFlow planar-fit coordinate rotation using OLS and Breeze matrices. Params covariance/wind column names and output names. Requires a full-rank wind regression sample. Fixes upstream repeated Fc output into Le,Hs,u_star.

jvm193.7 KiB5
piflow_phone_number_cleanAuthor dacp-examplesPiFlow · Data cleaning

Normalize digits, remove separators and keep only mainland mobile numbers with upstream prefixes 13–15, 17 or 18. Replace the selected string columns while preserving other fields.

python12.6 KiB4
piflow_pie_chartAuthor dacp-examplesPiFlow · Visualization

Grouped pie-chart data as records, not an image. Params: dimension,indicator,indicatorOption COUNT/SUM/AVG/MAX/MIN. SQLite scalar aggregates, bounded 128 MiB. Output indicator_ACTION column.

python10.5 KiB5
piflow_pptx_parserAuthor dacp-examplesPiFlow · Documents

Extract paragraph text from PPTX slide parts in numerical part order, with the one-based slide number.

python19.9 KiB3
piflow_province_cleanAuthor dacp-examplesPiFlow · Data cleaning

Normalize common Chinese province names, suffixes and exact romanized names. Replace the selected string columns while preserving other fields.

python12.6 KiB4
piflow_rainfall_data_cullingAuthor dacp-examplesPiFlow · Data processing

EcoFlow rainfall quality control: flag selected flux columns at rainy timestamps and following 30 minutes. left=flux,right=meteorology; both require year,month,day,hour. Params fields,rainFallField,flagValue integer string. Upstream drops flux rows whose rainfall is null or unmatched.

jvm190.9 KiB5
piflow_random_forest_predictionAuthor dacp-examplesPiFlow · Machine learning

Apply a trained Spark random forest model to Arrow records and append prediction.

jvm22.6 KiB4
piflow_random_forest_trainingAuthor dacp-examplesPiFlow · Machine learning

Train a Spark random forest model on Arrow records and emit the trained model as a ZIP byte stream.

jvm21.2 KiB3
piflow_rdf_to_dfAuthor dacp-examplesPiFlow · Graphs

Parse RDF triples with explicit syntax. Output entity labels and literal properties, plus linked resources.

python1.1 MiB3
piflow_regex_text_processAuthor dacp-examplesPiFlow · Text

Append a text column with all matches of an RE2 regular expression replaced.

python30.3 KiB3
piflow_reichstein_checkAuthor dacp-examplesPiFlow · Data processing

EcoFlow Reichstein u-star screening implemented without Spark: six equal-count temperature groups, twenty wind bins, Pearson<0.4, median valid threshold; flag nighttime low-wind FC. Params soilTemperatureField,frictionWindSpeedField,FC strings,flagValue int32,percentFlux numeric(0,1]. Excludes null/nonfinite/flagged samples from estimation; requires120 valid observations. Fixes upper-bin bound and deterministic threshold order. Bounded128MiB.

python11.4 KiB5