Operator catalog

More tags45

All operators

Showing 1–21 of 21
OperatorVersionsExecutorTrustDownloadsActions
piflow_add_day_idAuthor dacp-examplesPiFlow · Data processing

Spark EcoFlow: sort year/month/day/hour; add id and day_id. Params: partitionNum string. Inputs materialized, max 128 MiB / one million rows.

jvm189.9 KiB6
piflow_add_day_stateAuthor dacp-examplesPiFlow · Data processing

EcoFlow sunrise/sunset day/night classifier using longitude, latitude and day_id (day-of-year); hour uses Beijing time. Params: longitude, latitude as strings. Spark batch, 128 MiB input limit.

jvm190.2 KiB5
piflow_batch_field_threshold_cleanAuthor dacp-examplesPiFlow · Data processing

EcoFlow inclusive threshold cleaning. left: records to clean; right: field,min_value,max_value,flag_value threshold records. No params. Fixes upstream Row-to-string field matching; field names must match exactly. Spark may coerce affected column types.

jvm189.6 KiB5
piflow_current_and_post_value_interpolationAuthor dacp-examplesPiFlow · Data processing

Interpolate two consecutive missing same-hour daily observations between valid endpoints. Requires year,month,day,hour. Params fields,qualityControlMarkerValue string. Fixes ignored marker and upstream absolute-difference error to support decreasing series.

jvm190.8 KiB5
piflow_dr_coordinate_rotationAuthor dacp-examplesPiFlow · Data processing

EcoFlow double coordinate rotation. Params map covariance/wind columns and output Fc,Le,Hs,u_star names. Nonzero horizontal wind required; outputs five-decimal strings. Original Breeze matrix algorithm.

jvm193.7 KiB5
piflow_execute_sql_stopAuthor dacp-examplesData transformation · SQL

Read-only SQLite SQL over a named input relation (not Spark SQL). Params: sql, viewName. Scalar inputs, max 128 MiB. Output types inferred; empty computed columns have null type.

jvm61.0 KiB6
piflow_filterAuthor dacp-examplesData transformation · SQL

Filter Arrow batches by a SQLite SQL predicate (not Spark SQL). Params: condition, e.g. quality = 0 AND temperature >= 12. Scalar columns only.

jvm61.0 KiB8
piflow_front_and_back_values_are_validAuthor dacp-examplesPiFlow · Data processing

EcoFlow same-hour interpolation using valid preceding and following days. Requires day_id,hour. Params fields and qualityControlMarkerValue integer string; fixes upstream ignored marker.

jvm190.8 KiB5
piflow_joinAuthor dacp-examplesData transformation · Relational

Key-based inner/left/right/full_outer record join. Params: joinMode, correlationColumn. Colliding right non-key columns receive _right suffix. 128 MiB per input.

jvm52.0 KiB7
piflow_mdv_gAuthor dacp-examplesPiFlow · Data processing

EcoFlow MDV sliding-window interpolation at the same hour on neighboring days, excluding current day. Requires day_id,hour. Params fields,mdv_num (positive odd string),qualityControlMarkerValue (integer string).

jvm190.5 KiB5
piflow_mdv_iAuthor dacp-examplesPiFlow · Data processing

EcoFlow fixed-window MDV interpolation grouped by ceiling(day_id/mdv_num),hour. Preserves original zero contribution of flagged values to the group mean. Adds zone; fixes multiple-field replacement. Params fields,mdv_num positive string,qualityControlMarkerValue integer string.

jvm190.5 KiB5
piflow_mergeAuthor dacp-examplesData transformation · Relational

Stream left records followed by right records, retaining duplicates. Merge schemas by field name: preserve left field order, append right-only fields and fill missing values with null. Conflicting types are rejected. No params. Fixed left/right ports.

jvm52.0 KiB10
piflow_pf_coordinate_rotationAuthor dacp-examplesPiFlow · Data processing

EcoFlow planar-fit coordinate rotation using OLS and Breeze matrices. Params covariance/wind column names and output names. Requires a full-rank wind regression sample. Fixes upstream repeated Fc output into Le,Hs,u_star.

jvm193.7 KiB5
piflow_rainfall_data_cullingAuthor dacp-examplesPiFlow · Data processing

EcoFlow rainfall quality control: flag selected flux columns at rainy timestamps and following 30 minutes. left=flux,right=meteorology; both require year,month,day,hour. Params fields,rainFallField,flagValue integer string. Upstream drops flux rows whose rainfall is null or unmatched.

jvm190.9 KiB5
piflow_single_field_threshold_cleanAuthor dacp-examplesPiFlow · Data processing

EcoFlow inclusive field thresholds with an additional Spark SQL condition; replace invalid values with flagValue. Params: fields,maxValue,minValue,flagValue,additionalCondition (strings). Spark may coerce affected column types.

jvm191.0 KiB5
piflow_storage_item_ecAuthor dacp-examplesPiFlow · Data processing

EcoFlow EC CO2 storage correction for 30-minute observations sorted by day_id,hour. Params Fc,co2_Avg column names,para_H height metres string,flagValue numeric string. Adds id; retains upstream missing/large-jump condition.

jvm190.6 KiB5
piflow_storage_item_pm__hsAuthor dacp-examplesPiFlow · Data processing

EcoFlow seven-layer sensible heat profile storage correction. left flux,right meteorology; date columns year,month,day,hour. Params Hs,rho_a_Avg column names,equipmentHeightDifference seven metre heights,everyTaFields seven temperature columns,time_interval positive seconds string. Fixes ignored density column name.

jvm192.3 KiB5
piflow_storage_item_pm_leAuthor dacp-examplesPiFlow · Data processing

EcoFlow seven-layer latent heat profile storage correction. left flux,right meteorology; year,month,day,hour required. Params LE column,equipmentHeightDifference seven metre heights,everyTaFields seven Celsius columns,everyPvaporFields seven vapor-pressure columns,time_interval seconds string. Original vapor conversion216.5*p/(273.15+T).

jvm192.4 KiB5
piflow_wild_spot_culling_mdv_gAuthor dacp-examplesPiFlow · Data processing

EcoFlow outlier removal from neighboring id windows, excluding current row. One field per step. Requires id; params fields,mdv_num positive odd string,multiple nonnegative string,statisticalType stddev/variance,flagValue numeric string. Fixes upstream SQL syntax and undefined contrast_flag.

jvm192.0 KiB5
piflow_wild_spot_culling_mdv_iAuthor dacp-examplesPiFlow · Data processing

EcoFlow outlier removal in fixed windows ceiling(id/mdv_num). Requires id. Params fields (one column),mdv_num positive string,multiple nonnegative string,statisticalType stddev/variance,flagValue numeric string. Fixes upstream ignored statistic.

jvm191.3 KiB5
piflow_wpl_correctionAuthor dacp-examplesPiFlow · Data processing

EcoFlow WPL flux correction, preserves original conversions and five-decimal string output Fc,Le,Hs. All params are column-name strings. co2_Avg in mg/m3,h2o_Avg in g/m3,rho_a_Avg in kg/m3,Ts_Avg in Celsius. Requires non-null numeric columns.

jvm191.0 KiB5