Operator catalog

More tags45

All operators

Showing 321–360 of 384
OperatorVersionsExecutorTrustDownloadsActions
piflow_column_checkAuthor dacp-examplesPiFlow · Data processing

Validate full ordered column names against a reference and return left input. Params: none. Inputs: left, right. Left-input buffering limit: 128 MiB. Does not check physical units.

python8.6 KiB6
piflow_complement_by_memcacheAuthor dacp-examplesPiFlow · Data processing

Replace selected columns on a lookup key hit. Rows without a matching key keep their original values.

python35.7 KiB4
piflow_consistency_checkAuthor dacp-examplesPiFlow · Data processing

Left-join reference keys; prepend flagField=0 for matches, 1 otherwise. Params: flagField, correlationColumn (comma-separated). Duplicate reference keys retain join multiplicity. 128 MiB input limit each.

python9.7 KiB6
piflow_convert_schemaAuthor dacp-examplesPiFlow · Data processing

Rename columns; does not cast data types. Params: schema, e.g. station_id->station,temperature->air_c.

python9.1 KiB6
piflow_convert_schema_typeAuthor dacp-examplesPiFlow · Data conversion

Convert all input fields to text or cast selected fields to string and 32-bit integer.

python43.0 KiB3
piflow_csv_string_parserAuthor dacp-examplesPiFlow · CSV · Data conversion

Parse headerless UTF-8 CSV text bytes to string records. Params: delimiter, schema; absent schema generates _0, _1, etc. Input stream replaces PiFlow string property.

python8.3 KiB6
piflow_custom_viewAuthor dacp-examplesPiFlow · Visualization · CSV

Custom view exported as UTF-8 CSV byte stream with a header. Replaces upstream HDFS visualization file with an output stream; no params, no rendering.

python9.3 KiB6
piflow_data_sortAuthor dacp-examplesPiFlow · Data processing

Globally sort a bounded input on one or more fields. Ascending order places nulls first, and descending places them last.

python35.7 KiB4
piflow_decompress_gzipAuthor dacp-examplesPiFlow · Archives

Decode the gzip member stream and return bytes. A tar.gz input returns the decompressed TAR bytes.

python19.9 KiB4
piflow_discretizationAuthor dacp-examplesPiFlow · Statistics

Assign zero-based numeric bins using upstream equal-width unit cuts or exact equal-frequency quantiles.

python12.6 KiB5
piflow_distinctAuthor dacp-examplesPiFlow · Data processing

Deduplicate records across Arrow batches; retains the first row per key. Params: optional columnNames; empty means all columns. Key-state limit: 128 MiB.

python9.0 KiB6
piflow_docx_parserAuthor dacp-examplesPiFlow · Documents

Extract nonempty text paragraphs from the main document part of a DOCX package, in document order.

python19.9 KiB4
piflow_drop_fieldAuthor dacp-examplesPiFlow · Data processing

Remove record columns incrementally. Params: columnNames (comma-separated string); absent names ignored.

python9.2 KiB6
piflow_email_cleanAuthor dacp-examplesPiFlow · Data cleaning

Convert full-width characters, lowercase the address and return an empty string for malformed addresses. Replace the selected string columns while preserving other fields.

python12.6 KiB5
piflow_evaluate_json_pathAuthor dacp-examplesPiFlow · JSON

Extract one field from each JSON object. Dot-separated paths traverse nested objects; missing fields return null.

python19.9 KiB4
piflow_excel_parserAuthor dacp-examplesPiFlow · Excel

Read the first sheet of each XLS or XLSX Item. Normalize first-row headers and merge string columns by name.

python699.8 KiB4
piflow_excel_readAuthor dacp-examplesPiFlow · Data conversion

XLSX byte stream to string records (no inference), dates as ISO strings, blanks null, cached formula values only. Params optional header boolean(default true),dataAddress [Sheet!]A1[:F35]. Replaces filePath with source bytes. Limits compressed/expanded128MiB and one million selected cells; materializes workbook bytes, preserves worksheet selection. No legacy XLS.

python519.3 KiB9
piflow_excel_writeAuthor dacp-examplesPiFlow · Data conversion

Records to XLSX byte stream; replaces filePath/saveMode with a new workbook output. Params optional header boolean(default true),dataAddress start cell e.g. Sheet1!A1 (no bounded write ranges). Dates become ISO text, formula-looking strings remain text. Bounded128MiB/one million rows; temporary XLSX serialization, not constant-memory streaming. Vendored openpyxl3.1.5/et_xmlfile2.0.0.

python518.5 KiB6
piflow_excel_write_multiple_sheetsAuthor dacp-examplesPiFlow · Excel

Write one worksheet for each of the left and right record inputs, and return one XLSX stream.

python538.5 KiB4
piflow_excel_write_sortAuthor dacp-examplesPiFlow · Excel

Sort records on the selected columns and emit an XLSX byte stream with data written at the requested cell.

python560.2 KiB3
piflow_field_type_conversionAuthor dacp-examplesPiFlow · Data conversion

Cast selected fields to double precision, 32-bit integer or fixed-precision decimal. Preserve other columns.

python35.7 KiB3
piflow_flatten_xml_parserAuthor dacp-examplesPiFlow · XML

Flatten a selected XML parent and optionally emit one row per child of a nested collection.

python19.9 KiB4
piflow_folder_csv_parserAuthor dacp-examplesPiFlow · CSV

Read CSV items from one folder. Skip a first row matching the configured schema and preserve field text.

python30.5 KiB5
piflow_folder_json_parserAuthor dacp-examplesPiFlow · JSON

Read JSON objects and arrays from one folder and concatenate by field name.

python30.5 KiB4
piflow_forkAuthor dacp-examplesPiFlow · Streams

Duplicate each input batch to two fixed ports, left and right. No params. Adaptation: explicit ports replace PiFlow dynamic outports.

python8.5 KiB8
piflow_histogramAuthor dacp-examplesPiFlow · Visualization

Produce grouped histogram data as records, not an image. Params: abscissa, customizedProperties mapping columns to COUNT/SUM/AVG/MAX/MIN. SQLite aggregates, scalar columns, 128 MiB.

python10.5 KiB5
piflow_html_parserAuthor dacp-examplesPiFlow · Documents

Extract visible UTF-8 text in document order. Each logical text block becomes one record.

python19.9 KiB4
piflow_identity_number_cleanAuthor dacp-examplesPiFlow · Data cleaning

Normalize Chinese identity number text and check the final checksum character of an 18-digit number. Replace the selected string columns while preserving other fields.

python12.6 KiB5
piflow_json_parserAuthor dacp-examplesPiFlow · JSON

Read one UTF-8 JSON object or a top-level array of objects as Arrow records, preserving nested fields.

python12.6 KiB4
piflow_json_saveAuthor dacp-examplesPiFlow · JSON

Serialize records as UTF-8 JSON Lines. The result is a bytes stream and each line is one JSON object.

python12.6 KiB4
piflow_label_propagationAuthor dacp-examplesPiFlow · Graphs

Run synchronous label propagation using both directions of each edge. The lowest label wins a tie.

python30.6 KiB5
piflow_line_chartAuthor dacp-examplesPiFlow · Visualization

Produce grouped line-chart data as records, not an image. Params: abscissa, customizedProperties mapping columns to COUNT/SUM/AVG/MAX/MIN. SQLite aggregates, scalar columns, 128 MiB.

python10.5 KiB5
piflow_load_graphAuthor dacp-examplesPiFlow · Graphs

Read whitespace-separated integer edges and output edge and distinct vertex records.

python30.5 KiB6
piflow_max_min_normalizationAuthor dacp-examplesPiFlow · Statistics

Scale the selected column to [0, 1]. The step materializes the input; a constant column produces nulls.

python12.7 KiB4
piflow_mock_dataAuthor dacp-examplesPiFlow · Data processing

Random Arrow records. source is a trigger stream consumed and discarded (matching upstream unused default input); output independent of input. Params schema e.g. name:String,age:Int,optional:Double:true; count string0..1000000. Types String/Int/Long/Float/Double/Boolean. Batches1024.

python10.3 KiB5
piflow_multi_folder_json_parserAuthor dacp-examplesPiFlow · JSON

Read JSON documents from two independently connected item collections and combine records by field name.

python30.6 KiB3
piflow_pdf_parserAuthor dacp-examplesPiFlow · Documents

Extract native text from ordinary unencrypted PDF pages. Each record contains a page number and its text layer.

python820.3 KiB5
piflow_person_id_expansionAuthor dacp-examplesPiFlow · Data cleaning

Split a document type and number pair into the corresponding identity field and append deterministic source identifiers.

python30.3 KiB4
piflow_phone_number_cleanAuthor dacp-examplesPiFlow · Data cleaning

Normalize digits, remove separators and keep only mainland mobile numbers with upstream prefixes 13–15, 17 or 18. Replace the selected string columns while preserving other fields.

python12.6 KiB5
piflow_pie_chartAuthor dacp-examplesPiFlow · Visualization

Grouped pie-chart data as records, not an image. Params: dimension,indicator,indicatorOption COUNT/SUM/AVG/MAX/MIN. SQLite scalar aggregates, bounded 128 MiB. Output indicator_ACTION column.

python10.5 KiB5