upstream/tools/datamash/datamash-ops.xml
1<tool id="datamash_ops" name="Datamash" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="@PROFILE@">
2 <description>(operations on tabular data)</description>
3 <macros>
4 <import>macros.xml</import>
5 </macros>
6 <expand macro="requirements" />
7 <expand macro="stdio" />
8 <command>
9 <![CDATA[
10 datamash
11 $header_in
12 $header_out
13 $need_sort
14 $print_full_line
15 $ignore_case
16 $narm
17 @FIELD_SEPARATOR@
18 #if str($grouping) != ''
19 --group '$grouping'
20 #end if
21 #for $oper in $operations
22 ${oper.op_name}
23 ${oper.op_column}
24 #end for
25 < $in_file > '$out_file'
26 ]]>
27 </command>
28 <expand macro="inputs_outputs">
29 <param argument="--group" name="grouping" type="text" label="Group by fields" help="Group consecutive rows with equal values in the chosen fields. If no columns are specified, each operation is performed in the entire input file. Comma separated list of column indices, e.g. 1,5">
30 <sanitizer invalid_char="">
31 <valid initial="string.digits">
32 <add value="," />
33 </valid>
34 <mapping initial="none">
35 <add source=" " target=""/>
36 </mapping>
37 </sanitizer>
38 <validator message="Invalid value in field. Allowed is a comma separated list of integer values or the empty string" type="regex">(^$)|(^\s*\d+\s*(,\s*\d+\s*)*$)</validator>
39 </param>
40 <param argument="--sort" name="need_sort" type="boolean" truevalue="--sort" falsevalue="" label="Sort input" help="Input file must be sorted by the grouping columns. Enable this option to automatically sort the input."/>
41 <param argument="--header-in" type="boolean" truevalue="--header-in" falsevalue="" label="Input file has a header line" />
42 <param argument="--header-out" type="boolean" truevalue="--header-out" falsevalue="" label="Print header line" />
43 <param argument="--full" name="print_full_line" type="boolean" truevalue="--full" falsevalue="" label="Print all fields from input file" />
44 <param argument="--ignore-case" type="boolean" truevalue="--ignore-case" falsevalue="" label="Ignore case when grouping" />
45 <param argument="--narm" type="boolean" truevalue="--narm" falsevalue="" label="Skip NA or NaN values" />
46 <repeat name="operations" default="1" min="1" title="Operation to perform on each group">
47 <param name="op_name" type="select" label="Type">
48 <option value="count">count</option>
49 <option value="sum">sum</option>
50 <option value="min">minimum</option>
51 <option value="max">maximum</option>
52 <option value="absmin">Absolute minimum</option>
53 <option value="absmax">Absolute maximum</option>
54 <option value="mean">Mean</option>
55 <option value="pstdev">Population Standard Deviation</option>
56 <option value="sstdev">Sample Standard Deviation</option>
57 <option value="median">Median</option>
58 <option value="q1">1st quartile</option>
59 <option value="q3">3rd quartile</option>
60 <option value="iqr">Inter-quartile range</option>
61 <option value="mad">Median Absolute Deviation</option>
62 <option value="pvar">Variance (population)</option>
63 <option value="svar">Variance (sample)</option>
64 <option value="sskew">Skewness (sample)</option>
65 <option value="pskew">Skewness (population)</option>
66 <option value="skurt">Kurtosis (sample)</option>
67 <option value="pkurt">Kurtosis (population)</option>
68 <option value="jarque">Jarque-Bera Normality test</option>
69 <option value="dpo">D Agostino-Pearson Omnibus Normality Test</option>
70 <option value="mode">Mode</option>
71 <option value="antimode">Anti-Mode</option>
72 <option value="rand">One random value from the group</option>
73 <option value="unique">Combine all unique values</option>
74 <option value="collapse">Combine all values</option>
75 <option value="countunique">Count Unique values</option>
76 </param>
77 <param name="op_column" data_ref="in_file" label="On column" type="data_column" />
78 </repeat>
79 </expand>
80 <tests>
81 <test>
82 <param name="in_file" value="group_compute_input.txt" ftype="tabular" />
83 <param name="grouping" value="2" />
84 <param name="header_in" value="true" />
85 <param name="header_out" value="true" />
86 <param name="need_sort" value="true" />
87 <param name="print_full_line" value="false" />
88 <param name="ignore_case" value="false" />
89 <repeat name="operations">
90 <param name="op_name" value="sum" />
91 <param name="op_column" value="3" />
92 </repeat>
93 <output file="group_compute_output.txt" name="out_file" ftype="tabular" />
94 </test>
95 <test>
96 <param name="in_file" value="group_compute_input.txt" ftype="tsv" />
97 <param name="grouping" value="2" />
98 <param name="header_in" value="true" />
99 <param name="header_out" value="true" />
100 <param name="need_sort" value="true" />
101 <param name="print_full_line" value="false" />
102 <param name="ignore_case" value="false" />
103 <repeat name="operations">
104 <param name="op_name" value="sum" />
105 <param name="op_column" value="3" />
106 </repeat>
107 <output file="group_compute_output.txt" name="out_file" ftype="tsv" />
108 </test>
109 <test>
110 <param name="in_file" value="group_compute_input.csv" ftype="csv" />
111 <param name="grouping" value="2" />
112 <param name="header_in" value="true" />
113 <param name="header_out" value="true" />
114 <param name="need_sort" value="true" />
115 <param name="print_full_line" value="false" />
116 <param name="ignore_case" value="false" />
117 <repeat name="operations">
118 <param name="op_name" value="sum" />
119 <param name="op_column" value="3" />
120 </repeat>
121 <output name="out_file" ftype="csv">
122 <assert_contents>
123 <has_n_lines n="7"/>
124 <has_line line="Arts,1310"/>
125 </assert_contents>
126 </output>
127 </test>
128 <test><!-- test with a file containing NA and NaN values and the -narm parameter " -->
129 <param name="in_file" value="na_values_input.tsv" ftype="tabular" />
130 <param name="grouping" value="2" />
131 <param name="header_in" value="true" />
132 <param name="print_full_line" value="false" />
133 <param name="need_sort" value="true" />
134 <param name="narm" value="true" />
135 <repeat name="operations">
136 <param name="op_name" value="mean" />
137 <param name="op_column" value="3" />
138 </repeat>
139 <output name="out_file" ftype="tabular">
140 <assert_contents>
141 <has_n_lines n="2"/>
142 <has_line_matching expression="DE\t173.5"/>
143 <has_line_matching expression="NL\t177.5"/>
144 </assert_contents>
145 </output>
146 </test>
147 </tests>
148 <help>
149<![CDATA[
150@HELP_HEADER@
151
152**Syntax**
153
154This tools performs common operations (such as summing, counting, mean, standard-deviation) on input file, based on tabular data. The tool can also optionaly group the input based on a given field.
155
156-----
157
158**Example 1**
159
160- Find the average score in statistics course of college students, grouped by their college major. The input file has three fields (Name,Major,Score) and a header line::
161
162 Name Major Score
163 Bryan Arts 68
164 Gabriel Health-Medicine 100
165 Isaiah Arts 80
166 Tysza Business 92
167 Zackery Engineering 54
168 ...
169 ...
170
171- Grouping the input by the second column (*Major*), sorting the input, and performing operations **mean** and **sample standard deviation** on the third column (*Score*), gives::
172
173 GroupBy(Major) mean(Score) sstdev(Score)
174 Arts 68.9474 10.4215
175 Business 87.3636 5.18214
176 Engineering 66.5385 19.8814
177 Health-Medicine 90.6154 9.22441
178 Life-Sciences 55.3333 20.606
179 Social-Sciences 60.2667 17.2273
180
181Note that input needs sorting here, since the column used for grouping (*Major*) is not sorted.
182
183This sample file is available at http://www.gnu.org/software/datamash .
184
185**Example 2**
186
187- Using the UCSC RefSeq Human Gene Track, available at: http://hgdownload.soe.ucsc.edu/goldenPath/hg38/database/refGene.txt.gz
188
189- List the number and identifiers of isoforms per gene. The gene identifier is in column 13, the isoform/transcript identifier is in column 2. Grouping by column 13 and performing **count** and **Combine all values** on column 2, gives::
190
191 GroupBy(field-13) count(field-2) collapse(field-2)
192 A1BG 1 NM_130786
193 A1BG-AS1 1 NR_015380
194 A1CF 6 NM_001198818,NM_001198819,NM_001198820,NM_014576,NM_138932,NM_138933
195 A2M 1 NM_000014
196 A2M-AS1 1 NR_026971
197 A2ML1 2 NM_001282424,NM_144670
198 ...
199
200- Count how many transcripts are listed for each chromosome and strand. Chromosome is on column 3, Strand is in column 4. Transcript identifiers are in column 2. Grouping by columns **3,4** and performing operation **count** on column 2, gives::
201
202 GroupBy(field-3) GroupBy(field-4) count(field-2)
203 chr1 + 2456
204 chr1 - 2431
205 chr2 + 1599
206 chr2 - 1419
207 chr3 + 1287
208 chr3 - 1249
209 ...
210
211@HELP_FOOTER@
212]]>
213 </help>
214 <expand macro="citation"/>
215</tool>