If we need more complex filtering than relpipe-tr-grep
can offer, we can write an AWK transformation.
Then we can use AND and OR operators and functions like regular expression matching or numerical formulas.
The tool relpipe-tr-awk
calls real AWK program (usually GNU AWK) installed on our system and passes data of given relation to it.
Thus we can use any AWK feature in our pipeline while processing relational data.
Relational attributes are mapped to AWK variables, so we can reference them by their names instead of mere field numbers.
The --for-each
option is used for both filtering (instead of --where
)
and arbitrary code execution (for data modifications, adding records, computations or intentional side effects).
In AWK, filtering conditions are surrounded by (…)
and actions by {…}
.
Both can be combined together and multiple expressions can be separated by ;
semicolon.
The record()
function should be called instead of AWK print
(which should never be used directly).
Calling record()
is not necessary, when only filtering is done (and there are no data modifications).
With AWK we can filter records using standard numeric operators like ==, <, >, >= etc.
find -print0 | relpipe-in-filesystem \
| relpipe-tr-awk \
--relation '.*' \
--for-each '(size > 2000)' \
| relpipe-out-tabular
and e.g. list files with certain sizes:
filesystem: ╭──────────────────────┬───────────────┬────────────────┬────────────────┬────────────────╮ │ path (string) │ type (string) │ size (integer) │ owner (string) │ group (string) │ ├──────────────────────┼───────────────┼────────────────┼────────────────┼────────────────┤ │ ./relpipe-tr-awk.cpp │ f │ 2880 │ hacker │ hacker │ │ ./CLIParser.h │ f │ 5264 │ hacker │ hacker │ │ ./AwkHandler.h │ f │ 17382 │ hacker │ hacker │ ╰──────────────────────┴───────────────┴────────────────┴────────────────┴────────────────╯ Record count: 3
String values can be searched for certain regular expression:
relpipe-in-fstab \
| relpipe-tr-awk \
--relation '.*' \
--for-each '(mount_point ~ /cdrom/)' \
| relpipe-out-tabular
e.g. fstab
records having cdrom
in the mount_point
:
fstab: ╭─────────────────┬─────────────────┬──────────────────────┬───────────────┬──────────────────┬────────────────┬────────────────╮ │ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │ ├─────────────────┼─────────────────┼──────────────────────┼───────────────┼──────────────────┼────────────────┼────────────────┤ │ │ /dev/sr0 │ /media/cdrom0 │ udf,iso9660 │ user,noauto │ 0 │ 0 │ ╰─────────────────┴─────────────────┴──────────────────────┴───────────────┴──────────────────┴────────────────┴────────────────╯ Record count: 1
Case-insensitive search can be switched on by adding:
--define IGNORECASE integer 1
We can combine multiple conditions using ||
and &&
logical operators:
relpipe-in-fstab \
| relpipe-tr-awk \
--relation '.*' \
--for-each '(type == "btrfs" || pass == 1)' \
| relpipe-out-tabular
and build arbitrary complex filters
fstab: ╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬───────────────────────────────────────┬────────────────┬────────────────╮ │ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │ ├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼───────────────────────────────────────┼────────────────┼────────────────┤ │ UUID │ 29758270-fd25-4a6c-a7bb-9a18302816af │ / │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 1 │ │ UUID │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home │ btrfs │ relatime │ 0 │ 2 │ ╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴───────────────────────────────────────┴────────────────┴────────────────╯ Record count: 2
Nested (…)
work as expected.
And AWK can do much more – it offers plenty of functions and language constructs that we can use in our transformations. Comperhensive documentation can be found here: Gawk: Effective AWK Programming.
Relational pipes, open standard and free software © 2018-2022 GlobalCode