Debugging AWK transformations
- Relational pipes
- Principles
- Roadmap
- FAQ
- Specification
- Implementation
- Examples
- License
- Screenshots
- Download
- Support & contact
In most cases, AWK transformations should be quite straightforward, but sometimes we need to look inside the box.
Mapping attributes to variables
Relations have named attributes but in a language like AWK we work with named variables.
In most cases, the names will match 1:1. But not always.
The mapping is needed because not all valid attribute names are also valid variable names in particular language, thus sometimes some escaping or prefixing is necessary.
So there is --debug-variable-mapping option for printing the mappings between attributes and variables.
relpipe-in-fstab \
| relpipe-tr-awk \
--relation '.*' \
--for-each '1' \
--debug-variable-mapping \
| relpipe-out-tabular
This option prepends additional relation with these metadata to the stream:
fstab.variableMapping: ╭────────────────────┬───────────────────╮ │ attribute (string) │ variable (string) │ ├────────────────────┼───────────────────┤ │ device │ device │ │ dump │ dump │ │ mount_point │ mount_point │ │ options │ options │ │ pass │ pass │ │ scheme │ scheme │ │ type │ type │ ╰────────────────────┴───────────────────╯ Record count: 7 fstab: ╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬───────────────────────────────────────┬────────────────┬────────────────╮ │ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │ ├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼───────────────────────────────────────┼────────────────┼────────────────┤ │ UUID │ 29758270-fd25-4a6c-a7bb-9a18302816af │ / │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 1 │ │ │ /dev/sr0 │ /media/cdrom0 │ udf,iso9660 │ user,noauto │ 0 │ 0 │ │ │ /dev/sde │ /mnt/data │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 2 │ │ UUID │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home │ btrfs │ relatime │ 0 │ 2 │ │ │ /dev/mapper/sdf_crypt │ /mnt/private │ xfs │ relatime │ 0 │ 2 │ ╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴───────────────────────────────────────┴────────────────┴────────────────╯ Record count: 5
If we are interested only in the mappings, we should use it in combination with --drop option:
relpipe-in-fstab \
| relpipe-tr-awk \
--relation '.*' \
--for-each '1' \
--debug-variable-mapping \
--drop \
| relpipe-out-tabular
which skips the actual data:
fstab.variableMapping: ╭────────────────────┬───────────────────╮ │ attribute (string) │ variable (string) │ ├────────────────────┼───────────────────┤ │ device │ device │ │ dump │ dump │ │ mount_point │ mount_point │ │ options │ options │ │ pass │ pass │ │ scheme │ scheme │ │ type │ type │ ╰────────────────────┴───────────────────╯ Record count: 7
Because there were no collisions, variables have same names as attributes. But in this case:
relpipe-in-cli generate t 3 \
"if" string \
"6pack" string \
"spa ces" string \
| relpipe-tr-awk \
--relation t \
--debug-variable-mapping \
--drop \
| relpipe-out-tabular
mapping rules come in to the play:
t.variableMapping: ╭────────────────────┬───────────────────╮ │ attribute (string) │ variable (string) │ ├────────────────────┼───────────────────┤ │ 6pack │ _6pack │ │ if │ _if │ │ spa ces │ spa_ces │ ╰────────────────────┴───────────────────╯ Record count: 3
in order to make variable names valid in AWK.
Inspecting the internals of an AWK transformation
The relpipe-tr-awk calls AWK as a child process and passes data of given relation to it for actual processing.
Because it executes awk program found on $PATH, we can easily switch the AWK implementations.
In the source code repository, there is scripts/awk – a wrapper script.
We can modify the $PATH, so this wrapper will be called by relpipe-tr-awk.
This script captures CLI arguments, STDIN, STDOUT, STDERR and the exit code and saves them to files in the temp directory.
Using GNU Screen and the inotifywait we can build a kind of IDE and watch what happens inside during the transformation:
So we can inspect the generated AWK code and the inputs and outputs of the AWK process.
Recommended usage is described in the scripts/awk script.
Relational pipes, open standard and free software © 2018-2022 GlobalCode