For simple filtering, we can use relpipe-tr-grep
.
But what if we need to write some complex query that contains AND and OR operators?
What if we need e.g. compare numbers – not only match texts against regular expressions?
There is a tool capable to do this and much more: relpipe-tr-scheme
!
Guile is the GNU implementation of Scheme language (something like Lisp and also full of parenthesis).
The relpipe-tr-scheme
reference implementation uses GNU Guile as a library, puts data in the Scheme context and evaluates Scheme expressions and then reads data from the Scheme context back and generates relational output from them.
Good news are that it is not necessary to know Lisp/Scheme to use this tool. For the first steps, it can be used just as a query language – like SQL, just a bit Polish.
We are looking for „satanistic“ icons in our filesystem – those that have size = 666 bytes.
find /usr/share/icons/ -type f -print0 \
| relpipe-in-filesystem \
| relpipe-tr-scheme --relation 'files.*' --where '(= $size 666)' \
| relpipe-out-tabular
Well, well… here we are:
filesystem:
╭───────────────────────────────────────────────────────────────────────┬───────────────┬────────────────┬────────────────┬────────────────╮
│ path (string) │ type (string) │ size (integer) │ owner (string) │ group (string) │
├───────────────────────────────────────────────────────────────────────┼───────────────┼────────────────┼────────────────┼────────────────┤
│ /usr/share/icons/elementary-xfce/actions/24/tab-new.png │ f │ 666 │ root │ root │
│ /usr/share/icons/elementary-xfce/apps/16/clock.png │ f │ 666 │ root │ root │
│ /usr/share/icons/elementary-xfce/mimes/22/x-office-spreadsheet.png │ f │ 666 │ root │ root │
│ /usr/share/icons/Tango/22x22/apps/office-calendar.png │ f │ 666 │ root │ root │
│ /usr/share/icons/Tango/16x16/actions/process-stop.png │ f │ 666 │ root │ root │
│ /usr/share/icons/breeze/actions/24/align-vertical-center.svg │ f │ 666 │ root │ root │
│ /usr/share/icons/breeze/devices/22/camera-photo.svg │ f │ 666 │ root │ root │
│ /usr/share/icons/oxygen/base/48x48/actions/tab-detach.png │ f │ 666 │ root │ root │
│ /usr/share/icons/oxygen/base/32x32/actions/insert-horizontal-rule.png │ f │ 666 │ root │ root │
│ /usr/share/icons/breeze-dark/actions/24/align-vertical-center.svg │ f │ 666 │ root │ root │
│ /usr/share/icons/breeze-dark/devices/22/camera-photo.svg │ f │ 666 │ root │ root │
│ /usr/share/icons/gnome/22x22/status/weather-overcast.png │ f │ 666 │ root │ root │
│ /usr/share/icons/gnome/16x16/actions/go-home.png │ f │ 666 │ root │ root │
╰───────────────────────────────────────────────────────────────────────┴───────────────┴────────────────┴────────────────┴────────────────╯
Record count: 13
The --relation 'files.*'
is a regular expression that says which relations should be processed in Scheme – others are passed through unchanged.
The --where '(= $size 666)'
is our condition.
The Polish
1
thing means that we write = $size 666
instead of $size = 666
.
It seems a bit weird but it makes sense – the =
is a function that compares two numbers and returns a boolean value –
so we just call this function and pass $size
and 666
arguments to it.
And because it is a function, there are (
parentheses)
.
Relational attributes are mapped to Scheme variables with same name, just prefixed with $
.
(we considered
₿
symbol, but $
seems to be still more common on keyboards in 2019)
While relational attribute name is an arbitrary string, Scheme variable names have some limitations, thus not all attributes can be mapped – those with spaces and some special characters are currently unsupported (this will be fixed in later versions by some kind of encoding/escaping).
We can also look for
--where '(> $size 100)'
which means „size is greater than 100“
or
--where '(< $size 100)'
which means „size is smaller than 100“.
The >=
and <=
also work as expected.
Scheme is strongly typed language and we have to use proper functions/operators for each type.
For strings, it is string=
instead of =
function:
relpipe-in-fstab \
| relpipe-tr-scheme --relation 'fstab' --where '(string= $type "btrfs")' \
| relpipe-out-tabular
The Btrfs filesystems in our fstab
:
fstab:
╭─────────────────┬──────────────────────────────────────┬──────────────────────┬───────────────┬──────────────────┬────────────────┬────────────────╮
│ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │
├─────────────────┼──────────────────────────────────────┼──────────────────────┼───────────────┼──────────────────┼────────────────┼────────────────┤
│ UUID │ a2b5f230-a795-4f6f-a39b-9b57686c86d5 │ /home │ btrfs │ relatime │ 0 │ 2 │
╰─────────────────┴──────────────────────────────────────┴──────────────────────┴───────────────┴──────────────────┴────────────────┴────────────────╯
Record count: 1
There is also string-prefix?
which evaluates whether the first string is a prefix of the second string:
relpipe-in-fstab \
| relpipe-tr-scheme --relation 'fstab' --where '(string-prefix? "/mnt" $mount_point)' \
| relpipe-out-tabular
So we can find filesystems mounted somewhere under /mnt
:
fstab:
╭─────────────────┬───────────────────────┬──────────────────────┬───────────────┬───────────────────────────────────────┬────────────────┬────────────────╮
│ scheme (string) │ device (string) │ mount_point (string) │ type (string) │ options (string) │ dump (integer) │ pass (integer) │
├─────────────────┼───────────────────────┼──────────────────────┼───────────────┼───────────────────────────────────────┼────────────────┼────────────────┤
│ │ /dev/sde │ /mnt/data │ ext4 │ relatime,user_xattr,errors=remount-ro │ 0 │ 2 │
│ │ /dev/mapper/sdf_crypt │ /mnt/private │ xfs │ relatime │ 0 │ 2 │
╰─────────────────┴───────────────────────┴──────────────────────┴───────────────┴───────────────────────────────────────┴────────────────┴────────────────╯
Record count: 2
There are much more functions – can be found in the Guile documentation
– like case-insensitive variants (e.g. string-ci=
) or regular expression search (string-match
).
Like in SQL, we can join multiple conditions together with logical operators AND and OR.
In Scheme these operators are also functions – they are written in the same (
fashion)
.
So we can e.g. look for icons that are „satanistic“ or „Orwellian“:
find /usr/share/icons/ -type f -print0 \
| relpipe-in-filesystem --file path --file size \
| relpipe-tr-scheme --relation 'files.*' --where '(or (= $size 666) (= $size 1984) )' \
| relpipe-out-tabular
Files with sizes 666 bytes or 1984 bytes:
filesystem:
╭───────────────────────────────────────────────────────────────────────┬────────────────╮
│ path (string) │ size (integer) │
├───────────────────────────────────────────────────────────────────────┼────────────────┤
│ /usr/share/icons/elementary-xfce/actions/48/mail-mark-important.png │ 1984 │
│ /usr/share/icons/elementary-xfce/actions/24/tab-new.png │ 666 │
│ /usr/share/icons/elementary-xfce/apps/16/clock.png │ 666 │
│ /usr/share/icons/elementary-xfce/mimes/22/x-office-spreadsheet.png │ 666 │
│ /usr/share/icons/Humanity-Dark/status/22/krb-no-valid-ticket.svg │ 1984 │
│ /usr/share/icons/Tango/22x22/apps/office-calendar.png │ 666 │
│ /usr/share/icons/Tango/16x16/actions/process-stop.png │ 666 │
│ /usr/share/icons/breeze/actions/24/align-vertical-center.svg │ 666 │
│ /usr/share/icons/breeze/devices/22/camera-photo.svg │ 666 │
│ /usr/share/icons/oxygen/base/48x48/actions/tab-detach.png │ 666 │
│ /usr/share/icons/oxygen/base/32x32/actions/insert-horizontal-rule.png │ 666 │
│ /usr/share/icons/Humanity/status/22/krb-no-valid-ticket.svg │ 1984 │
│ /usr/share/icons/breeze-dark/actions/24/align-vertical-center.svg │ 666 │
│ /usr/share/icons/breeze-dark/devices/22/camera-photo.svg │ 666 │
│ /usr/share/icons/gnome/48x48/status/user-busy.png │ 1984 │
│ /usr/share/icons/gnome/22x22/status/weather-overcast.png │ 666 │
│ /usr/share/icons/gnome/16x16/actions/go-home.png │ 666 │
╰───────────────────────────────────────────────────────────────────────┴────────────────╯
Record count: 17
Or we can look for icons that are in SVG format and (at the same time) Orwellian:
find /usr/share/icons/ -type f -print0 \
| relpipe-in-filesystem --file path --file size \
| relpipe-tr-scheme \
--relation 'files.*' \
--where '(and (string-suffix? ".svg" $path) (= $size 1984) )' \
| relpipe-out-tabular
Which is quite rare and we have only two such icons:
filesystem:
╭──────────────────────────────────────────────────────────────────┬────────────────╮
│ path (string) │ size (integer) │
├──────────────────────────────────────────────────────────────────┼────────────────┤
│ /usr/share/icons/Humanity-Dark/status/22/krb-no-valid-ticket.svg │ 1984 │
│ /usr/share/icons/Humanity/status/22/krb-no-valid-ticket.svg │ 1984 │
╰──────────────────────────────────────────────────────────────────┴────────────────╯
Record count: 2
We can nest ANDs and ORs and other functions as deep as we need and build even very complex queries. Prentheses nesting is fun, isn't it?
Relational pipes, open standard and free software © 2018-2022 GlobalCode