YAML is a text format for serializing tree structures into text (like XML).
It can be sometimes tricky, but basic scenarios are intuitive and straightforward.
We can use YAML to describe relational data and load them using the relpipe-in-yaml
input filter:
film:
- title: Beetlejuice
year: 1988
director: Tim Burton
- title: Vacation
year: 1983
director: Harold Ramis
- title: Christmas Vacation
year: 1989
director: Jeremiah S. Chechik
- title: Celebrity
year: 1998
director: Woody Allen
- title: Great Balls of Fire!
year: 1989
director: Jim McBride
- title: Flatliners
year: 1990
director: Joel Schumacher
- title: Heathers
year: 1988
director: Michael Lehmann
- title: Blue Velvet
year: 1986
director: David Lynch
This YAML notation is almost as simple as Recfiles and like Recfiles it can be easily written by hand.
We can also use the JSON notation (it can be read by the same tool – relpipe-in-json
is currently just a symlink to relpipe-in-yaml
).
JSON is inferior to YAML and XML, but the JSON input filter might be useful when interacting with software that can produce only JSON.
The film
relation would look like this in JSON format (shortened):
{
"film": [
{
"title": "Beetlejuice",
"year": "1988",
"director": "Tim Burton"
},
{
"title": "…",
"year": "…",
"director": "…"
},
]
}
When we want to declare data types
– which is usually a good idea –
we add the attribute-metadata
header and move the data under the record
node:
"film":
"attribute-metadata":
- "name": "title"
"type": "string"
- "name": "year"
"type": "integer"
- "name": "director"
"type": "string"
"record":
- "title": "Beetlejuice"
"year": 1988
"director": "Tim Burton"
- "title": "Vacation"
"year": 1983
"director": "Harold Ramis"
- "title": "Christmas Vacation"
"year": 1989
"director": "Jeremiah S. Chechik"
- "title": "Celebrity"
"year": 1998
"director": "Woody Allen"
- "title": "Great Balls of Fire!"
"year": 1989
"director": "Jim McBride"
- "title": "Flatliners"
"year": 1990
"director": "Joel Schumacher"
- "title": "Heathers"
"year": 1988
"director": "Michael Lehmann"
- "title": "Blue Velvet"
"year": 1986
"director": "David Lynch"
This form (with data types and quotes) is also produced by the relpipe-out-yaml
tool.
When we load such YAML, we get this relational data:
film:
╭──────────────────────┬────────────────┬─────────────────────╮
│ title (string) │ year (integer) │ director (string) │
├──────────────────────┼────────────────┼─────────────────────┤
│ Beetlejuice │ 1988 │ Tim Burton │
│ Vacation │ 1983 │ Harold Ramis │
│ Christmas Vacation │ 1989 │ Jeremiah S. Chechik │
│ Celebrity │ 1998 │ Woody Allen │
│ Great Balls of Fire! │ 1989 │ Jim McBride │
│ Flatliners │ 1990 │ Joel Schumacher │
│ Heathers │ 1988 │ Michael Lehmann │
│ Blue Velvet │ 1986 │ David Lynch │
╰──────────────────────┴────────────────┴─────────────────────╯
Record count: 8
Of course, there might be multiple relations, not only one.
Once we convert the YAML to relational format, we can run any transformations and conversions as usual. For example we can use SQL to sort the recordsy by year:
cat film-1.yaml \
| relpipe-in-yaml \
| relpipe-tr-sql --relation 'film' "SELECT * FROM film ORDER BY year" \
| relpipe-out-recfile > relpipe-data/examples/film-1.rec
and Recfile output filter to get:
%rec: film
%type: year int
title: Vacation
year: 1983
director: Harold Ramis
title: Blue Velvet
year: 1986
director: David Lynch
title: Beetlejuice
year: 1988
director: Tim Burton
title: Heathers
year: 1988
director: Michael Lehmann
title: Christmas Vacation
year: 1989
director: Jeremiah S. Chechik
title: Great Balls of Fire!
year: 1989
director: Jim McBride
title: Flatliners
year: 1990
director: Joel Schumacher
title: Celebrity
year: 1998
director: Woody Allen
Or we may use XPath filter:
relpipe-tr-xpath --relation '.*' --where 'year >= 1980 and year < 1990'
to get the 80s films:
film:
╭──────────────────────┬────────────────┬─────────────────────╮
│ title (string) │ year (integer) │ director (string) │
├──────────────────────┼────────────────┼─────────────────────┤
│ Beetlejuice │ 1988 │ Tim Burton │
│ Vacation │ 1983 │ Harold Ramis │
│ Christmas Vacation │ 1989 │ Jeremiah S. Chechik │
│ Great Balls of Fire! │ 1989 │ Jim McBride │
│ Heathers │ 1988 │ Michael Lehmann │
│ Blue Velvet │ 1986 │ David Lynch │
╰──────────────────────┴────────────────┴─────────────────────╯
Record count: 6
YAML comes into play when we are looking for a friendly format for user-supplied data.
Relational pipes, open standard and free software © 2018-2022 GlobalCode