Securepub's .sync.py synchronization script makes it easy for developers to work on logically related files
under the src/ directory without worrying about where Zola, Elm and esbuild tooling wants them to be kept. It
also performs simple conversion and code generation that would typically be done in a makefile.
.sync.py is configured by _sync.conf files in subdirectories of src/.
.sync.py supports four kinds of transformations:
- File synchronization
- Merging yaml frontmatter from .md files
- Converting yaml frontmatter to json for workflows
- Converting yaml frontmatter to Elm flags
Running .sync.py
.sync.py is typically run by targets in the .justfile.
incremental mode
When run without any arguments .sync.py uses fswatch to detect changes to files under src/. If
a file changes the _sync.conf file in the corresponding subdirectory is checked to see if a transformation
should be applied.
In the .justfile, the 'dev-sync' target does this for interactive development.
force mode
When run with the --force argument .sync.py instead reads all the
_sync.conf files in the immediate children of src/ and applies the corresponding
transformation in the same manner it would have in interactive mode had the file changed.
In the .justfile, the 'copy' target does this before a site build.
remove mode
When run with the --remove argument .sync.py removes all the target files it would have written had it been run with --force mode.
In the .justfile, the 'remove' target does this to cleanup build artifacts.
File Synchronization
.sync.py supports copying files from the src/ directory to the work/ directory.
Files listed in the "sync:" section of _sync.conf will be copied to specified
locations below the work/ directory.
For example if the _sync.conf in a directory called somedir contains
--- sync: foo.md: zola/content/_v1.2401_/foo.md ---
Then src/somedir/foo.md will be copied to work/zola/content/_v1.2401_/foo.md:
src/
somedir/
_sync.conf ╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶┐
foo.md ─────────────────┐ ╵
│ ╵
work/ │ ╵
zola/ │ .sync.py
content/ │
_v1.2401_/ │
foo.md ◀────────────┘
Yaml frontmatter merging
.sync.py supports merging to allow multiple .md files to share common yaml data.
The "merge:" section of the _sync.conf identifies files whose frontmatter should be merged into the frontmatter of files listed beneath it before being written to the destination.
For example if we have
---
merge:
zola/_profile.yaml:
zola/add.md: zola/content/_v1.2401_/profile/
zola/update.md: zola/content/_v1.2401_/profile/
---
Then the frontmatter of zola/_profile.yaml will be both
-
merged into the frontmatter of zola/add.md and
the result written to zola/content/_v1.2401_/profile/add.md -
merged into the frontmatter of zola/update.md and
the result written to zola/content/_v1.2401_/profile/update.md
as shown below
src/
10.profile/
_sync.conf ╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶┐
zola/ ╵
_profile.md ╵ _profile.md
+ add.md ╵ + update.md
│ ╵ │
work/ │ ╵ │
zola/ │ .sync.py │
content/ │ │
_v1.2401_/ │ │
profile/ │ │
add.md ◀──────────┘ │
update.md ◀──────────────────────────┘
Converting yaml to json
.sync.py supports converting .yaml files to .json files.
The "tojson:" section of the _sync.conf identifies files which should be parsed as yaml and written to the destination as json with the file extensions changed to .json.
For example the _sync.conf of src/31.topic/_sync.conf contains
--- tojson: workflow/load_page.yaml: zola/static/_v1.2401_/workflow/topic/ ---
so that src/31.topic/workflow/load_page.yaml
is converted to work/zola/static/_v1.2401_/workflow/topic/load_page.json
src/
31.topic/
_sync.conf ╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶┐
workflow/ ╵
load_page.yaml ─────────┐ ╵
│ ╵
work/ │ ╵
zola/ │ .sync.py
static/ │
_v1.2401_/ │
workflow/ │
topic/ │
load_page.json ◀──┘
Converting yaml to Elm flags
.sync.py supports converting yaml to json and generating Elm flags from the result.
The "toelmflags:" section of the _sync.conf identifies files containing frontmatter which should be parsed as yaml, written as json and the result passed to Securepub's version of the json-to-elm ↗︎ code generator.
For example the _sync.conf of src/31.topic/_sync.conf contains
--- toelmflags: zola/_topic.yaml: elm/src/Topic/Flags.elm ---
so that
- the frontmatter of src/31.topic/zola/_topic.yaml is parsed as yaml
- the result written to work/elm/src/Topic/Flags.json and
- the result of json-to-elm ↗︎ written to work/elm/src/Topic/Flags.elm
src/
31.topic/
_sync.conf ╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶┐
zola/ ╵
_topic.yaml ──────────┐ ╵
│ ╵
work/ │ ╵
elm/ │ .sync.py
src/ │
Topic/ │
Flags.json ◀────────┘
│
│
│ json-to-elm
│
▼
Flags.elm
A complete example
The _sync.conf of src/60.join/_sync.conf shows all the features working together:
--- tojson: workflow/login_with_link.yaml: zola/static/_v1.2401_/workflow/join/ toelmflags: zola/_index.md: elm/src/Join/Flags.elm sync: elm/Join.elm: elm/src/ elm/Join/Decode.elm: elm/src/Join/ elm/Join/Model.elm: elm/src/Join/ elm/Join/Valid.elm: elm/src/Join/ elm/Join/View.elm: elm/src/Join/ ts/main.ts: main/join/ ts/io.ts: main/join/ zola/_index.md: zola/content/_v1.2401_/join/ zola/_index.sp.md: zola/content/_v1.2401_/join/ ---
This configuration
- generates login_with_link.json from login_with_link.yaml
- generates Flags.elm from _index.md
- copies files to work/elm, work/main and work/zola
as shown below
src work
└─60.join │
├─ _sync.conf ├─ elm
├─ elm │ └─ src
│ ├── Join.elm ╶╶╶╶╶╶╶╶╶sync╶╶╶╶╶╶╶▶ ├─ Join.elm
│ └── Join │ └─ Join
│ ├── Decode.elm │ ├─ Decode.elm
│ │ ╭╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶▶ ├─ Flags.elm
│ │ ├╶toelmflags╶╶╶╶▶ ├─ Flags.json
│ ├── Model.elm ╵ │ ├─ Model.elm
│ ├── Valid.elm ╵ │ ├─ Valid.elm
│ └── View.elm ╵ │ └─ View.elm
│ ╵ │
│ ╵ │
│ ╵ ├─ zola
│ ╵ │ ├─ content
│ ╵ │ │ └─ _v1.2401_
├─ zola ╵ │ │ └─ join
│ ├── _index.md ╶╶╶╶╶┴╶sync╶╶╶╶╶╶╶╶╶╶╶╶╶▶ ├─ _index.md
│ └── _index.sp.md │ │ └─ _index.sp.md
│ │ │
│ │ │
│ │ └─ static
│ │ └─ _v1.2401_
│ │ └─ workflow
├─ workflow │ └─ join
│ └── login_with_link.yaml ╶╶╶╶╶╶╶╶tojson╶╶╶╶╶▶ └─ login_with_link.json
│ │
│ └─ main
└─ ts └─ join
├── io.ts ╶╶╶╶╶╶╶╶╶╶╶sync╶╶╶╶╶╶╶▶ ├─ io.ts
└── main.ts └─ main.ts