Echo Area

Writing a blog with Org-mode, coleslaw, and GitLab CI

24 January 2021 10:00 AM (meta | org-mode | coleslaw | ox-coleslaw | ci | gitlab)

My previous deployment process wasn't very will organized. Using gitolite I had set up some post-receive hooks on my server that would run some coleslaw that happened to have been installed on my server (which had become quite ancient by now).

The new process is a bit more structured.

g export Export org-mode files generate Generate HTML using coleslaw export->generate copy Copy coleslaw files copy->generate deploy Deploy generate->deploy

Preparing to generate

Org mode is the only markup format that I really like working in. But it's pretty strictly tied to Emacs. Luckily Org mode is very good at converting to other formats, and coleslaw accepts raw HTML as one of the formats.

Exporting org-mode to coleslaw

Exporting is pretty simple. Before doing the actual export I need to run cask so that any dependencies get installed. This installs org-plus-contrib, htmlize, and ox-coleslaw. In the future this might install more if I need to add more dependencies for exporting, such as language major modes for exporting with syntax highlighting and such things.

Once the dependencies have been installed, I call Emacs in batch mode, which does:

  • Initialize package.el by calling the package-initialize function.

  • Load the project.el file, which defines how Org mode should export the files.

  • Export everything defined in project.el by calling the org-publish-all function.

generate-posts:
  before_script:
    - cask
  script:
    - cask emacs -batch -f package-initialize -l project.el -f org-publish-all
  artifacts:
    paths:
      - html

The hard work is done by Org mode, which is converting everything to HTML. The project.el file defines how this works.

To differentiate between what should become .post files and .page files I decided to put them in separate directories and then call org-coleslaw-publish-to-post and org-coleslaw-publish-to-page respectively. They both publish their results to the html/ directory. 

(setq org-publish-project-alist
      '(("posts"
         :base-directory "posts/"
         :publishing-directory "html/"
         :publishing-function org-coleslaw-publish-to-post)
        ("pages"
         :base-directory "pages/"
         :publishing-function org-coleslaw-publish-to-page
         :publishing-directory "html/")))

Once this is done, the .gitlab-ci.yml says to publish everything in HTML as the artifacts for this step.

Copy .post and .page files

There are still a number of files that were written in Markdown before I made ox-coleslaw, and these just need to be copied into the html/ directory and published as artifacts for this step. 

copy-rest:
  script:
    - mkdir html
    - cp -r .coleslawrc *.page *.post themes/ html/
  artifacts:
    paths:
      - html

This also copies the .coleslawrc file so that when we run coleslaw from the html/ directory it has the right settings.

Converting from coleslaw to HTML

Once everything's been prepared I just need to call coleslaw. 

build:
  image: registry.gitlab.com/ryuslash/blog
  needs:
    - job: generate-posts
      artifacts: true
    - job: copy-rest
      artifacts: true
  script:
    - cd html && coleslaw
  artifacts:
    paths:
      - public/

This specifies that it needs the artifacts from the previous two steps. Since they both published their artifacts into the html/ directory, this merges the results of both those steps into one directory.

The coleslaw configuration specifies that it should generate the files into the public/ directory. 

(;; Required information
 ;; ...
 :staging-dir "../public/"
)

This directory is then published as the step's artifact and used by the deploy step to actually upload to my server.

About the docker image

For this step I wrote a Docker image that installs Roswell and then uses that to install coleslaw. 

RUN ros install coleslaw-org/coleslaw \
    && coleslaw --help 2>/dev/null \
    && chmod a+rx /usr/local/bin/coleslaw

I call coleslaw --help because Roswell doesn't seem to actually compile coleslaw until the first time you run it. And for some reason the coleslaw executable's permissions didn't get set up correctly.

I manually build and publish this docker image for the moment, but I intend to automate that at some point later.

Deploy

The deploy step just ends up getting the public/ directory from the previous step and uses rsync to send it up to the server.

Next

Now that I've got a bit more structure in the build process it should be easier to extend it.

  • For one I want to change the way everything looks. And now I might be able to add something like compiling some less code into CSS and such.

  • I've also been thinking about running some checks as I build, such as if all the links still work and such.

  • Add caching of the org timestamps and Emacs dependencies.

Comments are closed.