<p><strong>This post is linked to from the <a href="/blog-website.html">Blog Website Project</a></strong></p>
<hr>
<p>Nobody wants to test in production.</p>
<p>This guide covers how to preview your Pelican content locally as you write it.
Every time you save a markdown file or update the Pelican theme, the entire
site will re-generate and be hosted by a local web-server for you to review.</p>
<h1 id="install-dependencies">Install Dependencies</h1>
<ul>
<li><a href="https://docs.docker.com/install/">Docker</a></li>
<li><a href="https://git-scm.com/book/en/v2/Getting-Started-Installing-Git">Git</a></li>
<li>Text Editor: Your choice, I use <a href="https://github.com/vim/vim">Vim</a></li>
<li>Browser: Chrome has nice developer tools but anything works</li>
</ul>
<hr>
<h1 id="get-the-pelican-image">Get the Pelican Image</h1>
<p><strong>Option 1: Build your own</strong></p>
<p>This is the approach I suggest. You can see the guide I wrote explaining how to
build a Pelican image <a href="/docker-pelican-image.html">here</a>.</p>
<p><strong>Option 2: Use a pre-made image</strong>
I've shared my image for public use <a href="https://hub.docker.com/r/kpericak/pelican">here</a>.
If you plan on using it, you're best bet is to check how I use it on GitHub.</p>
<hr>
<h1 id="launch-a-persistent-pelican-container">Launch a Persistent Pelican Container</h1>
<p>Once you've either pulled or built the image, it will be listed in your Docker
image list.</p>
<pre><code class="language-bash">docker image list
</code></pre>
<p>Set the following variables. Note the paths should be absolute (vs relative).</p>
<ul>
<li><code>$image_name</code>: docker image name</li>
<li><code>$conf_file</code>: path to <code>pelicanconf.py</code> configuration file</li>
<li><code>$content_dir</code>: path to <code>content/</code> directory, where markdown files reside</li>
<li><code>$output_dir</code>: path to <code>output/</code> directory, where generated files will land.</li>
<li><code>$listen_port</code>: Which port to listen on for the dev web-server</li>
</ul>
<p>Optionally, you can set a theme path too. If you omit this and its associated
volume mount then the theme built into the Pelican image will be used instead.
This enables theme development.</p>
<ul>
<li><code>$theme_path</code>: Path to the Pelican theme</li>
</ul>
<p>Start the Docker container in daemon mode (<code>-d</code>)  as persistent container.
By setting the run command to <code>tail -f /dev/null</code>, the container will never
stop itself and instead sit idle until told to do otherwise.</p>
<p>Mount the directories specified by those variables into the container.</p>
<p>Forward <code>$listen_port</code> to the workstation so the site can be opened in a local
browser.</p>
<pre><code class="language-bash">image_name=""
conf_file=""
content_dir=""
output_dir=""
theme_path=""
listen_port=8000

docker run -d \
  --name pelican \
  -v $conf_file:/pelicanconf.py \
  -v $conftent_dir:/content \
  -v $output_dir:/output \
  -v $theme_path:/theme" \
  -p 0.0.0.0:$listen_port:8000 \
  image_name \
  tail -f /dev/null
</code></pre>
<hr>
<h1 id="launch-development-web-server">Launch Development Web-Server</h1>
<p>At this point you have a pelican container running, but it isn't doing
anything. The files are all mounted in, and it's prepared to forward its local
port 8000 to your specified workstation port, so all that's left is to start
Pelican's built-in development service.</p>
<p>This service will watch both the markdown files and the theme files for
changes, then reload the site files. It will also host the website on a
non-production web-server.</p>
<pre><code class="language-bash">docker exec -it pelican \
  pelican \
    -s /pelicanconf.py \
    --debug \
    --autoreload \
    --listen \
    --port 8000 \
    --bind 0.0.0.0 \
    /content \
    -o /output
</code></pre>


<h1 id="what-is-pelican">What is Pelican</h1>
<p><a href="https://github.com/getpelican/pelican">Pelican</a> is a free and open source
static website generator. It takes a collection of files
written in <a href="https://en.wikipedia.org/wiki/Markdown">Markdown</a> as its input,
applies a theme and configuration, then outputs a full static website.</p>
<p>A static website is a collection of HTML and CSS files that don't need any
database or back-end programming language. These websites can be hosted by any
web-server regardless of language support.</p>
<hr>
<h1 id="creating-a-pelican-project">Creating a Pelican Project</h1>
<p>While not necessary, its a really good idea to make a git project to source
control your blog content.</p>
<p>The project will hold the configuration for Pelican, which defines the plugins,
variables for the style-sheet, and the site name. This project also holds all
the markdown files which make up the blog content.</p>
<h2 id="project-layout">Project Layout</h2>
<p>The files should be placed in a structure as follows.</p>
<ul>
<li>Configuration goes in the <code>pelicanconf.py</code> file.</li>
<li>Blog posts go in <code>content/posts</code>, and can be sorted into subdirectories.
The nested directory structure has no impact on how the site is rendered.</li>
<li>Static pages go in <code>content/pages</code>. These are handled specially by the
templating engine.</li>
<li><code>content/images</code> stores image files</li>
<li><code>output/</code> doesn't need to exist here, but it's used by the <code>-o</code> (output)
argument in Pelican as the directory to store the generated website files.
This directory should be ignored by Git.</li>
</ul>
<pre><code class="language-text">.
+-- pelicanconf.py
+-- content/
|   +-- posts/
|       +-- first.md
|       +-- how-this-site-works/
|           +-- building-a-website.md
|   +-- pages/
|       +-- about.md
|   +-- images/
|       +-- pelican.png
+-- output/
</code></pre>
<hr>
<h1 id="pelican--markdown-syntax-reference">Pelican &#x26; Markdown Syntax Reference</h1>
<h2 id="post-metadata">Post Metadata</h2>
<p>The top of each post starts with some metadata that Pelican uses to scaffold
the site.</p>
<pre><code class="language-text">title: Title of the post or page
summary: Detailed summary
slug: short-name-goes-in-the-url
category: whereThisWillBeSorted
tags: tag1, tag2
date: 2019-08-28
modified: 2019-08-28
status: published or draft
</code></pre>
<ul>
<li><code>title</code>: defines the string at the top of the article and in the index</li>
<li><code>summary</code>: used in the index pages. If not defined, a large and
somewhat ugly subset of the top part of your post will auto-populate this
field.</li>
<li><code>slug</code>: defines the URL from which the post page is accessible</li>
<li><code>category</code>: place this post into a category, accessible from the sidebar in
this particular theme.</li>
<li><code>tags</code>: a comma delimited list of indexed sub-categories. Can have many.</li>
<li><code>date</code>: When the page was authored</li>
<li><code>modified</code>: When the page was last changed</li>
<li><code>status</code>: either <code>published</code> or <code>draft</code>. Defines whether the page is ready to
be displayed to the public.</li>
</ul>
<p>You can also define custom attributes to use in your themes. I've modified a
theme to allow custom thumbnail files for the index and a blog image which look
like this:</p>
<pre><code class="language-text">image: pelican.png
thumbnail: pelican-thumb.png
</code></pre>
<h2 id="headers">Headers</h2>
<pre><code class="language-text">Regular text
# Big header (h1)
## Smaller Header (h2)
### Even smaller header
</code></pre>
<h2 id="horizontal-line">Horizontal Line</h2>
<pre><code class="language-text">---
</code></pre>
<h2 id="table-of-contents">Table of Contents</h2>
<ul>
<li>
<ul>
<li><a href="#numbered-list">Numbered List</a></li>
<li><a href="#bullet-list">Bullet List</a></li>
<li><a href="#html">HTML</a></li>
<li><a href="#links">Links</a></li>
<li><a href="#images">Images</a></li>
</ul>
</li>
<li><a href="#preview-the-site">Preview the Site</a>
<ul>
<li><a href="#pull-the-pelican-image">Pull the Pelican Image</a></li>
<li><a href="#launch-the-pelican-container">Launch the Pelican container</a></li>
<li><a href="#launch-the-local-web-server-and-auto-rebuild-service">Launch the local web server and auto-rebuild service</a></li>
<li><a href="#browse-to-your-site">Browse to your site</a></li>
</ul>
</li>
<li><a href="#reference-links">Reference Links</a></li>
<li><a href="#next-up">Next Up</a></li>
</ul>
<h2 id="numbered-list">Numbered List</h2>
<p>The numbers will automatically increment.</p>
<pre><code class="language-text">1. First item
1. Second item
1. Third Item
</code></pre>
<h2 id="bullet-list">Bullet List</h2>
<pre><code class="language-text">- First Item
- Second Item
    - Nested List item with 4 spaces before the dash
- Third Item
</code></pre>
<h2 id="html">HTML</h2>
<p>Markdown files can also support raw HTML content, allowing you a lot of
flexibility.</p>
<pre><code class="language-html">&#x3C;table class="mytable>
  &#x3C;tr>&#x3C;th>Header&#x3C;/th>&#x3C;/tr>
  &#x3C;tr>&#x3C;td>Cell&#x3C;/td>&#x3C;/td>&#x3C;/tr>
&#x3C;/table>
</code></pre>
<h2 id="links">Links</h2>
<p>Linked text goes in square brackets the URL in round brackets.</p>
<pre><code class="language-text">[A web link](https://example.com)
</code></pre>
<h2 id="images">Images</h2>
<pre><code class="language-text">![Alt Text](/images/blog_workflow.png)
</code></pre>
<hr>
<h1 id="preview-the-site">Preview the Site</h1>
<p>The content is ready to be converted into a website. Use a Pelican container to
generate the site. The pelican container can also act as a local web server to
preview changes before pushing them.</p>
<h2 id="pull-the-pelican-image">Pull the Pelican Image</h2>
<p>Pull a pelican image from GCR or Docker Hub.
Mine is <code>gcr.io/kylepericak/pelican</code> on GCR and <code>kpericak/pelican</code> on Docker
Hub. The GCR image is private, since they charge me for the downloads.</p>
<pre><code class="language-bash">docker pull gcr.io/kylepericak/pelican
# or
docker pull kpericak/pelican
</code></pre>
<h2 id="launch-the-pelican-container">Launch the Pelican container</h2>
<p>Run this from the project root. It will mount the content/ and output/
directories into a new container named "pelican". It will also forward the
port Pelican's web server listens on (8000) to the workstation's port 8888.</p>
<p>The Pelican auto-reload feature looks like maybe its multithreaded or something,
so running it as the root process of the docker container doesn't work.
Instead, launch the container with the <code>tail -f /dev/null</code> command so it just
idles.</p>
<pre><code class="language-bash">docker run -d \
  --name pelican \
  -v $(pwd)/content:/content \
  -v $(pwd)/output:/output \
  -p 8888:8000 \
  gcr.io/kylepericak/pelican \
  tail -f /dev/null
</code></pre>
<h2 id="launch-the-local-web-server-and-auto-rebuild-service">Launch the local web server and auto-rebuild service</h2>
<p>This will build the site files, launch the web server, and watch for new
file changes so it can automatically rebuild.</p>
<p>The example <code>docker exec</code> command runs interactively. I like to run it in a
dedicated tmux pane to show if anything has gone wrong.</p>
<pre><code class="language-bash">docker exec -it pelican \
  pelican \
    -s /pelicanconf.py \
    --debug \
    --autoreload \
    --listen \
    /content \
    -o /output
</code></pre>
<h2 id="browse-to-your-site">Browse to your site</h2>
<p><a href="http://localhost:8888">http://localhost:8888</a></p>
<hr>
<h1 id="reference-links">Reference Links</h1>
<ul>
<li><a href="http://docs.getpelican.com/en/3.6.3/content.html">Official Pelican Syntax Reference</a></li>
</ul>
<hr>
<h1 id="next-up">Next Up</h1>
<p>Now that the content is written, its time to
<a href="/htsw-4-gcs-static-website.html">host the static website on Google Cloud Storage</a>.</p>


<p><strong>This post is linked to from the <a href="/blog-website.html">Blog Website Project</a></strong></p>
<hr>
<p>Pelican is a free and open source static website generator written in Python.
It accepts a theme, a config file, and a directory of markdown documents as
input and uses them to generate the HTML and CSS files that make up a static
website.</p>
<p>This post covers how to build a Docker image that contains the Pelican software
and the theme which I chose to use with it. This image is reusable and
<a href="https://hub.docker.com/r/kpericak/pelican">freely available on Docker Hub</a>.</p>
<h1 id="dockerfile">Dockerfile</h1>
<p>As with any Docker image, the most important piece is the Dockerfile.</p>
<p>You can see the current file I'm using <a href="https://github.com/kylep/pelican/blob/master/Dockerfile">here</a>.</p>
<p>In this Dockerfile, Pelican, its theme, and plugins are installed to the image.</p>
<pre><code class="language-text">FROM ubuntu:bionic
ADD requirements.txt ./
RUN apt-get update \
    &#x26;&#x26; apt-get install -y --no-install-recommends \
         git \
         python \
         python-pip \
         python-setuptools \
    &#x26;&#x26; rm -rf /var/lib/apt/lists/* \
    &#x26;&#x26; pip install -r requirements.txt \
    &#x26;&#x26; git clone https://github.com/kylep/voidy-bootstrap.git /theme \
    &#x26;&#x26; mkdir -p /tmp-plugins/ /plugins \
    &#x26;&#x26; git clone --recurse \
           https://github.com/getpelican/pelican-plugins.git /tmp-plugins \
    &#x26;&#x26; cp -r /tmp-plugins/series /plugins/ \
    &#x26;&#x26; rm -r /tmp-plugins
</code></pre>
<h3 id="from-ubuntubionic">FROM ubuntu:bionic</h3>
<p>This section specifies the Docker base image. The above docker file uses
<a href="https://hub.docker.com/_/ubuntu/">the official Docker Hub Ubuntu image</a>.</p>
<h3 id="add-requirementstxt">ADD requirements.txt</h3>
<p>The requirements.txt file lists the python packages which Pelican depends upon.
Docker's <code>ADD</code> will copy that file into the image at the specified path.</p>
<p>The minimum packages required are:</p>
<pre><code class="language-text">pelican
markdown
</code></pre>
<h3 id="run-">RUN ...</h3>
<p>The run task is basically a bash script all squished into one line. While its
harder to read with all the line continuation and <code>&#x26;&#x26;</code> symbols, using them
leads to Docker only building a single layer.</p>
<p><a href="https://docs.docker.com/develop/develop-images/dockerfile_best-practices/">See the best practices for writing docker images</a>.</p>
<p>Since this is an Ubuntu image, <code>apt</code> is the package manager. It installs Git
and some python packages. Git is required to pull the Pelican theme from
Github.</p>
<p>Only the minimum Python packages are installed by apt, since pip is the
preferred python package manager.</p>
<p><code>rm -rf /var/lib/apt/lists/*</code> removes the leftover apt artifacts, helping to
shrink the Docker image.</p>
<p>After the theme is pulled from GitHub, Pelican's open source plugins are also
pulled. The ones desired are copied to <code>/plugins</code>, then the (huge) checkout is
deleted to reduce image size.</p>
<hr>
<h1 id="running-the-docker-image">Running the Docker Image</h1>
<p>Locally, you need to mount the content into the container. Something like this
will work:</p>
<pre><code class="language-bash">docker run \
  --name pelican \
  -v $(pwd):/workspace \
  gcr.io/kylepericak/pelican:latest \
  pelican -o /workspace/output -s /workspace/pelicanconf.py
</code></pre>
<p>The above command will load the <code>content/</code> and <code>pelicanconf.py</code> files into
Pelican, and write the static website to <code>output/</code>.</p>


<p>The objective of this project is to build a technology blog website,
satisfying the following requirements:</p>
<ul>
<li><strong>Affordable</strong>: The website should cost as little as possible to maintain</li>
<li><strong>Maintainable</strong>: Authoring and updating content on the website should be
simple and fast.</li>
<li><strong>Extensible</strong>: This is a software developer's website. It needs to enable
custom code so fun features can be built into it.</li>
<li><strong>Custom Domain</strong>: Some free/cheap blog options require a custom FQDN suffix,
such as <code>x.wordpress.com</code> sites. This blog needs to work with any domain
name.</li>
<li><strong>Fun</strong>: The blog needs to use technologies that are current and interesting.</li>
</ul>
<hr>
<h1 id="project-posts--progress">Project Posts &#x26; Progress</h1>
<p>This project includes, or will include, the following posts.
If any aren't finished, check back later!</p>
<table>
<thead>
<tr>
<th align="left">Date</th>
<th align="left">Article</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">2019-08-01</td>
<td align="left"><a href="/docker-pelican-image.html">Building a Pelican Docker image</a></td>
</tr>
<tr>
<td align="left">2019-08-03</td>
<td align="left"><a href="/writing-pelican-content.html">Writing blog content for Pelican</a></td>
</tr>
<tr>
<td align="left">2019-08-04</td>
<td align="left"><a href="/pelican-dev-environment.html">A simple Pelican development environment</a></td>
</tr>
<tr>
<td align="left">2019-08-05</td>
<td align="left"><a href="/google-container-registry.html">Saving Docker images in Google's Container Registry</a></td>
</tr>
<tr>
<td align="left">2019-08-06</td>
<td align="left"><a href="/google-cloud-build.html">Google Cloud Build with GitHub Trigger</a></td>
</tr>
<tr>
<td align="left">2019-08-09</td>
<td align="left"><a href="/google-cloud-storage-website.html">Hosting a website using Google Cloud Storage</a></td>
</tr>
<tr>
<td align="left">2019-12-02</td>
<td align="left"><a href="/cloudflare-https.html">Using Cloudflare to provide HTTPS</a></td>
</tr>
</tbody>
</table>
<hr>
<h1 id="design--workflow">Design &#x26; Workflow</h1>
<p>At a high level, with this setup you just write your content and push it to
git. The rest takes care of itself.</p>
<p>Here's how it works:</p>
<ol>
<li>Blog content is written as text in markdown files. I use Vim.</li>
<li>When the content is ready, code is pushed to a GitHub repository.</li>
<li>A Google Cloud Build trigger watches for commits. When one happens, it loads
the Git master branch to its temporary workspace.</li>
<li>Cloud Build loads a pre-created Pelican Docker image is from Google's
Container Registry, and executes it against the markdown files to generate
a static website's .html and .css files.</li>
<li>Cloud Build uses an rsync image to transfer the files to a Google Storage
bucket.</li>
<li>Google Storage hosts the static site files on a custom domain as a website.</li>
</ol>
<p><img src="/images/blog_workflow.png" alt="Blog architecture diagram"></p>
<hr>
<h1 id="setup-configuring-pelican-and-gcp">Setup: Configuring Pelican and GCP</h1>
<h2 id="pelican-theme">Pelican Theme</h2>
<p>Before Pelican can render the markdown files into a site worth looking at,
it needs a theme to work with.</p>
<p>The easiest way to go is to <a href="http://www.pelicanthemes.com/">select a pre-made theme.</a></p>
<p>Personally, I forked a theme then heavily modified it.
Forking the theme prevents it from changing, and lets you change things as
needed. I'd suggest creating a fork of the theme even if you won't change it.</p>
<h2 id="pelican-docker-image">Pelican Docker Image</h2>
<h3 id="containerize-pelican">Containerize Pelican</h3>
<p>Next, the pelican software needs to be containerized for Cloud Build to use it.
You can either grab an existing image, such as the one I've shared on Docker
Hub, or build your own.</p>
<p><strong><a href="/docker-pelican-image.html">How to build a Docker image for Pelican</a></strong></p>
<h3 id="push-pelican-image-to-gcr">Push Pelican Image to GCR</h3>
<p>The Pelican image needs to be in the Google Container Registry for Cloud Build
to use it. GCR is like Docker Hub, but private and not free. It's pretty
affordable.</p>
<p><strong><a href="/google-container-registry.html">Using Google Cloud Registry</a></strong></p>
<h2 id="development-environment">Development Environment</h2>
<p>Google Storage's website hosting feature is great, but it has a cache that I
haven't yet figured out how to bypass. This means that if you push a mistake to
the storage, then push a fix, the fix may not display for like an hour.</p>
<p>To get past this, using a local service for development is a nice solution.
Also, this way you're not testing your code in production.</p>
<p>You can see the utility script I made <a href="https://github.com/kylep/kyle.pericak.com/blob/master/bin/pelican-dev.sh">here</a>,
but basically you just run a local Pelican image, then run <code>pelican</code> against
your content files with <code>--listen</code> and <code>--autoreload</code>.</p>
<p><strong><a href="/pelican-dev-environment.html">Pelican Website: Local Development Environment</a></strong></p>
<h2 id="cloud-build">Cloud Build</h2>
<p>Google Cloud Build is the automation framework used to watch for changes
committed to github, then transform the content files into a website and save
them to Google Cloud Storage.</p>
<p>Configure Cloud Build to watch for commits to the master branch of the blog
content repository, then run the pelican and rsync jobs against it.</p>
<p><strong><a href="/google-cloud-build.html">Google Cloud Build: Automatically Deploy Changes from GitHub</a></strong></p>
<h2 id="google-storage-web-hosting">Google Storage Web Hosting</h2>
<p>At this point, you've written your content, converted it into web files, and
synchronized those files with Google Cloud Storage. To make the website
accessible, you need to configure Google Cloud Storage to host the content as
a website. This step is also where you point your custom domain name at the new
site.</p>
<p><strong><a href="/google-cloud-storage-website.html">Google Cloud Storage: Static Website Hosting</a></strong></p>
<h1 id="secure-the-site-with-https">Secure the site with HTTPS</h1>
<p>These days there's really no good excuse to not have HTTPS on your site. On
a self-hosted site I usually use <a href="https://letsencrypt.org/">LetsEncrypt</a> with <a href="/certbot.html">certbot</a>, but
for this static site instead I've used the built-in HTTPS feature of
CloudFlare's free CDN solution.</p>
<p><strong><a href="/cloudflare-https.html">Using Cloudflare to provide HTTPS to an HTTP static site</a></strong></p>


Kyle Pericak

"It works in my environment"

Categories

Tags