The Carpentries Workbench

Last updated on 2022-09-05 | Edit this page

Overview

Questions

How is a lesson site set up and published with GitHub?
What are the essential configuration steps for a new lesson?
How do you create and modify the pages in a lesson?
How do you preview changes made to a lesson?

Objectives

Identify the key tools used in The Carpentries lesson infrastructure.
Complete the fundamental setup steps for a new lesson repository.
Find the most important source files for a lesson website.
Edit the RMarkdown source of an episode using RStudio.
Build and view a local version of a lesson website.
Preview the changes to a rendered lesson in a Pull Request.

Check your setup!

Before you work with this episode, make sure you have installed:

R
RStudio
Git

And please make sure you have a GitHub account you can connect to.

create repo on GitHub from Rmd template
tour that, then clone locally
create_episode
preview
push
open pull request

In this next part of the workshop, we will explore the new infrastructure for lessons - The Carpentries Workbench. In this section, you will learn about the key components of the Workbench and gain some hands-on experience of creating and editing a lesson website using the Workbench.

GitHub Pages

From the beginning, Carpentries lessons have been freely available for all to use under a CC-BY license.

The source of all The Carpentries lessons is made publicly-available in repositories on GitHub. By making our repositories public like this, we encourage others to help us maintain and improve our lessons, and make it as easy as possible for them to re-use and modify our lessons for their own purposes.

Via a system called GitHub Pages, users can build and host websites from the files present in any repository on GitHub. For many years, this has been how The Carpentries presents its lesson websites to the world.

Using The Carpentries Workbench

Starting in 2023, Carpentries lesson websites will be built with The Carpentries Workbench, a toolkit that converts Markdown and RMarkdown files into the HTML that can be served by GitHub Pages.

Creating a Lesson Repository

To get started, we first need to create a new repository for our lesson. We will use a template to do this, so that the new repository contains the basic files and folders that the Workbench needs in order to build a lesson site. There are currently two templates to choose between:

A Markdown template, suited for lessons that do not need to contain executable code.
An RMarkdown template, best suited to lessons that will include R source code that will generate output.

We are going to use the RMarkdown template in this workshop.

Creating a new lesson repository

Follow the steps demonstrated by the workshop lead:

generate a new repository from the Rmd lesson repository template
complete the configuration for that new repository:
- Add a name for the repository
  - The name should be descriptive but fairly brief, with hyphens (-) to separate words
  - The name can always be changed later, via the repository settings
- In the “Description” field, write the title of your lesson
- Choose “Public” visibility
- Make sure the “Include all branches” box is checked. This is not required, but means that the setup on GitHub pages moves faster initially.

After pressing the Create repository using this template button, you should be presented with a brand new lesson repository.

Main Files and Folders

The repository contains a number of files and folders. The most important files and folders are:

episodes/ contains the main narrative content of the lesson
instructors/ contains information useful for instructors such as instructor notes relevant for the overall lesson or extra exercises that could be useful for learners who would like an extra challenge
learners/ contains information that would be useful for learners such as a reference document and setup instructions.
profiles/ contain learner profiles for people who might find this lesson useful. This will help learners assess if the workshop is for them and help the instructors know what they can expect from their audience.

Note that The Workbench recognises both Markdown and RMarkdown filetypes.

… and the rest!

Most of these are source files for the content of our new lesson, but a few are accompanying files primarily intended for the repository itself rather than the lesson website. These are:

CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE.md
README.md
.gitignore
and the .github/ folder.

We will address all of the files later in the workshop. For now, we will move on to complete the basic setup of the lesson.

Configuring a Lesson Repository

Activating GitHub Pages

We need to tell GitHub to begin serving the lesson website via GitHub Pages. To do this, navigate to Settings, then choose Pages from the left sidebar. Under Source, choose the gh-pages branch, leave the folder set to / (root), and click Save. Now, copy the URL in the box: this will be the address of your lesson site.

Which branch to use?

Although we configure GitHub Pages to serve the lesson website from the gh-pages branch, the default working branch for a lesson will be main. For the rest of this training, you should add and edit files on main, and in future, when you open Pull Requests to update the lesson content, these should also be made to main. The gh-pages branch should never be modified by anyone other than the automated actions-user account.

Returning to the repository home page (e.g. by clicking the name of the project near the top left of the window), click the gear wheel icon near the top right, to edit the About box. Paste the Pages URL into the Website box and click Save changes.

After following these steps, when you navigate to the pages URL, you should be see a lesson website with The Carpentries logo and “Lesson Title” in the top-left corner. You may need to wait a minute or two for the GitHub to render the website.

Setting global parameters in `config.yaml`

The lesson title can be adjusted by modifying the config.yaml file in the repository. The config.yaml file contains several global parameters for a lesson—to determine some of the page styling, contact details for the lesson, etc—and is written in YAML, a structured file format of key-pair values. As well as the title of the lesson, you can and should adjust some of the other values in config.yaml, but you should not need to add new values or learn a lot about YAML to be able to configure your lesson.

Practice with `config.yaml` (5 minutes)

Complete the configuration of your lesson by adjusting the following fields in config.yaml:

email: add an email address people can contact with questions about the lesson/project.
created: the date the lesson was created (today’s date) in YYYY-MM-DD format.
keywords: a (short) list of keywords for the lesson, which can help people find your lesson when searching for resources online.
source: change this to the URL for your lesson repository.

We will revisit the life_cycle and carpentry fields in config.yaml later in this training.

Improving the `README.md`

The README.md file is the “front page” of your lesson repository on GitHub, and is written in Markdown. You should use it to provide basic information about the project, for anyone who finds their way to the source files for your lesson. Although we do not have time in this workshop, if you intend to keep working on your lesson when you leave, we recommend that you take a few minutes to update the README with some basic information about the project, e.g.:

the lesson title
a short description of the lesson
a list of the names of the authors, optionally linked to their GitHub profile

Working with a Local Copy

Before we explore the source files for the content of a lesson, we will make a copy of the new lesson repository on our local system. To do so, we need to clone the repository to a place on our system where we can find it easily.

In the example below, we clone the repository to our user’s Desktop on our filesystem, using the SSH protocol. (If you configured a personal access token for authenticating to GitHub, instead of SSH keys, you should use the HTTPS protocol.)

BASH

cd ~/Desktop
git clone git@github.com:USER_NAME/REPOSITORY.git

Now that we have a copy of the lesson repository locally, we need to make sure that we have everything we need to continue working with that repository on our own system.

Installing the Workbench

Open RStudio
In the console, type the following commands to install the workbench and all the required packages:

R

# register the repositories for The Carpentries and CRAN
options(repos = c(
  carpentries = "https://carpentries.r-universe.dev/",
  getOption("repos")
))

# Install the template packages to your R library
install.packages(c("sandpaper", "varnish", "pegboard", "tinkr"))

For Linux Users

If you have a linux system, you may need to install C libraries that the workbench dependencies require (such as libxml). A set of dependencies for ubuntu can be found using the following command:

BASH

curl https://carpentries.r-universe.dev/stats/sysdeps 2> /dev/null \
| jq -r '.library | select(. != null)'

OUTPUT

c++
curl
fontconfig
freetype
fribidi
harfbuzz
icu
libgit2
libjpeg-turbo
libpng
libtiff
libxml2
libxslt
openssl

Test your installation: copy and paste this code and see if you get similar output.

R

library('sandpaper')
library('fs')
library('rmarkdown')
rmarkdown::pandoc_version()
#> [1] '2.18'
tmp <- tempfile()
sandpaper::no_package_cache()
#> ℹ Consent for package cache revoked. Use `use_package_cache()` to undo.
sandpaper::create_lesson(tmp, open = FALSE)
#> → Creating Lesson in '/tmp/RtmpOBHY8v/file541125775e3a'...
#> ℹ No schedule set, using Rmd files in 'episodes/' directory.
#> → Creating Lesson in '/tmp/RtmpOBHY8v/file541125775e3a'...→ To remove this message, define your schedule in 'config.yaml' or use `set_episodes()` to generate it.
#> → Creating Lesson in '/tmp/RtmpOBHY8v/file541125775e3a'...✔ First episode created in '/tmp/RtmpOBHY8v/file541125775e3a/episodes/01-introduction.Rmd'
#> → Creating Lesson in '/tmp/RtmpOBHY8v/file541125775e3a'...ℹ Workflows up-to-date!
#> → Creating Lesson in '/tmp/RtmpOBHY8v/file541125775e3a'...✔ Lesson successfully created in '/tmp/RtmpOBHY8v/file541125775e3a'
#> → Creating Lesson in '/tmp/RtmpOBHY8v/file541125775e3a'...
#> /tmp/RtmpOBHY8v/file541125775e3a
sandpaper::build_lesson(tmp, preview = FALSE, quiet = TRUE)
fs::dir_tree(tmp, recurse = 1)
#> /tmp/RtmpOBHY8v/file541125775e3a
#> ├── CODE_OF_CONDUCT.md
#> ├── CONTRIBUTING.md
#> ├── LICENSE.md
#> ├── README.md
#> ├── config.yaml
#> ├── episodes
#> │   ├── data
#> │   ├── fig
#> │   ├── files
#> │   └── 01-introduction.Rmd
#> ├── index.md
#> ├── instructors
#> │   └── instructor-notes.md
#> ├── learners
#> │   └── setup.md
#> ├── links.md
#> ├── profiles
#> │   └── learner-profiles.md
#> └── site
#>     ├── DESCRIPTION
#>     ├── README.md
#>     ├── _pkgdown.yaml
#>     ├── built
#>     └── docs

If everything looks ok (e.g. no warnings or errors), we can go ahead and open the lesson repository as a project in RStudio (File -> Open Project…).

Having Trouble?

If you are having trouble installing a particular component of the workbench, we have created an RStudio cloud instance that you can use to follow along with the desktop commands: https://rstudio.cloud/content/4442910.

Previewing the Lesson

To see how the lesson website looks with this new episode added, we can build and serve a local version of the site, which will be automatically updated whenever we make changes to the source files.

R

sandpaper::serve()

After a short wait, a preview of the built lesson site should pop into the Viewer panel of RStudio. (You can “pop” it out into a web browser tab by clicking the little Show in new window button near the top of the Viewer panel.)

Lesson Content Source Files

Using the Files tab within RStudio, we can begin to look at where the content of a lesson is stored and written.

The “home page” of a lesson is created from the index.md file: this file should contain a brief introduction to the lesson, designed to give visitors to the page a first impression of the lesson and whether it is appropriate for them.

The main body of the lesson is written in episodes: the individual chunks or sections that the lesson is separated into. The episode pages of the lesson site will be constructed from Markdown files in the episodes folder of the lesson repository.

The episodes folder of the new repository contains a single file, 01-introduction.Rmd. The content of this file includes examples of how to write RMarkdown files for The Carpentries Workbench.

There are two important things to note:

The file begins with a short header, written in the same key-value pair syntax we encountered in config.yaml. The fields in this header describe the episode title and the estimated time (in minutes) required to teach it and complete the exercises.
The example content begins after the blank line that follows this header. It includes several lines that start with strings of colons (:::::::). These mark the beginnings and ends of structural elements within the page, called fenced divs. Each fenced div starts and ends with a string of colons. A word at the end of the starting colons indicates what kind of block it is. Fenced divs are used for a variety of different blocks of content, including:
- challenges and solutions
- instructor notes
- callouts i.e. optional content that can provide extra context or answer a frequently asked question about a particular topic in a lesson

MARKDOWN

---
title: "Using RMarkdown"
teaching: 10
exercises: 2
---

:::::::::::::::::::::::::::::::::::::: questions 

- How do you write a lesson using RMarkdown and `{sandpaper}`?

::::::::::::::::::::::::::::::::::::::::::::::::

::::::::::::::::::::::::::::::::::::: objectives

- Explain how to use RMarkdown with The Carpentries Workbench
- Demonstrate how to include pieces of code, figures, and nested challenge blocks

::::::::::::::::::::::::::::::::::::::::::::::::

Creating a New Episode

To add a new episode to a lesson, we can run the sandpaper::create_episode_rmd() function:

R

sandpaper::create_episode_rmd("Next Episode")

When you call this function, you should see a new file called 02-next-episode.Rmd appear in the episodes folder. It will have the same example content as 01-introduction.Rmd but with a different title in the YAML header, based on the name you specified for the new episode in the call to create_episode.

Pushing to GitHub

Push new episode back to github using either the Git tab in the top left pane or

Editing an Episode

We will use the code block and output below to demonstrate one of the new features provided by The Carpentries Workbench: automated feedback about changes to the rendered lesson content that would be made by a pull request.

MARKDOWN

```{r pyramid, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
pie(
  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
  init.angle = 315, 
  col = c("deepskyblue", "yellow", "yellow3"), 
  border = FALSE
)
```

Change the colour of the pyramid

Following along with the workshop leads, and working in RStudio on the local copy of your repository, follow these steps:

Create a new branch in Git, and switch to it
Find the code block above in the new episode file that you created earlier.
Change the colours of the pyramid parts of the figure above (defined in the col parameter of pie). Here is a resource for the available colour names in R
Commit the change
Push the branch to your remote repository on GitHub
Open a pull request to merge that branch into the main branch of your repository
Look at the preview of the changes to rendered content. This should appear as a comment in the pull request discussion tab shortly after the PR has been opened.

Show me the solution

MARKDOWN


```{r pyramid-green, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
pie(
  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
  init.angle = 315, 
  col = c("deepskyblue", "palegreen", "limegreen"), 
  border = FALSE
)
```

Key Points

Lesson sites are built from source repositories with GitHub Pages.
A new lesson repository can be created from a template maintained by The Carpentries, and configured by adjusting the config.yaml file.
The main pages of a lesson website are created from Markdown or RMarkdown files in the episodes folder.
A local version of a lesson can be built and viewed with sandpaper::serve().

The Carpentries Workbench

Overview

Questions

Objectives

Check your setup!

GitHub Pages

Using The Carpentries Workbench

Creating a Lesson Repository

Creating a new lesson repository

Main Files and Folders

… and the rest!

Configuring a Lesson Repository

Activating GitHub Pages

Which branch to use?

Setting global parameters in config.yaml

Practice with config.yaml (5 minutes)

Improving the README.md

Working with a Local Copy

BASH

Installing the Workbench

R

For Linux Users

BASH

OUTPUT

R

Having Trouble?

Previewing the Lesson

R

Lesson Content Source Files

MARKDOWN

Creating a New Episode

R

Pushing to GitHub

Editing an Episode

MARKDOWN

Change the colour of the pyramid

Show me the solution

MARKDOWN

Key Points

Setting global parameters in `config.yaml`

Practice with `config.yaml` (5 minutes)

Improving the `README.md`