Reflecting on Pilot Workshops

Last updated on 2025-10-14 | Edit this page

The page serves as a facilitation guide for a debrief session designed to help lesson developers share and reflect on their experience of teaching a lesson still under development. We refer to these events as “pilot workshops” but participants in debriefs may report on testing only a part of their new lesson, rather than a complete draft. The content of this page is adapted from an episode of Collaborative Lesson Development Training, modified for standalone debrief sessions when the training programme was restructured in 2025.

An overview of the iterative process of lesson design and development, adapted from Nicholls' five phases, with step 5, 'Evaluate curriculum' highlighted.
In this session, we will discuss the final step of the iterative lesson design process: how you can use the notes, information, and feedback you collected when piloting a lesson to identify ways that the design and content could be improved.

In this session, we will reflect on and discuss the experience of teaching new lesson material for the first time. We will also explore ways that you could improve your lessons and workshops based on your observations and the feedback you received during and after the event.

Discussion

Discussion: lesson pilots (45 minutes)

Report out to the facilitator and other session participants your lesson pilot. Participants should choose what to focus on, e.g. based on their own reflections and feedback received during/after the event. The following prompts may help to guide your report:

  • What worked well both in terms of content and delivery?
  • What did not work as well?
  • How close were your time estimates to the actual time needed to teach the material and finish exercises?
  • How did the audience perceive the difficulty level of the material?
  • What will you do differently next time?
  • What will you change in the material you taught?
  • What will you change in the way you collect feedback in future pilots?
  • Did you identify anything that could be added as an Instructor Note to guide you and others next time the lesson is taught?

If multiple participants are joining to debrief from the same lesson pilot event, only one of them needs to report out. Others may choose to add their own reflections and ask additional questions.

We hope that the experience of teaching a new lesson was useful, even if it is still in the very early stages of development. We recommend that you continue to take this iterative approach to lesson development: testing new content early and often, and not being afraid to make significant changes to design, content and delivery based on your experience.

Testimonial

Do not aim for perfection before teaching your lesson

Perfect always takes so long because it don’t exist.

- Jeff Rosenstock

Next Steps

Callout

Piloting somebody else’s lesson?

Participants joining this session to report on a beta pilot workshop, i.e. an event where they taught a new lesson developed by somebody else, may not be able to take all of the actions listed below. The facilitator should encourage them to focus on the constructive feedback they could provide to the developers of the lesson they taught and to consider what role they want to take in the ongoing development and maintenance of the lesson afterwards.

Moving forward from your lesson pilot, the next steps to consider are:

  • Record all the identified changes and improvements that can be made to the lesson material and assign a person from the team responsible for each task. We recommend opening a GitHub issue for each separate action item.
  • Schedule follow-up co-working sessions with your team to carry on working on fixing issues and adding new content to maintain the momentum.
  • Add/Update your lessons’ Instructor Notes based on what you learned and to help other instructors who will teach your lesson in the future.
  • Think about the timeline for your next pilot(s), even provisionally, to help you set milestones and targets to work towards.
  • Is the lesson ready to move to the next stage of the life cycle, e.g. pre-alpha -> alpha or alpha -> beta?
  • Consider writing up your pilot experience in a mini blog post that you can share with the community.

Designing and developing quality lessons is hard - there are many pieces of a puzzle that have to come into place: both pedagogical and organisational. Do not be disheartened with the amount and type of feedback you may have received. Even Carpentries lessons that have been around for 10+ years receive improvement suggestions and fixes almost daily. From our experience, bigger lessons that are delivered over a few days require several full pilots before they can even be considered for a beta release. Planning smaller lesson trials (where you test only a portion of a lesson) and doing them more often with a friendly audience from your local groups and close colleagues is more manageable and will help you make steady progress.

Key Points
  • “Perfect is the enemy of good” - your lesson does not need to be perfect before you pilot or release it for community review. Early feedback from the target audience will help you avoid straying off your lesson plan.
  • Identify changes and improvements you want to make as a result of trialling your lesson and schedule co-working sessions to work on these tasks.