Handbooks & Guidelines

Step-by-step guide for developing new cybersecurity courses for healthcare organisations

The curriculum development process consists of four phases, namely A. Planning, B. Articulating and Developing, C. Implementing, and D. Evaluating (Ct.gov, 2001). Each phase consists of multiple steps, which are shown in Table 1. How much time and effort should be dedicated to each phase and/or individual step depends on multiple factors, such as the time and resources available, or the intended scale of the program, or its position among other education programs and interventions.

Not each phase will need as much work every time a new program is developed. However, it is advisable to consider the elements of each phase and step before continuing with developing the curriculum further. This will make sure no time or effort is wasted in the development of a curriculum that is likely to be ineffective or no suitable for the people it is aimed at, or that significant changes need to be made later.

The SecureHospitals.eu project developed a Curriculum Wizard to provide tailored advice for cybersecurity trainers in healthcare organisations. This online tool provides useful tips and insights for trainers who are developing a new, or updating an existing training program.

Table 1: Overview of the components of an effective curriculum development process

A. Planning

1. Curriculum development committee

2. Current issues and trends in cybersecurity for healthcare domain

3. Specific needs and issues for the healthcare organisation, department, or unit
B. Articulating and Developing

4. Describe the program philosophy

5. Defining the program, course level and course goals

6. Developing and sequencing of course level and course objectives

7. Identifying Resource Materials to Assist with Program Implementation

8. Selecting or developing assessments and instruments to measure progress
C. Implementing

9. Implementing the program

10. Feedback on the program
D. Evaluating

11. Updating the program

12. Determining the success of the program

Edited from Ct.gov’s guidelines for curriculum development (2001, p. 2).

A. Planning

Before the development of each cybersecurity training program starts, the developers should focus on the planning phase. A good planning will help structure the rest of the development process and it helps to ensure the end goal is reached.

The development of a curriculum depends on the reason it was launched, which can be due to a particular specific assignment from management or board level staff, or on a trainer’s own initiative after noticing a security risks, other programs may be initiated as a result of participating in national or EU-wide cybersecurity campaigns.

Three overarching questions are at the core of the planning phase: 1) Who will be involved in the development, implementation and evaluation of the curriculum?; 2) What are the issues and trends for cybersecurity for the healthcare domain?; 3) What are the specific needs and issues for the target audience that should be covered in the training?

Curriculum development committee

Before starting the development process, it needs to be clear who will be involved in the development and implementation of the training program. Together, these people can from a curriculum development committee. People who could be involved in this process are, for example, the information security officer, data protection officer, department or team managers, other trainers, and even staff members who represent those who the training is aimed at.

It is possible to keep things more informal and flexible by not forming an official committee. However, it is helpful to ask these parties for input throughout the whole curriculum development process, to make sure the new curriculum fits the needs of the organisation and the intended trainees.

Key challenges for the committee to focus on are to coordinate tasks and to communicate about the development progress regularly. This can be done through regular meetings, either face to face or via virtual/digital means. The goal of the committee and the final curriculum should be clear and agreed upon by those involved in its development. Sometimes there is a direct assignment from the CISO or Board of the healthcare organisation, which already has implications for what needs to be developed. In other cases, the goal is more open ended and requires more in-depth analysis of the current issues and challenges.

Current issues and trends in cybersecurity for healthcare domain

In order to develop a new curriculum that is relevant to the healthcare organisations, it is helpful to look at the current issues and trends in cybersecurity for healthcare organisations. Sometimes it may also be helpful to look at problems that organisations of other sectors face, for instance financial institutions. However, the problems organisations in this sector face may be too specific to them and not applicable to the healthcare sector (yet).

Since cybersecurity challenges do not really adhere to national borders, it may be helpful to look beyond the issues on the national level and study the current trends at the European and international level. Useful resources to use are the Online Awareness and Information Hub and the Community of Practice of the Securehospitals.eu project, the ENISA website, the SANS institute website or their dedicated Secure Awareness Training page, and a variety of technology and innovation blogs such as TechCrunch and Gizmodo.

Relevant questions for this step are:

  • What are current national challenges and issues?
  • What are current European challenges and issues?
  • What are current challenges and issues in the US?
  • What are current trends and solutions for each level that can be relevant for the curriculum?
  • How do these challenges, issues, trends and solutions apply to (my) healthcare organisation or unit?

Specific needs and issues for the healthcare organisation, department or unit

To ensure the relevance and effectiveness of each training, it should be targeted at the specific issues and challenges that currently apply, from the organisational level down to the staff member level. This assessment continues from the work done in the previous step by analysing how higher-level trends and issues apply to the organisation, department or unit.

As part of this step, a Training Needs Assessment (TNA) can be conducted. A TNA is often the first step in the process of actually developing and implementing training programs. A TNA should be done periodically, in order to contribute to the overall training and educational strategy of staff in an organisation or a professional group (Gould et al., 2004). However, a TNA can take up a lot of time, and may not be relevant to conduct for every new program. For more information on how to develop a TNA, please check the article on the SecureHospitals.eu OAIH, titled “Assessing training needs” (SecureHospitals.eu, n.d.).

Relevant questions to answer as part of this step are:

  • What are the needs and issues that should be covered in the training?
  • What are the desired outcomes and expectations?
  • How does this program fit with other needs of the organisations?
  • What are good ways to test if the goals are achieved?
  • What resources are available to develop and implement this training (funding, knowledge, IT support systems if necessary)?
  • Are there potential scheduling issues?
  • What teaching or instructional methods that can be used?
  • What are the purposes of assessment?

B. Articulating and developing

The second phase of the curriculum development process focuses on the content of the program and assessing students’ progress. The choices made in this phase are based on three key aspects: 1) What is the healthcare organisation’s and/or trainer’s vision on learning?; 2) What are the goals that need to be achieved?; 3) Which resources are available for implementing the program?

Program philosophy

A program’s concept and structure depends on the organisation’s, the trainer’s or the committee’s view on effective learning strategies and approaches. This influences the instructional methods used, the way training participants are addressed, and how assessments and evaluations are incorporated in the process. Completing this step will help to develop a justification and framework for upcoming work in the curriculum development process.

Questions that can be addressed as part of this step are:

  • Why learn about cybersecurity or related topics?
  • What are the organisation’s or trainer’s core beliefs about learning cybersecurity and/or related topics?
  • How will assessments be used to improve the program and student learning?
  • What is the curriculum striving to reflect?

Defining program level, structure and goals

This step concerns itself more with the goals and structure of the curriculum. A curriculum can consist of multiple courses that together lead to achieving one or several learning goals, or the choice can be made to have a single training moment in which fewer and/or lower level goals are reached. Part of this step is to determine the goals that will be aimed to reach and how much time should be dedicated to this.

If the curriculum covers a longer running program with multiple trainings or training moments, it will help to define overarching goals that can be divided over the individual courses and sessions.

The goals that will be formulated in this step should be linked to the healthcare organisation’s goals and the issues and challenges that were defined in the planning phase. Key questions for this step are:

  • What are the goals that this training should help to achieve?
  • How much time is needed to achieve the goals?
  • Is the training part of a longer running training program?

Defining the target audience

The same program cannot be used for every staff member group within a particular healthcare organisation. For instance, top level and management staff is addressed differently than medical staff, and members of the IT department are likely to have a different baseline level of knowledge than those who work in facility management. Knowing who will be participating in the courses is a core task for every iteration of curriculum development and this step should not be skipped.

Questions for this step of the development process can be:

  • Who is the program targeted at?
  • Which baseline level of knowledge and/or experience does the intended audience have?
  • How receptive of the content of the training is the audience likely to be?
  • What are the legal requirements in terms of cybersecurity training for the intended audience?

Some of these questions may have already been answered as part of the third step in the planning phase, so it is useful to review the outcomes of that step and specify them further.

Developing course level and course objectives

This step entails specifying the program goals into specific course objectives (see the first step of this phase) that match the target audience (see the second step of this phase). The objectives are most useful if they are formulated in a way that they are achievable. The most commonly used method for this is applying the SMART acronym. SMART stands for: Specific, Measurable, Achievable, Relevant, Time bound (Mind Tools, n.d.).

  • Specific: the objective needs to be understandable and clear (to both trainers and participants).
  • Measurable: the progress for each objective should be trackable in a way.
  • Achievable: objectives need to be challenging, but also within reach of training participants.
  • Relevant: the objectives should match the needs of the target population and align with the healthcare organisation’s goals.
  • Time-bound: formulating a deadline for objectives creates focus and purpose.

Identifying instruction methods and resource materials

When it is clear what the goals, target audience and the more specific objectives are, a closer look can be taken at the actual design of the course. Sometimes the healthcare organisation or trainer has existing materials and programs that can be reused. However, the trainer or committee should assess whether the existing material fits with the goals and objectives that are formulated in earlier steps.

Aspects such as training group size, available time, and complexity of the material impact the choices made in methods and materials. Based on interviews held as part of the SecureHospitals.eu project, trainings are more effective when they incorporate the following aspects (SecureHospitals.eu, 2019):

  • Use real-life cases to show real problems, consequences and solutions
  • Use interactive instruction methods to increase engagement of training participants
  • Relate to the experiences of the target audience to keep the training relevant and interesting to the training participants.
  • Use positive language and refrain from instilling fear or creating negative experiences.
  • Use language that training participants are used to and understand, but
  • Do not avoid fundamental concepts. These help to create a better understanding of the current trends and challenges.

Selecting or developing assessments to measure participant progress

There are many ways in which participant progress can be tracked and assessed. The most suitable instrument depends on the program philosophy, overarching goals, and specific training objectives. In courses that focus on knowledge transfer, a knowledge test is more suitable. For trainings where a lot of peer interaction is required, peer to peer evaluation can be an excellent match. As such, based on the curriculum goals and specific objectives the best assessment methods and instruments can be selected. There are several ways to test the progress participants make during the course, either in solo or group format:

  • Knowledge tests (open questions or multiple choice)
  • Writing assignments (essays, project work)
  • Presentations or demonstrations
  • Specific topics: phishing tests
  • Practical or project work
  • Self-evaluation
  • Peer evaluation

Pre-existing assessment instruments from other training programs or courses can be adjusted to fit the newly developed training. The following questions are relevant when considering assessments and assessment instruments:

  • Will participant progress be tracked and assessed?
  • To what end will the progress be tracked and assessed?
  • How can the progress best be observed or measured?
  • What can the healthcare organisation do with these results?
  • What consequences are attached to failing to participate?
  • What consequences are attached to failing the assessments? (Does this lead to have legal/liability issues?)
  • Is the privacy of training participants protected?
  • Are training participants aware of what is being done with the training and assessment results?

C. Implementing

After developing the program or course, the training can be implemented. The trainer, or trainers, need to familiarise themselves with the new curriculum, the program or the individual course. If a trainer does not have the required knowledge in terms of content or skills in terms of instruction methods, they need sufficient time to acquire this, or hiring an external party should be considered.

First responses to the program

If there are sufficient time and resources available, a pilot training can be conducted to assess the response of a select group of the target audience. A pilot training is imperative for a program where the assessments of participant progress can have consequences for the participants, or where the evaluation is important to specific goals of the healthcare organisations.

If there is no time for a pilot training or if there is no intention to do one, it may still be helpful to plan a meeting with the committee, with the CISO, Information security offices, department heads, and/or representatives for the target audience to discuss the training and projected outcomes of the training

The first results of the pilot training and/or the meeting can be used to adjust the developed training program before it will be fully implemented.

Implementing the program

The first stage of implementation is focused on the timing of the program or the course: when will the course run? And will there be follow-up sessions or repeat trainings?

Another important question is when and where the training will be held, and which materials are necessary. This is partly addressed in Phase A (Planning) and Phase B (Articulating and Developing). This step is more focused on whether the training can realistically be conducted at the healthcare organisation or whether an external location should be used.

The trainer who will actually conduct the training, whether they are in direct service of the healthcare organisation or external to the healthcare organisation, needs to familiarise themselves with the training content, instruction methods. If they are external to the healthcare organisation, they may need to familiarise themselves with the particular culture or policies of the organisation.

D. Evaluating

During the development of the curriculum, training program or course, the developer or committee should already consider the evaluation possibilities and instruments. Additionally, it should be determined how the evaluation will be used for future implementations of the training.

Evaluating and updating the curriculum, training program or course

After the training program or course has run, it can be evaluated by the trainer or committee. Several types of information can be included in the evaluation process:

  • Assessment results from the training program or course;
  • Participant feedback (open answers or multiple-choice forms);
  • Experiences from the trainer or trainers; and
  • Reports from the helpdesk or incident response team of the healthcare organisation

The evaluation should contain the possibility for training participants and trainers to reflect on the content of the training, the instruction methods, the assessments, the location, timing, and effort it cost participants. Based on the outcomes of the evaluation, the training can remain as is, be adjusted or improved on several areas, or determined to be ineffective and redeveloped (see next, and final, step).

The following questions should be considered in the evaluation step:

  • Will the program or individual course be evaluated?
  • How will the program or the individual course be evaluated?
  • Will evaluation (only) happen directly after the last training session?
  • Will there be an evaluation moment at a later time – after a month – after the last session?
  • How will the results of the evaluation be used?
  • Is the privacy of training participants protected in the evaluation process?

Determining the success of the curriculum, training program or course

Based on the evaluation outcomes, the success of the training program or individual training session can be determined. The criteria for evaluation, or indicators of success, should have already been considered in the Planning phase.

Questions that are relevant for this step in the development process are: 

  • Are overall program goals reached?
  • Are course objectives reached?
  • How was the training received by training participants, trainers, management, and the board?
  •  Is the program or individual training reusable? Why or why not?
  • What revision are necessary for potential future implementations of the program or course?

There are no specific criteria provided that trainers can use to measure the impact of their training in this guide, because the criteria will differ from training to training. Some training programs might have the aim to increase internal communication about cybersecurity challenges, while other trainings are more focused toward an improved cybersecurity incident response. The trainer will have to formulate adequate criteria for the whole of the curriculum and for each individual training session.

When going through the development process, consider completing the SecureHospitals.eu Curriculum Wizard for tailored advice!

Literature

Ct.gov. (2001). A Guide to Curriculum Development: Purposes, Practices, Procedures (pp. 1–11). https://portal.ct.gov/-/media/SDE/Health-Education/curguide_generic.pdf

Gould, D., Kelly, D., White, I., & Chidgey, J. (2004). Training needs analysis. A literature review and reappraisal. International Journal of Nursing Studies, 41(5), 471–486. https://doi.org/10.1016/j.ijnurstu.2003.12.003

Mind Tools. (n.d.). SMART Goals: – How to Make Your Goals Achievable. Retrieved December 4, 2019, from http://www.mindtools.com/pages/article/smart-goals.htm

SecureHospitals.eu. (n.d.). Assessing training needs. SecureHospitals.Eu. Retrieved February 3, 2020, from https://www.securehospitals.eu/assessing-training-needs/

SecureHospitals.eu. (2019). Trainer interviews report. https://project.securehospitals.eu/deliverables/