28843 members and growing – the largest networking group in the maritime industry!


Friday, February 26, 2021

Maritime Logistics Professional

Improving Skills Assessment

Posted to Murray Goldberg's Blog (by on April 30, 2018

Murray Goldberg

In safety-critical environments such as the maritime industry, it is of utmost importance that an organization can ensure crew competence. And while there are many factors to an individual’s ability to carry out their job properly (such as their knowledge and behavior), a major component of seafarer competence is their practical skills. An operator must be able to objectively teach and assess the ability of their crew to perform the skills needed to safely and efficiently do their jobs. 

What are the approaches that the maritime industry is using to accomplish this? How effective are our current tools in teaching and assessing skills? And are there areas that we can improve on?

Current Practices
Traditional seafarer training has always placed heavy emphasis on practical skills acquisition, particularly on the teaching side. Operational tasks are typically taught with a combination of academy or classroom training, on-the-job training or job-shadowing, and comprehensive hands-on training and practice in real and simulated environments. To keep skills up to date, operators may conduct periodic drills and send their crew in for refresher training. In addition, the industry is continuously improving the way practical mariner skills are being taught. From gamification to virtual reality, operators and academies advance simulation training year after year.

The picture is not as clear when it comes to skills assessment. Structured methods and tools to measure skills in a detailed manner are hard to find. The matter is further complicated with the dynamic, team-based scenarios that are often present in maritime drills and other safety-critical assessment activities. 

Best practices for skills assessment typically involves having a trained assessor observe, and then assess, a practical demonstration of skills in a simulated or real-life environment at an academy or simulation center. More commonly, skills assessment consists of little more than a record of who participated in an exercise and some high-level debriefing notes. This practice can lead to highly variable and often subjective assessments.

How can we improve?
From more objectivity to better tracking of results, the assessment of skills is one area where the maritime industry can improve upon. Current practices waste a tremendous opportunity for competency assessment. Operators have very little knowledge as to the details of skill performance, both at an individual and at a team level. And if an operator does not have deep insight into how their crew can perform tasks, how can they ensure crew competence for safe operations?

1. Create opportunities for assessment.
Turn any demonstration of skill by teams or individuals into an opportunity for assessment. This can include on-board drills, simulation exercises, bridge team performance exercises and more. All of these are demonstrations of skills which can be assessed. Make sure you have a toolset that can facilitate comprehensive and objective real-time assessment. 

2. Ensure more objective and comprehensive skills assessment.
Go beyond recording participation and debriefing notes. The use of a rubric or a structured assessment framework, can improve objectivity and give more detailed insights into crew abilities. A rubric is a written guide for assigning points or grades. For example, an assessor can use a rubric to give points based on how fast a mariner dons a firefighting suit. A structured framework can offer a set of simple performance indicators that an assessor should be looking for. For example, in a collision exercise, indicators could include calling out the event, adjusting propulsion, notifying the master, etc. The assessor would then indicate whether they observed the indicator and who performed it. Rubrics and performance indicators remove subjectivity from the assessor and allow them to easily grade in a fast-paced situation.  

3. Record and track all assessment information.
Recording all assessment data (i.e. individual grades or scores on exercises, success rates, participation, etc.) allows operators to track performance over time, facilitating a process of continuous improvement. This is where technology can be your friend. While it is possible for an assessor to record and track all assessments with a pen and paper, the process can be cumbersome for real-time assessment and transferring the results onto a system for analysis (for e.g. Excel, or other data analysis software) is inefficient. With a software tool, data is easily recorded, and there are applications that exist today that will automatically generate reports which provide detail on how the team performed as a whole, and how each individual contributed to that performance. 

Actionable Data
Once you begin conducting and tracking objective skills assessment, a wealth of useful and actionable data is now available. These data points, at its most basic, will show whether your crew (or individual mariner) is competent or not. Further analysis of data can provide insight into efficiency, what you are doing well and where there are skill gaps. From there, you can even use the data to reveal how different factors, such as training or team composition, can affect exercises. This gives you the opportunity to optimize and improve performance in a targeted manner. An operator will be able to track skills overtime, compare performance across teams and departments and ultimately reveal insights into crew skill proficiency.

Tags: training safety Education

Q4 2019  - Short Sea Shipping Ports