28814 members and growing – the largest networking group in the maritime industry!

LoginJoin

Friday, November 27, 2020

Maritime Logistics Professional

One of The Most Common Mistakes in Maritime eLearning

Posted to Maritime Training Issues with Murray Goldberg (by on October 14, 2013

It is easy to make mistakes in the implementation of maritime eLearning. While it is true that making them can be very damaging to an eLearning program, they are all pretty easily avoided with just a bit of thought and knowledge. This is the first in a short series of articles that covers some of the most damaging, yet some of the most easily avoided mistakes when implementing eLearning programs in the maritime industry.


Maritime Training: The full library of maritime training articles can be found here.

Blog Notifications: For the latest maritime training articles, visit our company blog here. You can receive notifications of new articles on our company blog by following the blog.

Maritime Mentoring: International Maritime Mentoring Community - Find a Mentor, Be a Mentor

One of The Most Common Mistakes in Maritime eLearning


As in any human endeavor, it is easy to make mistakes in the implementation of maritime eLearning. Being aware of the potential will help in terms of avoidance. In my work over the last 6 or so years implementing eLearning in the maritime industry (and in the 10 years before then as an eLearning user, researcher, developer and business owner), I have become very aware of many of the more common implementation mistakes - many of them made by me. While it is true that making them can be very damaging to an eLearning program, they are all pretty easily avoided with just a bit of thought and knowledge. Thus, this short series of articles covers some of the most damaging, yet some of the most easily avoided mistakes when implementing eLearning programs in the maritime industry.

This first article could be very short - even just one sentence. That is, of course, if someone else were writing it. As a former academic I am not sure I've ever explained anything in only one sentence (those who know me see this immediately as a gross understatement). My argument is that there are many details and intricacies that are missed with that kind of brevity. But let me start this article with the sentence I would use if I were forced to write this using only one sentence. Here it is:


"If you deliver exams to assess the knowledge of your seafarers, make absolutely sure that someone in a position of authority is supervising the trainees as they do their exams."

I'm pretty sure I'd pass out if I were forced to stop there. So I won't. Instead, I'll explain why I say the above, and explain why it is important. But I'll try to be brief.

Before I get started, I'll mention that the next article will cover a related issue which I consider to be the second most common mistake in maritime eLearning. If you would like to receive an e-mail notification when that (and subsequent) articles are available, please sign up here (unless you've already done so) and I will add you to the list.

A bad practice - common in this industry

eLearning is becoming incredibly common in the maritime industry. This is great - at least when it is done well. And it is not hard to do well. All that is required is a little bit of knowledge and planning. Something that has arisen alongside the implementation of eLearning in this industry is the delivery of computer-delivered multiple-choice assessments. This is good. Really good in fact (again - if done properly) because when used to assess knowledge, they offer critical advantages that cannot be had with paper-based assessments. I've discussed those advantages in previous articles - but as a quick reminder they include the ability to randomize tests (so no two candidates receive the same exam) and - most importantly - the ability to derive metrics and analysis not possible with paper-based exams. These metrics and the resulting analysis can alert organizations to problems before they become performance or safety issues.

Again - all of this is hugely positive if the exams are delivered properly. The problem is, they are often not delivered properly. And the most common issue I have seen, which is also the one I get asked about the most, is the delivery of multiple-choice exams without supervision. That is - letting the trainee do an exam at the end of their computer-based training while alone. No supervision. While this practice is not unique to the maritime industry, it seems much more common here than I've experienced elsewhere.

I believe the main reason this has become a common practice is because the advent of web-based or CBT (cd-based computer) training has allowed trainees to learn independently - without supervision. For many training organizations they have set up their training this way and then simply extended the practice to allow trainees to also perform their assessments independently. The same organization which would never have considered sending someone home with a paper-based multiple-choice exam seems perfectly happy to allow the same trainee to perform an on-line multiple-choice exam at home (or on-board without supervision). In most environments - including where I come from in higher education, this would be unheard of except in special circumstances.

Is it always bad to deliver unsupervised exams?

The answer to this question is "no" - it is not always bad to deliver unsupervised exams. But it usually is. So when is it OK to do so?

The first circumstance in which unsupervised exams make perfect sense is when the exams are formative exams, not summative exams. What's the difference? Formative exams are essentially self-tests. They exist for the benefit of the trainee, not the assessor. They are there to allow trainees to periodically test their own knowledge to determine whether they are learning the material to the degree expected by the trainer. They help the trainee know when they are ready to do the official exam (or the summative assessment). Because the trainee is learning independently, they need to be able to take these self-tests whenever they feel the time is right. Therefore the exams should be taken independently. And since the results are for their own benefit, and because they are the only ones likely to see the score, there is no incentive for them to cheat. In fact, cheating defeats the purpose of the formative exam.

The other common scenario where exams are often performed without supervision is in the case of in-depth written exams used to test reasoning rather than knowledge retention. This is a practice that is sometimes seen in higher education. While this is likely still not an optimal practice when it comes to relying on the results, there are reasons for it to be done this way. One reason is because it allows the trainees to spend more time thinking about their work - often required for tests of reason. These kinds of exams are more difficult to do in a time-limited, high pressure, classroom setting. Another reason to allow these kinds of exams to be done independently is because it is easier to detect cheating for these than it is on multiple-choice exams. There is a human grading this kind of exam and that person is likely to be more familiar with the trainee's abilities and writing style - making it more difficult for the trainee to hand in work done by someone else. So here, the benefits of allowing the exam to be done independently can be considered to outweigh the reduced ability to rely on the integrity of the results. Even so, using this kind of exam in isolation and relying on the results would be a mistake.

Both of these scenarios are 100% applicable to the maritime industry.

Is there a way around having to supervise?

This is a question I often hear because it is expensive to have a remote trainee travel to the training center in order to write an exam (or to have a trainer travel to the trainee). In fact, there have been many efforts aimed at allowing students to take exams without supervision while ensuring that the answers are their own. It is my personal opinion that none of them work. They all simply present an entertaining challenge to the would-be cheater.

For example, I have seen systems where a camera affixed to the computer sends video of the student to a remote supervisor, or records the video in order to encourage the student to perform honestly. This scheme cannot capture what is happening outside the field of view of the camera - such as a cheat-sheet sitting on the desk or a knowledgeable helper off to the side.

There are also biometric solutions such as fingerprint readers aimed at verifying the identity of the test-taker. These do ensure that the actual student is present for the exam, but do nothing to ensure that the answers entered are his or hers.

In short, there are techniques that can be used to make it more difficult to cheat on unsupervised exams, but I've never seen one I believe to be fully effective.

So - what can we do about this?

In my view there is only one solution to this problem. If you care that the exam being given provides an accurate representation of the knowledge of the trainee, then you must have a trusted person in a position of authority supervise the taking of the exam. It need not be a trainer - any trusted person will do. Only this way can you ensure the integrity of the assessment results.

This is not a new problem, nor is it unique to electronic assessments. Universities and colleges have been offering distance education for eons. They have largely arrived at the same conclusion and as a result, test centers have been set up to solve the problem. Students visit a local test center where the university sends the exam and the test center employee (often called a proctor or invigilator) supervises the exam. Some universities have reciprocal agreements to proctor one another's exams for distance students. Both of these techniques could be applied in the maritime industry.

So - the bottom line is "don't do it". If you allow trainees to take assessments without supervision then the results cannot be trusted. If you care about the integrity of the results, you must have supervision.

The second most common mistake in maritime eLearning

This brings me to a related problem which I believe to be the second most common mistake in maritime eLearning. It also has to do with assessments. However, as I said in the opening of this article, I would try to be brief. I realize I have not succeeded,  but rather than make the problem worse, I will leave this topic to the next article.

If you would like to receive an e-mail notification when that article (and subsequent articles) are available and have not already done so, please sign up here and I will add you to the list.

Until then, thanks for reading and keep safe!

# # #

About The Author:

Murray Goldberg is the founder and President of Marine Learning Systems (www.marinels.com), the creator of MarineLMS - the learning management system designed specifically for maritime industry training. Murray began research in eLearning in 1995 as a faculty member of Computer Science at the University of British Columbia. He went on to create WebCT, the world’s first commercially successful LMS for higher education; serving 14 million students in 80 countries. Murray has won over a dozen University, National and International awards for teaching excellence and his pioneering contributions to the field of educational technology. Now, in Marine Learning Systems, Murray is hoping to play a part in advancing the art and science of learning in the maritime industry.

Maritime Training: The full library of maritime training articles can be found here.

Blog Notifications: For the latest maritime training articles, visit our company blog here. You can receive notifications of new articles on our company blog by following the blog.

Maritime Mentoring: International Maritime Mentoring Community - Find a Mentor, Be a Mentor