|
Walter & Group....
[GH] I respect the fact that different candidates look at the exam process in different ways. Some are so confident that they relish the challenge. Others may be very competent and capable of passing yet are turned off by the very fact that may face any sort of testing.
Craig Buckbee speaks to this. He is not alone :
gordy,
Dave Barron writes, " I am not sure how they will be able to perform in front of a group of students if they can not perform in front of the examiners."
i have heard this comment many times.
a group of examiners is VASTLY different than your typical saturday morning group of Orvis 101 students.
to me it really shows the difference between those that test well, are easy enough going about it, and those like myself
that can get physically sick over the thought of taking any kind of exam.
... off to fish the spinner fall on the Delaware.
craig
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[GH] From Gary Eaton :
Gordy,
There remains a lack of discipline in this discussion and in examinations. There are two different assessments going on in what is now termed the ORAL portion of the MCI exam. There is NO ORAL EXAM IN THE CCI! There IS a cursory written test for CCI.
TWO evaluations grouped into a single, so-called, oral exam (information & presentation), confound both the standards and the objectivity of the process. INFORMATION base most often preserves integrity when assessed as a properly formatted written process. The moment an examiner deviates from the published question format, objectivity is compromised. Typically diverse question formats serve to diversify the difficulty and the monotony of prolonged sit-down testing.
- Multiple choice questions
- Matching exercises
- True / False selections
- Graphical interpretation choices
- Data simile
- Definitions and vocabulary
ALL lend themselves very well to two-dimensional testing on computer or paper. Deeper assessment of knowledge and complex application of information can remain totally objectively discerned through comprehensive written exams.
Through much professional evaluation, I have become more familiar with question vetting, question evaluation within exams, test security, and tiered difficulty question distribution. Properly done, the process is superior to panel evaluations of pet questions.
Having more choices than matches may make matching more discrete. Creating matching choices that are very close in content definitely reduces percentage of marginal candidates slipping through.
Similar in effect is the multiple-multiple format where five choices are given and the examinees must choose between "a -1 & 2; b - 1, 2, & 3; c - 4 only; d - all of the above; e - none of the above". Written formats can be both complete knowledge assessments and totally objective.
Longer True and False formats involving partly true and mostly true vs all-true seem too subtle for the type of data we intend to extract. These processes exist, well-engineered by professional test development specialists.
I see tremendous real-value to making a strong written exam a pre-requisite to appearing for performance and oral presentation evaluations. Indeed, costs of producing a test bank of thousands of potential questions from which to formulate knowledge assessment has functioned extremely well for more demanding professions across widely varied fields.
The second portion lumped-into the ORALS is the explanation and teaching of skills assessment. This should be removed from the knowledge base assessment provided the written criteria are rigorously designed and examined. Combining performance with teaching skills and verbal communication of instructional ability seems inevitable.
The problems remain the intrusion of personalities and personal preferences into the nebulous arenas of performance criteria. There are myriad examples in my MCI prep notes from debriefing candidates, examining, and attending workshops. A simple one is the definition of the shape of an aerial mend that is satisfactory in Task 4. Some examiners clearly want a shark's tooth shape for 15-ft at least three-feet deviation from the straight-line base but no wider than 4-ft at the base. Others want a smooth arc with the apex directly at the prescribed.distance.
Again, prescribed line layout diagrams authorized by the CBOG would tend to minimize such subjectivity. Until these arenas of variance are definitively addressed, continual calls for change of format may be expected.
Finally, my experience suggests the CICP opens the door to expensive litigation by not having criteria-driven substantiation for every question, every scenario, and every skill set evaluated in the certification process. Wild card examiners, peculiar "favorite" questions of individual evaluators, and unclear expectations do not co-exist well with assessing fees for examination. Inconsistency embellished by disagreement provides fertile cracks for discordant battle.
Gary Eaton, MCI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[GH] Gary,
It's obvious that you have given this a great deal of thought !
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[GH] From Michael Jones :
Gordy~
I thing that Ckling nails my sentiment with this answer:
The written exam covers the breadth and depth of an MCI's knowledge in a structured and more objective manner. The oral exam to assess attitude, aptitude,more fly fishing/casting knowledge, communication , teaching ability,ambitions and future plans etc.( is this a person we want or capable of carrying out FFF objectives?)
The performance part for casting and teaching abilities, ie able to walk the talk.
I do believe there can be a balance struck that removes the subjective influence from a biased examiner, and arriving at the qualifications of a candidate to teach and explain, on their feet. I find it interesting that the replies from current Masters & BOG folks ignore the ability to perform robust oral inquiry during the practical component, and complain that a written test limits their impression of a candidates knowledge (I wonder what the Bar Association in any given state would have to say about that sentiment).
If you were to write a written exam question, what would be a good question that would provide an answer capable of passing, or failing from a written answer?
Here's one: Give 3 fly line core characteristics, and briefly explain how these affect a line's characteristics.
I bet you (Gordy) could write a 250 question written exam, with unbiased MCI questions before lunch!
Once you have a passing written exam in-hand, begin the oral/practical to qualify the candidate from a teaching & presenter perspective.
MJ
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[GH] Michael,
Yes..... I probably could. However, the tweaking of exact wording to satisfy everyone in an exam committee would take the better part of at least a year !
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|