CCIE Pursuit Blog

February 11, 2009

Open-Ended Question Grading: All or Nothing

Filed under: Cisco,Cisco Certification — cciepursuit @ 3:28 pm
Tags: , , , ,

In a discussion on the Cisco Networking Professionals Connection a candidate recently failed his lab.  He received a 0% grade on the new “open-ended question” portion of the lab even though he was sure that he got at least 2 of the 4 questions correct.  If he got two of the four questions correct then he should have received a 50% mark in that category, right?  Well, it turns out that this is NOT the case:

Replied by: lohaver – Marketing Programs Manager, CISCO SYSTEMS – Feb 10, 2009, 4:30pm PST

Hello Nuno,

I work on the CCIE team. The short answer section is graded on an “all or nothing” basis. If you answer the minimum number correct you will get 100%. If you don’t achieve the minimum, your score shows as 0%. It is indeed possible that you answered two questions correctly.

Lora O’Haver
Learning @Cisco

What the fuck???  Honestly, WHAT THE FUCK?!?!?  What is the “minimum number correct”?  I would have to assume that in this case it’s three.  Lora states that he could have correctly answered two and still not passed the section.  If it were four then she would (well, ‘should’) have said that you need to answer all of the questions correctly.  This “minimum number” gets more interesting in that some candidates have received 4 questions while others have received 5 questions.

This gets even more interesting.  From the responses on this thread it looks like the questions are worth a total of 21 points and the remainder of the lab is worth 79 points.  That means that you need to pass the question portion of the lab in order to pass the entire lab as the passing grade is 80 points and the most that you can score on the lab portion is 79 points.

I’ve fully supported the addition of the questions to the lab, but this new scoring “revelation” pisses me off.  IF the questions are worth 21 points towards your total score and they are graded on an “all or nothing” method based on meeting some undisclosed “minimum number of correct responses” then this is more than just a minor addition to the lab and Cisco really owes it to the candidates forking over $1400 to explain this better.  This “all or nothing” scoring is especially important for candidates to be aware of because if they aren’t aware of it, then they may be likely to request a reread of their exam.  If you get a 0% on a section that you are sure that you had at least half of the questions correct, then you’re most likely going to assume that the lab was graded incorrectly and request a reread ($250).

Advertisements

19 Comments »

  1. It’s fucking ridiculous.

    Glad mine is out the way and I don’t need to deal with this shit…

    Comment by Matt Hill — February 11, 2009 @ 3:51 pm | Reply

  2. As if it isn’t hard enough going to the lab. I am all for integrity and challenge but this is a tad bit ridiculous. All we can do is prepare as best as we can, and hope for the best.

    Comment by navfett — February 11, 2009 @ 4:32 pm | Reply

  3. It is honestly enough to say “Why am I doing this?” If the CCIE program is that compromised that you have to hammer the legitimate people that spend thousands on preparation is it really worth it to them? I am all good with failing the lab if I couldn’t handle the lab itself, but if I only hit 3 of 4 of the open ended questions with answers they are looking for?

    Again though if the question are no problem to answer if you prepped enough…but there is not enough transparency at this point from Cisco…

    Comment by cciejourney — February 11, 2009 @ 9:29 pm | Reply

  4. Plus I am already tired and crabby from studying for my written again :P… I don’t want to have to try to memorize all 900 pages of theory of this thing as well AGAIN 😀

    Comment by cciejourney — February 11, 2009 @ 9:59 pm | Reply

  5. Not good new @ all.
    Has there been any word that Cisco may introduce this to other CCIE labs?

    Comment by skybaba — February 12, 2009 @ 6:33 am | Reply

  6. So they’ve marginalized the lab in the interest of as few as three open-ended questions? What’s the motivation there? Seems like Cisco is just trying to make the exam more difficult to pass without really improving its quality.

    Comment by stretch — February 12, 2009 @ 8:56 am | Reply

  7. Looking at the bright side , if the questions are not too difficult and you pass them, you have too otherwise ,for what I understand, you fail the exam
    then you get those 21 points then 80-21= 59 is what you have to get out of the lab.Not a bad deal, how many people have in past failed the lab because of 10 15 points ? Even more an incentive to read the Cisco DOC or reread the
    CCIE written book great investment of our time anyway.

    Comment by Giorgio P. — February 12, 2009 @ 1:24 pm | Reply

  8. Girorgio, I don’t know if we can assume that the open ended questions will make the lab part much easier to pass in terms of getting your 80 overall points. It depends on how Cisco compensated for the points. I think it’s safe to say that they aren’t making the content easier. So on a best case scenario, where the content wasn’t harder but the points per section were smaller (adding up to 79 instead of 100), the open ended questions won’t buy much.

    The old test required you to get 80% of the lab points (80/100). The new test requires you to get about 75% of the lab points (59/79). So at best, that’s a 5 point advantage. But if you take away the time in answering the open ended questions, you end up with less time to complete the lab portion. So the advantage shrinks a little bit.

    Who knows if they give you fewer tasks to do. If so, that makes things easier. I think that’s one of the problems here. Not enough communication on Cisco’s part makes things confusing for us.

    Comment by jrensink78 — February 13, 2009 @ 8:12 am | Reply

  9. So does this mean we only need 60 points in the lab section?

    Ton

    Comment by Tony — February 13, 2009 @ 11:46 am | Reply

  10. What worries me is that a pass appears to be dependent on these open-ended questions. Your entire lab, 8 hours, thousands of $ and months of study are dependent on 4 or 5 open-ended questions. How much should I write? How in depth should I go? Should I have taken a creative writing course as part of my preparation? Not to mention that as they are open-ended I assume grading is fully at the discretion of the grader unlike the rest of the exam where you either meet the criteria or don’t, you either have connectivity or don’t.

    Additionally by taking this course of action Cisco has essentially admitted that the CCIE is completely compromised and has been so for some time. As such one has to question the value of this certification as undoubtedly if the exam is this compromised there is nothing stopping cheaters from getting these questions in advance and preparing they’re answers before the lab. All these factors combined leads me to a one word conclusion – Juniper.

    Comment by Dean — February 13, 2009 @ 2:26 pm | Reply

  11. Just one more thought, as I have this bee in my bonnet, couldn’t Cisco just allow us to go to our local Pearson Vue test centre pay say $50 and do our 4 open-ended questions there instead of juicing us for the full lab fee not to mention the cost of travel, accommodation, time off work etc.

    Better still why don’t they just make the written un-dumpable. Surely they can create a question bank comprising of the thousands of questions they must have given over the past X years and randomly generate a somewhat unique written exam for each candidate. I don’t know; have to say I’m getting seriously disillusioned with Cisco Certification.

    Comment by Dean — February 13, 2009 @ 2:49 pm | Reply

  12. Well I do not think the difficulty of the lab will change only because they have these questions. It would be interesting to find out if the points available for the lab,not the questions, are still 100 then having to get 60 or less and “buying” that with half an hour of your time seems still a good deal.

    Comment by Giorgio P. — February 14, 2009 @ 2:20 am | Reply

  13. Personally I think that you’re misinterpeting the scores.

    Open ended Questions are propably only worth about 10 points. They look like they are like 21 points but I think this is just because if you fail those, Cisco has put them into the system like they were 21 points. Otherwise the score is not consistent with the CCIE Score report. I mean that if the points for Open Ended are like 10 point’s and you have 100% for each section, that would mark as passed if they did not weight the scores. Now if you fail the the open ended part but configure 100% on all other section the score sheet will show it as fail by 21 points due rescaling.

    Comment by GoingtoBrussels — February 14, 2009 @ 12:08 pm | Reply

  14. Where are these 10 points coming from ? Nobody ,included Cisco mentioned that. What we know is that from now on the so called lab exam is not a lab exam only any more. There is a written part ,albeit small, and a practical one the lab . Both of them must be passed , we do not know for certain how the points are distributed , but we know that the grand total must be 80.
    It is only common sense to assume that the lab score will need to be less than 80.For what I heard from different forum in the past it was common enough for candidates to be short of 10-20 points and fail the lab. Still in my opinion answering some,not easy but not too difficult, questions already included in the blue print anyway and decreasing the lab score of some 10 -20
    points ? is still a good deal but will see on the day.

    Comment by Giorgio P. — February 16, 2009 @ 4:13 am | Reply

  15. Cisco stated you need to pass the open ended questions & the lab.
    I think you should see the grading of both seperatly but if you fail 1 of them you fail them all.
    Who says the open ended questions are counted in the total score?
    They just could be questions to see if you have the level before you “can” succeed in the lab.

    Comment by chibi — February 16, 2009 @ 5:33 am | Reply

  16. […] Cisco, Cisco Certification I received quite a few excellent comments on my recent “Open-Ended Question Grading: All or Nothing” post.  Many were as confused, disappointed, and angered as I was about finding out that […]

    Pingback by Comments and Speculation on Open-ended Lab Questions « CCIE Pursuit Blog — February 17, 2009 @ 2:51 pm | Reply

  17. I hope someone Cisco realize what they are doing. I will not be wasting my money on the lab test any more. I might reconsider if change the format and stop making us pay $1400 to answer 4 or 5 questions.
    Check these links

    http://ieoc.com/forums/p/5794/20619.aspx#20619

    http://www.networkworld.com/community/node/38697

    Comment by Andy — April 9, 2009 @ 1:08 pm | Reply

  18. Well, I’m going for my third attempt this week, and these questions are a concern for me. But I’ve spoken with the proctors on grading practices, since the second attempt was the week they put these questions out. From what I can gather, the point system isn’t straight up anyway. They weight the switching and routing sections, and then see if you have enough “extra” points to pass. It was explained thus to me by several proctors, and I cannot see this as an NDA issue.

    It is a R/S test. They want to see if you are proficient in these categories, then if you can use the Doc CD to do specific tasks in other areas. You aren’t going to pass if you really mess up the Routing or Switching section, even if you accumulate more than 80 pts and ace the exam. And you shouldn’t anyway.

    I’ve seen the questions, and they are not anything anyone who has TRULY mastered the material (knowledge and why, not just commands) shouldn’t be able to answer. I had to go back and acknowledge some deficiencies which was HARD. I scored 100% on Switching, 89% in routing, and 100% in BGP on my second attempt, but failed the questions. I’m just more determined than ever now. I’ll truly be an expert by the time I pass – my coworkers think I am now. Especially since I went back over theory in depth these past few months. I could teach it..

    Comment by Another fool — April 13, 2009 @ 12:01 pm | Reply

  19. I attempted my first attempt on March and also got 0% on the Open ended question. Personally I think the open ended questions are an advantage if you really lucky and got the questions that you have well prepared for, since there is a large pool of questions. I actually go back to ccie support to clarify why did I failed this section and was told that I only managed to score less than 3 correct questions. They can’t discuss the exam content with me as this is bound by NDA. I agree with some in this forum as there is less transparency in this case. I believe grading an open ended questions is very subjective and NDA is another hurdle 🙂

    But if you managed to pass this section, then it’s an advantage to you, since you only need to gain another 59 points to pass the lab exam. One of the ccie candidate whom I spoke to agree that the lab portion is easier than the previous before the introduction of open ended questions. Anyway he gained his ccie number after the introduction of this open ended question which is after several attempts :p

    Anyway, I personally think if you want be a CiscoCIE expert, you need to be bound by whatever certification requirements released by Cisco. No complain about it :p

    Comment by FailedAtFirstAttempt — May 1, 2009 @ 6:20 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: