CCIE Pursuit Blog

July 24, 2008

Cisco Steps Up Exam Security

Filed under: Cisco,Cisco Certification — cciepursuit @ 2:32 pm
Tags: , , ,

This hit my inbox today (emphasis mine):

Cisco and Pearson VUE Launch Global Test Delivery Exam Security Enhancements
Cisco and its global testing provider, Pearson VUE, a business of Pearson Inc. are pleased to announce a series of security enhancements that will reinforce the integrity and value of its Career certification program.

The advanced security enhancements include the use of digital photographs for candidate-identity verification and forensic analysis of testing data. The new measures, to be implemented beginning on Aug. 1, will include:

  • Photo on Score Report and Web – On completion of a certification exam at the test center, candidates will receive preliminary score reports imprinted with their photos and unique authentication codes. The authentication code can be used to access a candidate’s official score online at Pearson VUE’s website usually within 72 hours of the examination. The online score report will also display the candidate’s photo. Candidates may share access to their online records with employers or other third parties.
  • Forensic Analysis – Exam results and other testing data will be continuously analyzed by forensic software to detect aberrant testing behavior and to flag suspect exams for further investigation.
  • Preliminary Score Report – All paper score reports will be preliminary, pending the results of forensic analysis, until official exam scores are posted to the Web usually within 72 hours of exam completion.Once the exam scores are official, candidates may use the authentication codes on their score reports to access the Pearson VUE website for score and photo verification.

These new exam security measures are part of Cisco’s overall strategy to protect the value and integrity of its certifications. Other measures include simulation-based testing, dynamically generated questionsand emulations to help ensure that Cisco certified networking professionals continue to have the knowledge, skills, and credentials to perform well on the job.

To find out more about Cisco Career Certifications access the Cisco Learning Network at www.cisco.com/go/learnnetspace.

It looks like Cisco is getting more serious about the integrity of its certification testing.  Associating a photo with your test result will help to stop proxy test takers (those who sit an exam under a different person’s name).  I’m not exactly sure what dynamically generated questions are, but my guess is that this is generating unique questions and answers based on changes in variables in a question.  For instance, you might memorize the answer to a CCNA subnetting question by reading a brain dump.  You may get the same question on the exam, but the IP address and subnet mask is changed as well as the order of the answers.  In that case you would need to actually know how to subnet rather than just know to click 176.18.242.0/18.

The most interesting aspect of the new security features is the inclusion of forensic analysis of certification testing.  Rick Gregory at TrainingIndustry.com has an excellent article that details some of the issues that Cisco’s new security features are meant to combat and how they they do it.  The article also expands on some of the techniques mentioned in the email and what will happen to test takers who are suspected of cheating.  Here is a description of forensic analysis (again, emphasis mine):

Exam Data Forensics

During a Cisco certification exam, each keystroke is logged and a record is created that includes the length of the test period, how much time was spent on each question, whether an answer was changed, how much time was spent on the second answer, etc. After the exam is completed, but before the results are processed, each exam session is analyzed by forensic software that analyzes the session against established behaviors and suspect exams are flagged for investigation.

“When you analyze a testing program the size of Cisco’s, you develop an extensive knowledge base of behaviors you expect to see at every level of the examination,” said Trask. “We’ve broken the population down by age, sex, education, country, etc., and we know what to expect in almost every instance. We’ve established norms for behavior for an individual taking an exam for the first time, for taking an exam a second time after failing initially. We know how the distractors should be performing. We’ve amassed a wealth of knowledge for every test question.”

When an exam is flagged, it is investigated by VUE psychologists and security teams. If they determine that there is a problem with the exam, the results are invalidated and the candidate is offered a free chance to retake the test. They are given a different form of the test and have the opportunity to validate that they understood the information and should have passed the test.

Program Data Forensics

Another layer of security examines a wide range of other program attributes to determine what is occurring within the program. “We’re comparing testing center to testing center, and within a center, we are comparing one administrator to another administrator to see if inconsistencies emerge,” Trask said. “We look at things like candidates who live in one country but test in another; we review financial information, credit card information; just a very wide range of information to see patterns and inconsistencies.”

—Read The Rest Here—

This sounds like the most powerful weapon to combat cheating.  I won’t pretend to know the details of a forensic algorithm, but if your zip though five CCNA subnetting questions in a matter of seconds, then it’s pretty obvious that you’re either fluent in binary or you are choosing answers that you’ve most likely seen before.  You may want to think twice before posting about getting 1000/1000 and finishing the exam in 15 minutes.  🙂

The last bit about using forensic analysis to monitor testing center results as well as individual test administrators will hopefully crack down on leaked exams. 

Gregory’s article mentions test takers who take the exam in order to harvest questions.  That may be effective, but I have to imagine that slipping some cash to a testing center administrator would be a more effective way of stealing exam questions.  There are cameras all over the testing center.  How hard would it be for a person working at a testing center to simply copy questions by reading a test taker’s screen?  It would be interesting to introduce variations on questions in certain test center’s exams only and see if those questions appear on brain dumps.

Anyhoo…hopefully this will start to cut down on the number of exam cheaters and bring a little bit of the former prestige associated with the CCxA and CCxP certifications.  I also hope that this will mean that the tests themselves will concentrate more on core topics and less on out of left field questions meant to make the exams more difficult.

Advertisements

3 Comments »

  1. All this seems like lip service by Cisco, considering the very noticeable absence of any legal action toward braindump companies. If they were serious about maintaining certification integrity, I’m sure they could spare a few lawyers to put TK and friends out of business.

    Comment by stretch — July 25, 2008 @ 12:03 am | Reply

  2. The only thing that is going to stop producing paper certified people is to cancel paper testing.

    Multiple choice questions have an inherent weakness, they can be guessed by good test taking tactics.

    CCIE/RHCE/and the coming Microsoft Master certifications are the way to go!

    Comment by Branko Santo — July 26, 2008 @ 11:33 am | Reply

  3. Lip service it is. If Cisco/Microsoft really actually cared about the integrity of the exam process, they would do away with paper (like said above) and spend the forensic resources hiring more Cisco Professionals as Test Givers, meaning like the lab for the CCIE. I would be willing to pay more, if I knew a person would actually have to be knowledgeable and be able to prove their skills in a lab environment. Paper tests don’t prove real world skills…doing proves everything. I have known people to get a CCNA…and didn’t know how to plug in a router. Sad.

    Comment by Eddie Jackson — March 27, 2009 @ 9:59 am | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: