What is Basic Coding Proficiency?

2016 is the first year Code.org is reporting on student Basic Coding Proficiency. This is a metric that is based on student performance on Code Studio puzzles in our Computer Science Fundamentals courses. This metric takes into account the concepts used in each puzzle as well as their difficulty. To organize all this information, we have defined a matrix of learning objectives organized by concept and difficulty level. Every puzzle is tagged with one or more of these “concept-difficulty” values, and those values are fundamental to our metric of proficiency.

What Are Concept-Difficulty Values?

A concept-difficulty value is a programming concept (eg: Sequencing, Repeat Loops) and a difficulty level within that concept (from 1 to 5).

The Concepts are:

• Sequencing (Algorithms)
• Loops (e.g., repeat, repeat until, repeat while, and for)
• Event Handlers
• Variables
• Functions
• Conditionals

The Concept-difficulty tags are based on a matrix of learning objectives by concept and difficulty level within that concept. The concept-difficulty matrix is organized like this:

You can find the complete matrix here. We have tagged every puzzle in our Computer Science Fundamentals courses with the appropriate concepts and difficulty levels.

To understand this matrix, let’s walk through some examples using a single concept: Repeat Loops.

Repeat Loops Difficulty 1:

The learning objectives that map to puzzles of difficulty level 1 for repeat loops are:

Students will be able to...

• Move existing instructions into a loop with prompting
• Construct a simple 1-instruction loop with prompting
• Identify areas of a program that repeat exactly
• Modify or delete a single loop instruction with prompting
• Change the number of iterations of a “Repeat N times” loop

For example, in Course 2 Stage 13 Puzzle 1 students are prompted to use a repeat loop to get the bird to the pig using only “move forward” instructions.

Repeat Loops Difficulty 2

The learning objectives for difficulty level 2 include:

Students will be able to...

• Construct a simple one or two instruction loop without prompting
• Construct simple loop containing three or more instructions with prompting
• Move existing code into a loop without prompting
• Modify or delete a single instruction within a loop without prompting

For example, Course 2 Stage 8 Puzzle 2 instructs students to use a loop, but requires them to identify the three instructions that must be repeated:

Repeat Loops Difficulty 3

The learning objectives for a difficulty 3 puzzle include:

Students will be able to...

• Construct or modify simple repeat loop with 3 or more instructions without prompting
• Use two or more loops in a series with prompting

This example comes from our Accelerated Course, stage 2 puzzle 8. To solve this puzzle, students are told they can try using multiple loops, but must identify the patterns and associated instructions:

Repeat Loops Difficulty 4

Difficulty 4: In difficulty level 4 puzzles, students will be able to…

• Use two or more loops in a series without prompting
• Create a single nested loop layer with prompting

For example, in Course 3 Stage 13 Puzzle 5, students need to place a repeat 4 times loop inside a repeat 5 times loop:

Repeat Loops Difficulty 5

The learning objectives for difficulty level 5 puzzles include:

Students will be able to…

• Construct a single nested loop layer without prompting
• Construct multiple nested loops in sequence
• Construct nested loops more than one layer deep with prompting

This example - Course 3 Stage 21 Puzzle 13 - requires students to create two loops in sequence, and place them within an outer loop:

Combining Concept-Difficulties

Most programming puzzles beyond the simplest examples require students to combine multiple concepts. For this reason, any puzzle can have more than one concept-difficulty tag. For example, Course 3 Stage 8 Puzzle 9 combines Sequencing, Repeat, and Conditional concepts:

This example has a Sequencing difficulty of 4 (“Students will be able to add instructions to branching elements”), Repeat difficulty of 4 (“Students will be able to create a program with one 'repeat until' of 3 or more instructions”), and Conditionals difficulty of 4 (“Students will be able to add instructions to both 'if' and 'else' portions of one or more conditionals.”)

If you’re interested in the full set of concept-difficulties, you can find the complete matrix here.

Using Concept-Difficulty to Assess Proficiency

Every puzzle in Code Studio is an opportunity for students to demonstrate proficiency at the concepts used in that puzzle. When a student completes a puzzle without using any hints and with the optimal number of blocks, that is considered “successful completion.” Every successful completion of a puzzle is evidence of proficiency at the concept-difficulty level of the problem. After sufficient evidence at a level, a student is considered to be at that level of proficiency within the concept.

“Proficiency” within a concept is defined as the successful completion (using no hints and with the optimal number of blocks) of 3 puzzles of difficulty level 3 within that concept. Reaching “Basic Coding Proficiency” is defined as reaching proficiency in three different concepts.

In summary: To demonstrate Basic Coding Proficiency, a student must prove their skill in at least three different concepts. For each concept, the student must complete three or more puzzles of difficulty level 3 without hints and with the optimal number of blocks.