Dan Davis EDM 2016 Presentation Slides

GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED LEARNING PATHDAN DAVIS, GUANLIANG CHEN, CLAUDIA HAUFF, GEERT-JAN HOUBEN

TU DELFT WEB INFORMATION SYSTEMS GROUP

WHAT IS A LEARNING PATH ?WHAT IS A LEARNING PATH ?

WHAT IS A LEARNING PATH ?THE SEQUENCE OF EVENTS A STUDENT TAKES TOWARDS A LEARNING OBJECTIVE

RQ to what extent do learners adhere to the designed learning path?

VIDEO QUIZ PROGRESS FORUM

WATCH START VIEW STARTSUBMITEND

SUBMITEND

VIDEO QUIZEND

QUIZSUBMIT

QUIZSTART

VIDEO

VIDEO

VIDEO

QUIZEND

QUIZEND

QUIZSUBMIT

QUIZSTART

QUIZSUBMIT

QUIZSTART

Data Analysis

Functional Programming

Framing

Responsible Innovation

SUBMIT ENDFORUMSTART

SUBMIT ENDFORUMSTART

v Q Q Q Q Q Q Q Q Q Q Q Q Q QQ

Q Q Q Q Q Q Q Q Q Q Q Q Q Q

Q Q Q Q Q Q Q Q Q Q Q Q

Q

Q Q Q Q Q Q

v v

v v v F F F F F F

F F F

F F F F F F F F F F F F F F F

v v v v v v v

v v v v v v v

v v v v v v

Q

DESIGNED LEARNING PATHS

Data Analysis

Functional Programming

Framing

Responsible Innovation

RELATED WORK RELATED WORK

Identifying latent study habits by mining learner behavior patterns in massive open online courses. In CIKM ’14MIAOMIAO WEN & CAROLYN ROSÉ

Demographic differences in how students navigate through MOOCs. In L@S ‘14PHILIP GUO & KATHARINA REINECKE

MOOC ENROLLED PASS CHAINSPASS/NON

QUIZQUESTIONSRATE

TRIES VIDEOSACCESSED

FP101x

RI101x

Frame101x

EX101x

37k

9k

34k

33k

5.3%

4.3%

2.4%

6.5%

1.06M/807K

66k/30k

95k/141k

1.02M/855k

288

75

26

136

1

1—3

2

2

67.5%

49.7%

51%

45%

*

* Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%

Frame101x

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

3

3

4

6

36

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

1 1

1

121

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

3

7 7

7

6

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

5 1

1 1

1

1

1

101

FP101x

RI101x

EX101x

Frame101x

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

3

3

4

6

36

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

1 1

1

121

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

3

7 7

7

6

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

5 1

1 1

1

1

1

101

FP101x

RI101x

EX101x

EACH COURSE IS UNIQUE

Frame101x

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

3

3

4

6

36

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

1 1

1

121

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

3

7 7

7

6

VIDEO

QUIZEND

FORUMSUBMIT

FORUMEND

PROGRESS

QUIZSTART

FORUMSTART

QUIZSUBMIT

5 1

1 1

1

1

1

101

FP101x

RI101x

EX101x

EACH COURSE IS UNIQUE

ANALYZE ACCORDINGLY

APPROACH APPROACH 1. VIDEO INTERACTIONS2. BEHAVIOR PATTERN CHAINS3. EVENT TYPE TRANSITIONS

FINDINGSFINDINGS

1. VIDEO INTERACTIONSDO LEARNERS WATCH VIDEO LECTURES IN THE PRESCRIBED ORDER?

VIDEO INTERACTIONS

Non-Passing

Passing

Week 1 Week 2Week 3

FP101xDesigned Lecture Order

Executed Paths

Non-Passing

Passing

Week 1Week 2

Week 3

VIDEO INTERACTIONS

Non-Passing

Passing

Week 1 Week 2Week 3

Non-Passing

Passing

Week 1 Week 2Week 3

Non-Passing

Passing

Week 1Week 2

Week 3

FP101x

RI101x

Frame101x

EX101x

2. BEHAVIOR PATTERN CHAINSWHAT ARE THE MOST COMMON SEQUENCES IN THE EXECUTED LEARNING PATHS?

FORUMEND

EIGHT-STEP CHAINEXAMPLE

LECTUREWATCH QUIZSTART QUIZSUBMIT QUIZSUBMIT

QUIZEND PROGRESS LECTUREWATCH

MOOC ENROLLED PASS CHAINSPASS/NON

QUIZQUESTIONSRATE

TRIES VIDEOSACCESSED

FP101x

RI101x

Frame101x

EX101x

37k

9k

34k

33k

5.3%

4.3%

2.4%

6.5%

1.06M/807K

66k/30k

95k/141k

1.02M/855k

288

75

26

136

1

1—3

2

2

67.5%

49.7%

51%

45%

*

* Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%

OPEN CARD SORTINGOPEN CARD SORTING

MOTIFA RECURRING OR REPEATED ELEMENT

MOTIF FREQTOTAL

FREQPASS

FREQFAIL

QUIZ COMPLETE 552,363(29.4%)

328,995(30.8%)

223,368(27.7%)

1.

BINGE WATCHING 149,784(8%)

59,498(5.6%)

90,286(11.2%)

2.

LECTURE -> QUIZ COMPLETE 100,179(5.3%)

50,415(4.7%)

49,764(6.2%)

3.

QUIZ COMPLETE -> FORUM 99,828(5.3%)

67,722(6.3%)

32,106(4%)

4.

FP101x

XQUIZ events only with at least 1 X = SUBMIT

WATCH events only

WATCH event(s) followed by XQUIZ events w/ >1 X = SUBMIT

XQUIZ events w/ >1 X = SUBMIT followed by XFORUM events

3. EVENT TYPE TRANSITIONSHOW DO STUDENTS NAVIGATE BETWEEN CERTAIN EVENT TYPES?

EVENT TYPE TRANSITIONS

Frame101x Non-PassingMarkov ModelState Visualization

(EXECUTED)


Frame101x Non-Passing Frame101x Passing

(EXECUTED)


Frame101x Non-Passing Frame101x Passing

(EXECUTED)

32%

35%

VIDEO

QUIZEND

QUIZSUBMIT

FORUMSUBMIT

FORUMEND

PROGRESS

FORUMSTART

QUIZSTART VIDEO

QUIZEND QUIZ

SUBMIT

FORUMSUBMIT

FORUMEND

PROGRESS

FORUMSTART

QUIZSTART

21%28%

44%

44%

DAVIS, D., CHEN, G., JIVET, I., HAUFF, C., & HOUBEN, G. J. (2016). ENCOURAGING METACOGNITION & SELF-REGULATION IN MOOCS THROUGH INCREASED LEARNER FEEDBACK. LAK 2016 LAL WORKSHOP

FEEDBACKEXAMPLE

Frame101x PassingFrame101x Non-Passing

RI101x Non-Passing RI101x Passing

EX101x PassingEX101x Non-Passing

FP101x Non-Passing FP101x Passing

APPLICATIONSAPPLICATIONS

1. BEYOND PASS-FAIL SEGMENTATION2. PATTERN MINING & CONFORMANCE CHECKING3. POSITIVE DEVIANCE

THANK YOUTU DELFT WEB INFORMATION SYSTEMS GROUP

DAN DAVIS, GUANLIANG CHEN, CLAUDIA HAUFF, GEERT-JAN HOUBEN

BIT.LY/WIS-LEARNING-ANALYTICS

Dan Davis EDM 2016 Presentation Slides

Science

Transcript of Dan Davis EDM 2016 Presentation Slides