1
00:00:12,639 --> 00:00:18,480
hello and welcome back to pike online

2
00:00:15,679 --> 00:00:22,240
2021 the education track we hope you all

3
00:00:18,480 --> 00:00:25,039
had a fabulous lunch break um and i am

4
00:00:22,240 --> 00:00:25,840
joined here by stephen tierney and ali

5
00:00:25,039 --> 00:00:30,000
wong

6
00:00:25,840 --> 00:00:34,320
on what promises to be a bit of an epic

7
00:00:30,000 --> 00:00:36,960
tale i'm really excited i was going to

8
00:00:34,320 --> 00:00:38,559
give a nice intro to both of our

9
00:00:36,960 --> 00:00:41,440
speakers but apparently they've got that

10
00:00:38,559 --> 00:00:44,079
covered so i don't have much of a job

11
00:00:41,440 --> 00:00:46,000
here at all other than to welcome you um

12
00:00:44,079 --> 00:00:48,160
but but i'm really bad at not doing that

13
00:00:46,000 --> 00:00:49,440
i need a job too so steven and ali are

14
00:00:48,160 --> 00:00:51,360
from the university of sydney both of

15
00:00:49,440 --> 00:00:53,760
them and they're going to take us on an

16
00:00:51,360 --> 00:00:54,640
epic journey of what i can only assume

17
00:00:53,760 --> 00:00:57,199
will be

18
00:00:54,640 --> 00:00:58,399
hope uh despair and i very much hope

19
00:00:57,199 --> 00:01:02,399
success

20
00:00:58,399 --> 00:01:05,360
um i i really want to see this heroes

21
00:01:02,399 --> 00:01:06,320
heroes tale the the arc of learning

22
00:01:05,360 --> 00:01:08,159
um

23
00:01:06,320 --> 00:01:10,720
i'm i'm excited to go on this journey

24
00:01:08,159 --> 00:01:12,080
with you both um they will be asked they

25
00:01:10,720 --> 00:01:14,479
will be answering questions perhaps they

26
00:01:12,080 --> 00:01:17,520
will also be asking questions uh at the

27
00:01:14,479 --> 00:01:20,960
end so as always throw those questions

28
00:01:17,520 --> 00:01:24,479
uh in the chat uh and

29
00:01:20,960 --> 00:01:26,320
i'm i'm keen tell me a story

30
00:01:24,479 --> 00:01:28,320
all right hi everyone uh thanks for

31
00:01:26,320 --> 00:01:30,560
tuning in to our talk today

32
00:01:28,320 --> 00:01:32,640
um this talk is about our recent

33
00:01:30,560 --> 00:01:35,280
experiences with teaching uh remotely

34
00:01:32,640 --> 00:01:36,960
due to covid over the sort of last year

35
00:01:35,280 --> 00:01:39,119
and a half and we're going to be telling

36
00:01:36,960 --> 00:01:42,880
our tale through the lens of lord of the

37
00:01:39,119 --> 00:01:45,360
rings there's going to be spoilers so

38
00:01:42,880 --> 00:01:46,880
just be warned but i make no apologies

39
00:01:45,360 --> 00:01:49,280
because it has been 20 years since it

40
00:01:46,880 --> 00:01:51,360
came out

41
00:01:49,280 --> 00:01:53,280
hi again everyone so i'm stephen i'm a

42
00:01:51,360 --> 00:01:55,360
lecturer at the university of sydney

43
00:01:53,280 --> 00:01:57,200
i kind of teach a mix of statistics data

44
00:01:55,360 --> 00:01:59,119
science and machine learning and i work

45
00:01:57,200 --> 00:02:01,200
in the discipline of business analytics

46
00:01:59,119 --> 00:02:04,000
within the business school

47
00:02:01,200 --> 00:02:06,320
and i'm ali i'm an astrophysics phd

48
00:02:04,000 --> 00:02:08,479
student at the university of sydney and

49
00:02:06,320 --> 00:02:10,879
i tutor programming in data science and

50
00:02:08,479 --> 00:02:12,800
computational modelling units and i've

51
00:02:10,879 --> 00:02:15,200
also recently joined the super lovely

52
00:02:12,800 --> 00:02:16,879
team at ed

53
00:02:15,200 --> 00:02:18,160
so we'll talk about ed a little bit

54
00:02:16,879 --> 00:02:18,959
later

55
00:02:18,160 --> 00:02:21,599
but

56
00:02:18,959 --> 00:02:23,440
first i want to go back to our proposal

57
00:02:21,599 --> 00:02:25,120
and we originally wrote that we can

58
00:02:23,440 --> 00:02:27,360
reach any student no matter their

59
00:02:25,120 --> 00:02:29,200
location time zone or computing device

60
00:02:27,360 --> 00:02:30,560
with a real-time collaborative coding

61
00:02:29,200 --> 00:02:33,680
environment

62
00:02:30,560 --> 00:02:35,599
that sounds like a pretty big brag

63
00:02:33,680 --> 00:02:37,599
and that's not the main point of today's

64
00:02:35,599 --> 00:02:40,400
talk we don't want to just show off we

65
00:02:37,599 --> 00:02:42,560
also want to share our journey uh how we

66
00:02:40,400 --> 00:02:44,239
got to that point uh so that you can all

67
00:02:42,560 --> 00:02:46,319
learn from our mistakes

68
00:02:44,239 --> 00:02:47,840
and what we consider must-haves to be

69
00:02:46,319 --> 00:02:49,280
able to do that

70
00:02:47,840 --> 00:02:51,280
and whether that's

71
00:02:49,280 --> 00:02:52,879
for small classes or teaching

72
00:02:51,280 --> 00:02:55,519
one and a half thousand students as in

73
00:02:52,879 --> 00:02:57,519
our case and also how we actually went

74
00:02:55,519 --> 00:02:59,200
about implementing those ideas and

75
00:02:57,519 --> 00:03:02,080
techniques

76
00:02:59,200 --> 00:03:03,840
and we're coming from a tertiary level

77
00:03:02,080 --> 00:03:05,360
position but we think our talk is going

78
00:03:03,840 --> 00:03:06,959
to be useful for educators for all

79
00:03:05,360 --> 00:03:10,000
education levels whether that's primary

80
00:03:06,959 --> 00:03:10,000
or secondary as well

81
00:03:10,480 --> 00:03:15,360
so just to give you some context uh we

82
00:03:13,599 --> 00:03:16,560
have been teaching this unit bus 6002

83
00:03:15,360 --> 00:03:17,680
data science and business for a few

84
00:03:16,560 --> 00:03:19,840
years

85
00:03:17,680 --> 00:03:21,920
it's a postgraduate unit at the

86
00:03:19,840 --> 00:03:24,080
university of sydney business school

87
00:03:21,920 --> 00:03:26,000
it's a sort of as an introduction to

88
00:03:24,080 --> 00:03:27,760
data science and also an introduction to

89
00:03:26,000 --> 00:03:30,159
programming

90
00:03:27,760 --> 00:03:31,680
and we teach with python

91
00:03:30,159 --> 00:03:33,920
and we teach with a view to actually

92
00:03:31,680 --> 00:03:35,920
doing applied data science

93
00:03:33,920 --> 00:03:37,920
unfortunately for us though uh our

94
00:03:35,920 --> 00:03:41,120
students often have zero programming

95
00:03:37,920 --> 00:03:43,760
experience uh they're coming into this

96
00:03:41,120 --> 00:03:46,720
uh program this degree uh usually

97
00:03:43,760 --> 00:03:48,159
pivoting out of some other undergraduate

98
00:03:46,720 --> 00:03:52,239
program

99
00:03:48,159 --> 00:03:54,480
that's typically unrelated entirely

100
00:03:52,239 --> 00:03:55,519
this is really a challenging environment

101
00:03:54,480 --> 00:03:56,720
to work in

102
00:03:55,519 --> 00:03:57,920
although i think a lot of you are

103
00:03:56,720 --> 00:03:59,599
working in similar environments where

104
00:03:57,920 --> 00:04:01,840
you're teaching novices for the first

105
00:03:59,599 --> 00:04:01,840
time

106
00:04:02,720 --> 00:04:05,680
and just to give you an idea of what

107
00:04:03,680 --> 00:04:08,080
we're teaching over our sort of 12-week

108
00:04:05,680 --> 00:04:10,799
semester period

109
00:04:08,080 --> 00:04:12,560
we start with python so the basics of

110
00:04:10,799 --> 00:04:13,920
variables conditionals iteration those

111
00:04:12,560 --> 00:04:15,920
sorts of things so programming

112
00:04:13,920 --> 00:04:18,239
fundamentals and then we move through

113
00:04:15,920 --> 00:04:19,840
some data wrangling exploratory analysis

114
00:04:18,239 --> 00:04:22,079
and even numerical computing using

115
00:04:19,840 --> 00:04:24,400
standard packages like pandas matplotlib

116
00:04:22,079 --> 00:04:25,440
and numpy and then finally in the last

117
00:04:24,400 --> 00:04:26,800
few weeks we get the students to

118
00:04:25,440 --> 00:04:29,360
actually do some statistical inference

119
00:04:26,800 --> 00:04:31,120
and modeling using psychic learn

120
00:04:29,360 --> 00:04:32,960
for the pythonistas amongst you that's

121
00:04:31,120 --> 00:04:35,199
probably a piece of cake but for these

122
00:04:32,960 --> 00:04:37,360
students that's really fast

123
00:04:35,199 --> 00:04:39,759
and just to give you an idea in sort of

124
00:04:37,360 --> 00:04:41,280
an undergraduate computer science course

125
00:04:39,759 --> 00:04:42,479
usually

126
00:04:41,280 --> 00:04:44,080
sort of the programming fundamentals are

127
00:04:42,479 --> 00:04:47,600
covered over an entire semester not just

128
00:04:44,080 --> 00:04:47,600
the first few weeks like we teach it

129
00:04:48,639 --> 00:04:54,320
and in the past we used to teach using

130
00:04:51,600 --> 00:04:56,880
the one ring to rule them all jupiter

131
00:04:54,320 --> 00:04:59,840
so every tutorial was uh the material

132
00:04:56,880 --> 00:05:01,360
was distributed as a jupiter notebook um

133
00:04:59,840 --> 00:05:04,000
and we chose notebooks because they are

134
00:05:01,360 --> 00:05:05,520
the de facto tool for data science uh so

135
00:05:04,000 --> 00:05:07,280
we thought okay well we need to give

136
00:05:05,520 --> 00:05:09,039
this equip the students with the ability

137
00:05:07,280 --> 00:05:10,400
and knowledge of actually using the real

138
00:05:09,039 --> 00:05:11,280
tools

139
00:05:10,400 --> 00:05:12,639
but

140
00:05:11,280 --> 00:05:14,320
over time we started to notice some

141
00:05:12,639 --> 00:05:16,479
problems with that

142
00:05:14,320 --> 00:05:18,639
so often our notebooks are really large

143
00:05:16,479 --> 00:05:20,320
they were really long and tedious

144
00:05:18,639 --> 00:05:21,360
just trying to cram all that information

145
00:05:20,320 --> 00:05:23,520
in

146
00:05:21,360 --> 00:05:24,960
and because of that they

147
00:05:23,520 --> 00:05:26,800
students kind of fell a little bit into

148
00:05:24,960 --> 00:05:28,320
a passive learning mode they would just

149
00:05:26,800 --> 00:05:29,759
sort of sit back and run the pre-written

150
00:05:28,320 --> 00:05:30,800
examples rather than writing much code

151
00:05:29,759 --> 00:05:32,479
themselves

152
00:05:30,800 --> 00:05:34,400
and there are other some technical

153
00:05:32,479 --> 00:05:36,160
problems with notebooks like when we

154
00:05:34,400 --> 00:05:38,000
when we start students on notebooks they

155
00:05:36,160 --> 00:05:40,080
kind of don't understand the notebook or

156
00:05:38,000 --> 00:05:41,680
the jupiter model

157
00:05:40,080 --> 00:05:43,120
they sort of don't understand out of

158
00:05:41,680 --> 00:05:44,560
order execution and then how there's a

159
00:05:43,120 --> 00:05:45,680
sort of a jupiter server running in the

160
00:05:44,560 --> 00:05:47,600
background that they access through a

161
00:05:45,680 --> 00:05:49,120
browser that's quite confusing for

162
00:05:47,600 --> 00:05:52,240
students and you have to get students to

163
00:05:49,120 --> 00:05:54,000
install jupiter at some point as well

164
00:05:52,240 --> 00:05:55,600
and we used to teach in

165
00:05:54,000 --> 00:05:57,360
physical classrooms that would normally

166
00:05:55,600 --> 00:05:59,919
be some sort of computer lab or a

167
00:05:57,360 --> 00:06:01,199
learning studio here's a picture from

168
00:05:59,919 --> 00:06:02,400
one of the newest buildings of the

169
00:06:01,199 --> 00:06:04,960
business school

170
00:06:02,400 --> 00:06:06,880
in a learning studio

171
00:06:04,960 --> 00:06:09,919
and in this sort of setting we would

172
00:06:06,880 --> 00:06:12,240
have two staff members always for us

173
00:06:09,919 --> 00:06:13,759
a class size of about 30 students we

174
00:06:12,240 --> 00:06:16,880
would have a tutor

175
00:06:13,759 --> 00:06:18,560
who sort of runs and manages the class

176
00:06:16,880 --> 00:06:20,240
and then we have a python helper who is

177
00:06:18,560 --> 00:06:22,319
there to provide assistance to that

178
00:06:20,240 --> 00:06:23,520
tutor and also individual assistants to

179
00:06:22,319 --> 00:06:25,199
the students

180
00:06:23,520 --> 00:06:26,800
after all frodo wouldn't have got far

181
00:06:25,199 --> 00:06:27,600
without sam

182
00:06:26,800 --> 00:06:29,360
and

183
00:06:27,600 --> 00:06:30,880
this made it really easy to monitor

184
00:06:29,360 --> 00:06:32,319
student progress and build rapport with

185
00:06:30,880 --> 00:06:33,840
students over the semester and students

186
00:06:32,319 --> 00:06:36,960
also formed bonds and friendships with

187
00:06:33,840 --> 00:06:39,360
each other which was really nice

188
00:06:36,960 --> 00:06:42,080
so we had a system the system was

189
00:06:39,360 --> 00:06:44,560
working it was great but then our

190
00:06:42,080 --> 00:06:47,600
semester one last year around about week

191
00:06:44,560 --> 00:06:50,800
four something that no one saw coming uh

192
00:06:47,600 --> 00:06:53,120
was covid so very quickly the university

193
00:06:50,800 --> 00:06:55,759
was locked down and we kind of just had

194
00:06:53,120 --> 00:06:58,080
to like migrate everything online

195
00:06:55,759 --> 00:07:00,160
overnight so as some of you guys know

196
00:06:58,080 --> 00:07:02,319
teaching programming is already

197
00:07:00,160 --> 00:07:04,800
challenging enough but then teaching

198
00:07:02,319 --> 00:07:07,280
online without all the proper resources

199
00:07:04,800 --> 00:07:09,759
in place felt like a near impossible

200
00:07:07,280 --> 00:07:12,160
fate and so there we were coverts

201
00:07:09,759 --> 00:07:14,840
happening and we've just been thrust

202
00:07:12,160 --> 00:07:17,840
into this remote teaching with zero

203
00:07:14,840 --> 00:07:20,240
experience and so we're kind of

204
00:07:17,840 --> 00:07:22,000
struggling a little uh we felt lost we

205
00:07:20,240 --> 00:07:23,840
felt like we were blind our usual

206
00:07:22,000 --> 00:07:25,520
methods of teaching where we'd be

207
00:07:23,840 --> 00:07:27,680
hands-on in the classroom talking to

208
00:07:25,520 --> 00:07:29,440
students helping them with their coach

209
00:07:27,680 --> 00:07:31,919
heading with them to see how they were

210
00:07:29,440 --> 00:07:34,800
doing uh that was all just like gone

211
00:07:31,919 --> 00:07:37,759
and we needed a way of replicating the

212
00:07:34,800 --> 00:07:40,000
success of our in-person tutorials to

213
00:07:37,759 --> 00:07:42,560
work in an online environment through

214
00:07:40,000 --> 00:07:45,599
video conferencing tools like zoom

215
00:07:42,560 --> 00:07:47,680
and i'll be honest uh we didn't do well

216
00:07:45,599 --> 00:07:50,080
initially so attendance was just

217
00:07:47,680 --> 00:07:52,639
dropping questions weren't being asked

218
00:07:50,080 --> 00:07:54,080
and students just weren't uh completing

219
00:07:52,639 --> 00:07:57,120
the tutorials

220
00:07:54,080 --> 00:07:59,360
but the worst was yet to come so little

221
00:07:57,120 --> 00:08:01,039
did we know there was an army of

222
00:07:59,360 --> 00:08:04,080
students approaching that we would have

223
00:08:01,039 --> 00:08:07,199
to teach we were facing 1200 students in

224
00:08:04,080 --> 00:08:09,440
the next semester and then 1600 in the

225
00:08:07,199 --> 00:08:11,360
semester after that and so we thought

226
00:08:09,440 --> 00:08:12,879
well we need our own little army of

227
00:08:11,360 --> 00:08:16,319
tutors equipped with the best

228
00:08:12,879 --> 00:08:18,240
educational tools in order to survive

229
00:08:16,319 --> 00:08:20,400
so we didn't have that much time to

230
00:08:18,240 --> 00:08:22,879
prepare but we needed to kind of stop

231
00:08:20,400 --> 00:08:25,680
and think about what we needed in the

232
00:08:22,879 --> 00:08:27,199
classroom for our mission

233
00:08:25,680 --> 00:08:28,800
uh the first thing that was really

234
00:08:27,199 --> 00:08:30,960
important was we needed to get the

235
00:08:28,800 --> 00:08:33,519
students engaged so we know from the

236
00:08:30,960 --> 00:08:36,000
literature that students learn best in

237
00:08:33,519 --> 00:08:37,599
an active environment but unfortunately

238
00:08:36,000 --> 00:08:38,880
our students they're all hiding behind

239
00:08:37,599 --> 00:08:40,560
their screens

240
00:08:38,880 --> 00:08:43,200
silently watching their tutors walk

241
00:08:40,560 --> 00:08:45,040
through each jupiter notebook and a lot

242
00:08:43,200 --> 00:08:47,440
of students were unfortunately just not

243
00:08:45,040 --> 00:08:50,880
participating in class or doing any of

244
00:08:47,440 --> 00:08:52,000
the exercises and attendance was pretty

245
00:08:50,880 --> 00:08:53,839
dismal

246
00:08:52,000 --> 00:08:55,519
then another really important thing that

247
00:08:53,839 --> 00:08:57,200
we needed online was we needed

248
00:08:55,519 --> 00:08:59,519
visibility and we just had none we were

249
00:08:57,200 --> 00:09:01,680
totally blind so all of the students are

250
00:08:59,519 --> 00:09:03,600
sitting there with their cameras off and

251
00:09:01,680 --> 00:09:05,600
we had no idea if they were

252
00:09:03,600 --> 00:09:07,680
understanding what was going on whether

253
00:09:05,600 --> 00:09:09,839
they were just confused or just bought

254
00:09:07,680 --> 00:09:11,200
out of their minds or to be honest even

255
00:09:09,839 --> 00:09:13,760
if they were there

256
00:09:11,200 --> 00:09:15,600
at all so we knew some of the students

257
00:09:13,760 --> 00:09:17,519
were working but we just couldn't see

258
00:09:15,600 --> 00:09:19,680
what they were doing we tried all sorts

259
00:09:17,519 --> 00:09:21,760
of things like breakout rooms or screen

260
00:09:19,680 --> 00:09:24,560
sharing but it just wasn't effective

261
00:09:21,760 --> 00:09:26,560
because logistically you can't have 30

262
00:09:24,560 --> 00:09:29,600
or so students trying to share their

263
00:09:26,560 --> 00:09:31,680
code simultaneously and also online

264
00:09:29,600 --> 00:09:34,080
students just weren't as proactive

265
00:09:31,680 --> 00:09:36,000
in seeking help and we couldn't really

266
00:09:34,080 --> 00:09:38,240
just jump around the classroom and talk

267
00:09:36,000 --> 00:09:41,040
to the students as we had done

268
00:09:38,240 --> 00:09:43,680
in the past now related to

269
00:09:41,040 --> 00:09:45,519
visibility is feedback so

270
00:09:43,680 --> 00:09:47,760
for all of the students still continuing

271
00:09:45,519 --> 00:09:50,000
on this journey with us we needed to

272
00:09:47,760 --> 00:09:52,160
support them with feedback and that's a

273
00:09:50,000 --> 00:09:53,839
really hard thing to do when you can't

274
00:09:52,160 --> 00:09:56,080
see how they're going

275
00:09:53,839 --> 00:09:57,839
not to mention it's like really super

276
00:09:56,080 --> 00:09:59,200
awkward if

277
00:09:57,839 --> 00:10:00,880
you're trying to give some feedback to

278
00:09:59,200 --> 00:10:03,360
an individual student and there's a

279
00:10:00,880 --> 00:10:05,440
whole class watching because no one

280
00:10:03,360 --> 00:10:06,959
wants to be wrong or kind of feel silly

281
00:10:05,440 --> 00:10:08,560
in front of a class appears except

282
00:10:06,959 --> 00:10:10,240
sometimes you do get the one student who

283
00:10:08,560 --> 00:10:12,640
is okay with that

284
00:10:10,240 --> 00:10:13,920
but most students they needed space to

285
00:10:12,640 --> 00:10:16,079
make mistakes

286
00:10:13,920 --> 00:10:18,560
and to learn from them

287
00:10:16,079 --> 00:10:21,440
now students also expected and they

288
00:10:18,560 --> 00:10:24,000
deserved consistency so we needed to

289
00:10:21,440 --> 00:10:25,680
teach and be consistent along our own

290
00:10:24,000 --> 00:10:28,640
modes of teaching whether they were on

291
00:10:25,680 --> 00:10:30,079
campus or uh learning remotely uh

292
00:10:28,640 --> 00:10:32,480
they're paying the same fees they're

293
00:10:30,079 --> 00:10:34,800
earning the same degree and in later

294
00:10:32,480 --> 00:10:36,720
semesters we had parallel streams online

295
00:10:34,800 --> 00:10:38,800
and face-to-face and we needed the

296
00:10:36,720 --> 00:10:40,800
classes to be the same so we needed a

297
00:10:38,800 --> 00:10:43,760
way to get the learning materials and

298
00:10:40,800 --> 00:10:46,240
also methods of feedback in a way in

299
00:10:43,760 --> 00:10:47,279
some sort of way that worked in both um

300
00:10:46,240 --> 00:10:49,519
settings

301
00:10:47,279 --> 00:10:51,920
then finally probably the biggest

302
00:10:49,519 --> 00:10:54,000
challenge we had was scalability so with

303
00:10:51,920 --> 00:10:56,560
enrollments climbing uh quickly we

304
00:10:54,000 --> 00:10:58,959
needed to be able to find a way to

305
00:10:56,560 --> 00:11:01,279
deliver high quality education to

306
00:10:58,959 --> 00:11:04,000
literally hundreds and hundreds of

307
00:11:01,279 --> 00:11:06,800
students not only for this semester but

308
00:11:04,000 --> 00:11:09,600
for every semester afterwards

309
00:11:06,800 --> 00:11:12,640
so previously these things had seemed

310
00:11:09,600 --> 00:11:16,240
impossible but i come to you at the turn

311
00:11:12,640 --> 00:11:18,079
of the tide so we came up with this idea

312
00:11:16,240 --> 00:11:20,480
to use the newly developed lesson

313
00:11:18,079 --> 00:11:22,000
feature on the ed platform and the

314
00:11:20,480 --> 00:11:24,320
lessons allow you to generate

315
00:11:22,000 --> 00:11:26,640
interactive content which students can

316
00:11:24,320 --> 00:11:28,959
work through alongside

317
00:11:26,640 --> 00:11:31,200
their tutors so what we'll do is we'll

318
00:11:28,959 --> 00:11:32,399
quickly and briefly kind of show you

319
00:11:31,200 --> 00:11:33,760
what this all

320
00:11:32,399 --> 00:11:35,680
means

321
00:11:33,760 --> 00:11:38,480
so on ed we have these things called

322
00:11:35,680 --> 00:11:40,720
lessons and each lesson consists of a

323
00:11:38,480 --> 00:11:42,480
number of slides so these can be

324
00:11:40,720 --> 00:11:44,079
anything from like static information

325
00:11:42,480 --> 00:11:46,320
think powerpoint and like lecture

326
00:11:44,079 --> 00:11:48,399
presentations and like here we are about

327
00:11:46,320 --> 00:11:50,800
hobbits and if it so interests you you

328
00:11:48,399 --> 00:11:52,399
can come and learn about hobbits uh but

329
00:11:50,800 --> 00:11:56,240
what's really cool is you can have

330
00:11:52,399 --> 00:11:57,120
interactive code examples on each page

331
00:11:56,240 --> 00:11:59,120
so

332
00:11:57,120 --> 00:12:01,440
you can have these little experiments

333
00:11:59,120 --> 00:12:03,360
that students can come and play with um

334
00:12:01,440 --> 00:12:05,040
so here's a code snippet where you enter

335
00:12:03,360 --> 00:12:06,639
your name here let's put in our name

336
00:12:05,040 --> 00:12:07,600
aragon

337
00:12:06,639 --> 00:12:09,760
and

338
00:12:07,600 --> 00:12:11,360
we can go ahead and run that

339
00:12:09,760 --> 00:12:12,880
so it's just really nice to have this

340
00:12:11,360 --> 00:12:14,000
kind of interactive slide for the

341
00:12:12,880 --> 00:12:16,079
students

342
00:12:14,000 --> 00:12:20,399
now another really cool thing on ed is

343
00:12:16,079 --> 00:12:23,120
that you can host quizzes and surveys

344
00:12:20,399 --> 00:12:25,519
so it's really nice because students can

345
00:12:23,120 --> 00:12:27,279
test their understanding and as cheers

346
00:12:25,519 --> 00:12:29,519
we can kind of see what concepts our

347
00:12:27,279 --> 00:12:30,720
students are stuck on so let's have a

348
00:12:29,519 --> 00:12:33,600
look here

349
00:12:30,720 --> 00:12:35,680
does anyone remember what gimli asks

350
00:12:33,600 --> 00:12:37,440
galagio for i think it's a locket of

351
00:12:35,680 --> 00:12:38,959
hair

352
00:12:37,440 --> 00:12:40,399
and also it means that students when

353
00:12:38,959 --> 00:12:43,519
they're testing their understanding they

354
00:12:40,399 --> 00:12:46,079
can ask uh tutors questions um in real

355
00:12:43,519 --> 00:12:48,480
time and then tutors are also able to

356
00:12:46,079 --> 00:12:50,480
look at the analytics and check to see

357
00:12:48,480 --> 00:12:51,600
what concepts our students are getting

358
00:12:50,480 --> 00:12:53,839
stuck on

359
00:12:51,600 --> 00:12:55,360
so they don't have to be proactive when

360
00:12:53,839 --> 00:12:57,200
asking for help

361
00:12:55,360 --> 00:13:00,320
now probably the most exciting thing

362
00:12:57,200 --> 00:13:04,320
about the lessons is that we have these

363
00:13:00,320 --> 00:13:05,920
um code challenges and here students can

364
00:13:04,320 --> 00:13:08,240
try and solve these code problems by

365
00:13:05,920 --> 00:13:10,079
typing their code they can go ahead and

366
00:13:08,240 --> 00:13:12,639
run it and they can also go ahead and

367
00:13:10,079 --> 00:13:14,480
mark it which means they get feedback

368
00:13:12,639 --> 00:13:16,160
straight away um

369
00:13:14,480 --> 00:13:17,360
and so they can they don't have to wait

370
00:13:16,160 --> 00:13:20,000
for a tutor to tell them if they're

371
00:13:17,360 --> 00:13:22,639
right or not and the most exciting part

372
00:13:20,000 --> 00:13:24,880
about this is that tutors can dive into

373
00:13:22,639 --> 00:13:27,360
a student's workspace so we can kind of

374
00:13:24,880 --> 00:13:29,360
come in and spy on the students um the

375
00:13:27,360 --> 00:13:31,519
way you would kind of do so by leaning

376
00:13:29,360 --> 00:13:34,079
over a student's shoulder in class and

377
00:13:31,519 --> 00:13:36,320
they can also come in and code alongside

378
00:13:34,079 --> 00:13:37,839
a student in the same manner as like you

379
00:13:36,320 --> 00:13:39,680
would work with someone through a google

380
00:13:37,839 --> 00:13:41,920
doc so stephen is now going to

381
00:13:39,680 --> 00:13:45,199
demonstrate this

382
00:13:41,920 --> 00:13:46,560
okay so i'm going to switch over to

383
00:13:45,199 --> 00:13:48,639
the sort of

384
00:13:46,560 --> 00:13:51,199
tutor perspective

385
00:13:48,639 --> 00:13:53,760
of the analytics page

386
00:13:51,199 --> 00:13:56,560
so hopefully you can see this

387
00:13:53,760 --> 00:13:59,440
so i'll just quickly explain the typical

388
00:13:56,560 --> 00:14:01,199
setup of how we would run a class

389
00:13:59,440 --> 00:14:02,480
so normally if we're teaching online

390
00:14:01,199 --> 00:14:04,880
then the tutor would create a zoom

391
00:14:02,480 --> 00:14:07,920
session which the students all join

392
00:14:04,880 --> 00:14:10,000
and the tutor would share their

393
00:14:07,920 --> 00:14:12,240
their screen with the students and walk

394
00:14:10,000 --> 00:14:13,760
through those concept slides and quizzes

395
00:14:12,240 --> 00:14:16,240
and allow time for students to do those

396
00:14:13,760 --> 00:14:18,480
quizzes and so on uh that asks students

397
00:14:16,240 --> 00:14:19,279
questions and and then when we get to

398
00:14:18,480 --> 00:14:20,800
the

399
00:14:19,279 --> 00:14:22,160
challenge questions the programming

400
00:14:20,800 --> 00:14:24,320
challenge questions the tutors would

401
00:14:22,160 --> 00:14:26,880
then uh direct the students actually

402
00:14:24,320 --> 00:14:28,000
complete those and give them uh

403
00:14:26,880 --> 00:14:29,519
depending on the complexity of the

404
00:14:28,000 --> 00:14:30,959
question uh anywhere from a couple of

405
00:14:29,519 --> 00:14:32,399
minutes to 10 minutes to complete those

406
00:14:30,959 --> 00:14:36,000
questions

407
00:14:32,399 --> 00:14:38,399
and during that time we as tutors or

408
00:14:36,000 --> 00:14:39,440
instructors would be monitoring those

409
00:14:38,399 --> 00:14:40,880
the student progress through the

410
00:14:39,440 --> 00:14:42,639
analytics page

411
00:14:40,880 --> 00:14:44,639
and so on this analytics page this is a

412
00:14:42,639 --> 00:14:46,959
class and we've got sort of a class of

413
00:14:44,639 --> 00:14:48,720
characters from lord of the rings

414
00:14:46,959 --> 00:14:50,800
and left on the column left column you

415
00:14:48,720 --> 00:14:53,120
can see the the students and then along

416
00:14:50,800 --> 00:14:55,440
the top row here you can see the actual

417
00:14:53,120 --> 00:14:57,279
name of the particular slide challenge

418
00:14:55,440 --> 00:14:58,720
quiz and so on

419
00:14:57,279 --> 00:15:00,720
the green ticks

420
00:14:58,720 --> 00:15:03,360
indicate that the student has passed or

421
00:15:00,720 --> 00:15:04,639
read that section just fine yellow means

422
00:15:03,360 --> 00:15:06,000
they've opened it but they haven't

423
00:15:04,639 --> 00:15:08,720
completed it or they haven't clicked

424
00:15:06,000 --> 00:15:11,600
mark or so on and the red means that

425
00:15:08,720 --> 00:15:13,760
they have tried to mark and they've

426
00:15:11,600 --> 00:15:16,079
they haven't passed that exercise

427
00:15:13,760 --> 00:15:17,440
now this will update in relative free

428
00:15:16,079 --> 00:15:19,600
time real time all we need to do is

429
00:15:17,440 --> 00:15:20,720
click the refresh button so i can see

430
00:15:19,600 --> 00:15:22,720
now that

431
00:15:20,720 --> 00:15:23,519
alison who is playing the role of aragon

432
00:15:22,720 --> 00:15:25,120
here

433
00:15:23,519 --> 00:15:26,720
has gone through and looked at some of

434
00:15:25,120 --> 00:15:28,160
these slides and attempted some of the

435
00:15:26,720 --> 00:15:29,839
quizzes and so on

436
00:15:28,160 --> 00:15:31,199
and if we keep going we can monitor that

437
00:15:29,839 --> 00:15:33,440
student's progress

438
00:15:31,199 --> 00:15:34,720
and what would normally happen is

439
00:15:33,440 --> 00:15:36,720
when we get to

440
00:15:34,720 --> 00:15:38,560
a point where we can see that a student

441
00:15:36,720 --> 00:15:39,759
has failed and

442
00:15:38,560 --> 00:15:41,680
particularly a programming challenge we

443
00:15:39,759 --> 00:15:42,639
would then click on that workspace so it

444
00:15:41,680 --> 00:15:44,639
looks like

445
00:15:42,639 --> 00:15:47,040
aragon's up to this how would you get to

446
00:15:44,639 --> 00:15:50,000
mordor exercise so let's go take a look

447
00:15:47,040 --> 00:15:51,759
at what aragon's up to so we can jump

448
00:15:50,000 --> 00:15:53,199
directly into their workspace and type

449
00:15:51,759 --> 00:15:55,360
alongside them let me just hide the

450
00:15:53,199 --> 00:15:57,600
slides on the side

451
00:15:55,360 --> 00:15:59,279
and hopefully you can see that i can see

452
00:15:57,600 --> 00:16:00,959
aragon's cursor here and that's allison

453
00:15:59,279 --> 00:16:02,160
typing in real time

454
00:16:00,959 --> 00:16:04,160
and they

455
00:16:02,160 --> 00:16:06,480
haven't they've had some difficulty with

456
00:16:04,160 --> 00:16:08,720
this challenge and i'm playing the role

457
00:16:06,480 --> 00:16:10,000
of the tutor here and typically what we

458
00:16:08,720 --> 00:16:12,399
see is that students will be working

459
00:16:10,000 --> 00:16:14,320
alongside and they may leave a note as a

460
00:16:12,399 --> 00:16:16,399
comment and say hey it's not working and

461
00:16:14,320 --> 00:16:18,560
then as an instructor we come in and say

462
00:16:16,399 --> 00:16:20,880
okay here's the problem or give the

463
00:16:18,560 --> 00:16:23,279
students a little hint so in this case

464
00:16:20,880 --> 00:16:25,040
it looks like there's a

465
00:16:23,279 --> 00:16:28,320
string quotation mark issue or string

466
00:16:25,040 --> 00:16:29,440
closure issue um because of this

467
00:16:28,320 --> 00:16:31,839
single quote

468
00:16:29,440 --> 00:16:32,720
inside the other single quote so i would

469
00:16:31,839 --> 00:16:36,800
say

470
00:16:32,720 --> 00:16:37,839
hey aragon uh just check your

471
00:16:36,800 --> 00:16:40,000
string

472
00:16:37,839 --> 00:16:41,279
uh looks like

473
00:16:40,000 --> 00:16:44,880
uh you've

474
00:16:41,279 --> 00:16:47,040
got a single quote inside it

475
00:16:44,880 --> 00:16:48,000
and hopefully that's enough of a hint

476
00:16:47,040 --> 00:16:50,000
uh

477
00:16:48,000 --> 00:16:51,759
for the student to go oh yeah i remember

478
00:16:50,000 --> 00:16:53,680
i i need to use a different style of

479
00:16:51,759 --> 00:16:55,360
quote when i've got a single quote uh

480
00:16:53,680 --> 00:16:57,279
inside a string

481
00:16:55,360 --> 00:16:59,600
and then uh

482
00:16:57,279 --> 00:17:01,120
from this point onwards uh

483
00:16:59,600 --> 00:17:02,079
the student would then click mark and

484
00:17:01,120 --> 00:17:03,360
check whether they've got the right

485
00:17:02,079 --> 00:17:05,600
answer again and we could go back to the

486
00:17:03,360 --> 00:17:06,400
analytics page and then hopefully

487
00:17:05,600 --> 00:17:09,360
if

488
00:17:06,400 --> 00:17:09,360
alison can click mark

489
00:17:12,559 --> 00:17:17,480
sorry i need to switch tabs to see what

490
00:17:13,919 --> 00:17:17,480
alison's up to

491
00:17:19,120 --> 00:17:23,600
yep there we go we can see that uh

492
00:17:21,679 --> 00:17:25,039
aragon's now past that exercise and

493
00:17:23,600 --> 00:17:26,160
aragon can move on and we can now we can

494
00:17:25,039 --> 00:17:28,799
move on to

495
00:17:26,160 --> 00:17:31,280
another student

496
00:17:28,799 --> 00:17:34,320
so that's the basic idea of how we teach

497
00:17:31,280 --> 00:17:36,320
we found it to be very effective

498
00:17:34,320 --> 00:17:38,720
going back to the sorts of things that

499
00:17:36,320 --> 00:17:41,120
we can do in challenges

500
00:17:38,720 --> 00:17:44,080
they can be quite simple we can do sort

501
00:17:41,120 --> 00:17:45,679
of like text based or diff based marking

502
00:17:44,080 --> 00:17:47,200
so we can get students to write a

503
00:17:45,679 --> 00:17:49,760
program and then compare what their

504
00:17:47,200 --> 00:17:52,000
program outputs for a fixed set of

505
00:17:49,760 --> 00:17:53,919
inputs and we can even do some more

506
00:17:52,000 --> 00:17:55,520
interesting things like

507
00:17:53,919 --> 00:17:58,240
uh

508
00:17:55,520 --> 00:18:00,320
actually introspect a

509
00:17:58,240 --> 00:18:02,559
matplotlib plot for example so in this

510
00:18:00,320 --> 00:18:04,640
exercise example i've got here

511
00:18:02,559 --> 00:18:06,480
we've got the middle earth sensor so the

512
00:18:04,640 --> 00:18:07,760
the students are asked to

513
00:18:06,480 --> 00:18:10,559
generate a

514
00:18:07,760 --> 00:18:12,640
uh a bar chart that shows the

515
00:18:10,559 --> 00:18:15,600
distribution of races within middle

516
00:18:12,640 --> 00:18:17,919
earth and so if i go ahead i've got a

517
00:18:15,600 --> 00:18:20,240
sort of a partially complete bit of code

518
00:18:17,919 --> 00:18:22,559
here and click mark

519
00:18:20,240 --> 00:18:25,039
we can see that the specific elements of

520
00:18:22,559 --> 00:18:26,400
the plot haven't been achieved so we can

521
00:18:25,039 --> 00:18:28,000
we actually introspect those plots and

522
00:18:26,400 --> 00:18:30,240
go okay does the

523
00:18:28,000 --> 00:18:31,919
y-axis label match does the title match

524
00:18:30,240 --> 00:18:34,880
so for example if i wanted to add a

525
00:18:31,919 --> 00:18:37,840
title i'm going to say plot dot title

526
00:18:34,880 --> 00:18:39,120
and then we'd say races of middle earth

527
00:18:37,840 --> 00:18:41,200
or something like that i think that's

528
00:18:39,120 --> 00:18:43,440
the desired title races middle earth so

529
00:18:41,200 --> 00:18:44,880
if i click mark again

530
00:18:43,440 --> 00:18:48,320
we should see that i passed that

531
00:18:44,880 --> 00:18:48,320
specific title test

532
00:18:52,080 --> 00:18:55,080
okay

533
00:18:56,000 --> 00:18:59,520
that's our demo

534
00:18:57,840 --> 00:19:01,280
um

535
00:18:59,520 --> 00:19:02,720
so what actually happened after the

536
00:19:01,280 --> 00:19:05,840
first time we did this so we rolled this

537
00:19:02,720 --> 00:19:08,480
out in semester 2 of 2020

538
00:19:05,840 --> 00:19:10,960
during the semester we ran our own

539
00:19:08,480 --> 00:19:14,559
survey just to get some feedback on what

540
00:19:10,960 --> 00:19:16,880
was happening and how the students felt

541
00:19:14,559 --> 00:19:19,120
particularly about using ed

542
00:19:16,880 --> 00:19:21,200
and we asked them a question

543
00:19:19,120 --> 00:19:22,320
do you think ed is an effective way to

544
00:19:21,200 --> 00:19:25,360
learn

545
00:19:22,320 --> 00:19:27,200
94.6 of them agreed on a leicard scale

546
00:19:25,360 --> 00:19:28,960
so they either picked strongly agree or

547
00:19:27,200 --> 00:19:30,720
agree

548
00:19:28,960 --> 00:19:32,799
and follow-up questions on the

549
00:19:30,720 --> 00:19:35,760
difficulty of the exercises the breadth

550
00:19:32,799 --> 00:19:39,559
the pacing presentation revealed very

551
00:19:35,760 --> 00:19:39,559
similar levels of agreement

552
00:19:39,600 --> 00:19:42,640
and here's some quotes from that uh

553
00:19:41,440 --> 00:19:45,520
survey so

554
00:19:42,640 --> 00:19:47,440
students often felt that uh ed was their

555
00:19:45,520 --> 00:19:49,120
favorite part of the unit

556
00:19:47,440 --> 00:19:50,640
so students thought it was magical they

557
00:19:49,120 --> 00:19:53,120
loved it

558
00:19:50,640 --> 00:19:55,120
and then they they felt like uh it was

559
00:19:53,120 --> 00:19:58,640
making uh the work more exciting and

560
00:19:55,120 --> 00:19:58,640
therefore easier to learn

561
00:20:01,280 --> 00:20:05,200
and we also

562
00:20:02,799 --> 00:20:08,000
got the highest ever unit of study

563
00:20:05,200 --> 00:20:10,799
survey outcome as a result so this is a

564
00:20:08,000 --> 00:20:14,480
centrally run survey by the university

565
00:20:10,799 --> 00:20:16,480
um and we believe that a large uh

566
00:20:14,480 --> 00:20:19,280
one of the reasons for the uplift in our

567
00:20:16,480 --> 00:20:21,120
unit of study score was because of how

568
00:20:19,280 --> 00:20:22,080
we used ed

569
00:20:21,120 --> 00:20:24,720
and

570
00:20:22,080 --> 00:20:26,559
we've continued to have uh higher uss

571
00:20:24,720 --> 00:20:28,320
scores since then

572
00:20:26,559 --> 00:20:30,159
and typically students can be pretty

573
00:20:28,320 --> 00:20:32,640
harsh on this survey uh because they're

574
00:20:30,159 --> 00:20:34,240
responding on anonymously so they're

575
00:20:32,640 --> 00:20:37,440
they're pretty harsh and honest so we're

576
00:20:34,240 --> 00:20:37,440
really happy with this result

577
00:20:39,280 --> 00:20:43,600
um and so we think this sort of

578
00:20:41,120 --> 00:20:44,880
experimental tutorial design paid off

579
00:20:43,600 --> 00:20:46,640
um and

580
00:20:44,880 --> 00:20:48,640
uh we were really happy to see at the

581
00:20:46,640 --> 00:20:51,200
end from that unit of study survey

582
00:20:48,640 --> 00:20:52,960
students saying that they felt like

583
00:20:51,200 --> 00:20:55,280
there was really no difference to

584
00:20:52,960 --> 00:20:56,640
learning off campus as to learning on

585
00:20:55,280 --> 00:20:58,240
campus and they

586
00:20:56,640 --> 00:20:59,840
felt like even though everything was

587
00:20:58,240 --> 00:21:01,840
online

588
00:20:59,840 --> 00:21:04,159
they were able to really learn

589
00:21:01,840 --> 00:21:05,919
successfully from it which was a big

590
00:21:04,159 --> 00:21:08,400
change from our first semester during

591
00:21:05,919 --> 00:21:08,400
covert

592
00:21:09,679 --> 00:21:13,760
so just to close the loop on our

593
00:21:13,919 --> 00:21:18,400
on our sort of our must-haves that we

594
00:21:16,720 --> 00:21:20,400
defined earlier

595
00:21:18,400 --> 00:21:22,799
on engagement we saw increased

596
00:21:20,400 --> 00:21:25,360
attendance so attendance was uh much

597
00:21:22,799 --> 00:21:27,520
more consistent over the semester

598
00:21:25,360 --> 00:21:29,600
and in within the classrooms students

599
00:21:27,520 --> 00:21:31,200
were much more active

600
00:21:29,600 --> 00:21:32,960
with visibility we have great visibility

601
00:21:31,200 --> 00:21:34,480
we can see what students are doing in

602
00:21:32,960 --> 00:21:37,120
real time

603
00:21:34,480 --> 00:21:39,440
and with feedback we can give them

604
00:21:37,120 --> 00:21:41,200
actionable and immediate feedback

605
00:21:39,440 --> 00:21:43,600
there's very little barrier to giving

606
00:21:41,200 --> 00:21:44,799
feedback it's kind of limited to how

607
00:21:43,600 --> 00:21:47,200
quickly you can type back to the

608
00:21:44,799 --> 00:21:47,200
students

609
00:21:47,440 --> 00:21:51,360
and then on consistency students as you

610
00:21:49,760 --> 00:21:53,600
saw in the previous quotes tended to

611
00:21:51,360 --> 00:21:55,360
think that okay learning this way is

612
00:21:53,600 --> 00:21:57,200
basically the same as learning in person

613
00:21:55,360 --> 00:21:59,440
to a certain degree so students were

614
00:21:57,200 --> 00:22:02,480
happy with the consistency aspect

615
00:21:59,440 --> 00:22:05,120
and for us the scalability side of thing

616
00:22:02,480 --> 00:22:06,960
uh we definitely didn't have we didn't

617
00:22:05,120 --> 00:22:11,200
have any issues that prevented us from

618
00:22:06,960 --> 00:22:13,120
teaching uh during the semester so

619
00:22:11,200 --> 00:22:15,039
in semester two last year we used this

620
00:22:13,120 --> 00:22:17,200
system with over 1200 students and then

621
00:22:15,039 --> 00:22:20,400
in semester one this year was sort of

622
00:22:17,200 --> 00:22:22,480
over 1500 students and we have about 44

623
00:22:20,400 --> 00:22:23,600
tutorials every week with many of those

624
00:22:22,480 --> 00:22:25,200
concurrent

625
00:22:23,600 --> 00:22:28,880
so definitely

626
00:22:25,200 --> 00:22:28,880
scalability wasn't a problem

627
00:22:29,760 --> 00:22:34,799
there are also pos some positive side

628
00:22:31,600 --> 00:22:36,640
effects from using ed this way

629
00:22:34,799 --> 00:22:38,159
it turns out that ed is quite student

630
00:22:36,640 --> 00:22:39,760
paced

631
00:22:38,159 --> 00:22:42,320
students can work at to whatever pace

632
00:22:39,760 --> 00:22:44,080
they like the um

633
00:22:42,320 --> 00:22:46,159
the automatic marking allows the slower

634
00:22:44,080 --> 00:22:47,600
students to to safely fall behind

635
00:22:46,159 --> 00:22:49,200
because they can catch up later and use

636
00:22:47,600 --> 00:22:51,360
the automatic marking

637
00:22:49,200 --> 00:22:53,120
for self-directed learning the faster

638
00:22:51,360 --> 00:22:54,720
students we equip those students with

639
00:22:53,120 --> 00:22:56,080
extension exercises so they could move

640
00:22:54,720 --> 00:22:57,600
ahead of the class and get to those more

641
00:22:56,080 --> 00:22:59,520
difficult challenging exercises if they

642
00:22:57,600 --> 00:23:01,520
wanted to

643
00:22:59,520 --> 00:23:03,600
and the other side effect of using ed

644
00:23:01,520 --> 00:23:05,840
was that students no longer had to

645
00:23:03,600 --> 00:23:07,760
install anaconda or jupiter

646
00:23:05,840 --> 00:23:10,240
all they needed was a browser because ed

647
00:23:07,760 --> 00:23:12,720
is a browser-based environment

648
00:23:10,240 --> 00:23:14,159
so we sort of saved a lot of time

649
00:23:12,720 --> 00:23:15,919
debugging and

650
00:23:14,159 --> 00:23:18,559
providing technical support to students

651
00:23:15,919 --> 00:23:18,559
with jupiter

652
00:23:19,919 --> 00:23:24,799
what we did wasn't perfect by any means

653
00:23:22,159 --> 00:23:26,960
uh there are still areas of improvement

654
00:23:24,799 --> 00:23:28,320
uh for example it's really hard to make

655
00:23:26,960 --> 00:23:29,520
a personal connection with the students

656
00:23:28,320 --> 00:23:31,360
even though we do have the ability to

657
00:23:29,520 --> 00:23:32,799
give students immediate feedback a lot

658
00:23:31,360 --> 00:23:34,400
of the students are resistant to turning

659
00:23:32,799 --> 00:23:36,400
on their cameras in zoom meetings and

660
00:23:34,400 --> 00:23:38,000
and so on so it's hard to make that

661
00:23:36,400 --> 00:23:39,919
personal connection with them with the

662
00:23:38,000 --> 00:23:41,760
students and then between students it's

663
00:23:39,919 --> 00:23:42,960
also hard to make personal connections

664
00:23:41,760 --> 00:23:44,320
so there's

665
00:23:42,960 --> 00:23:45,520
very little opportunity for peer

666
00:23:44,320 --> 00:23:47,279
learning but there are some features

667
00:23:45,520 --> 00:23:48,640
being added to ed that we've seen in the

668
00:23:47,279 --> 00:23:51,360
pipeline which might make that a bit

669
00:23:48,640 --> 00:23:53,919
easier

670
00:23:51,360 --> 00:23:55,679
other than the sort of the social aspect

671
00:23:53,919 --> 00:23:57,200
it is actually still quite a lot of work

672
00:23:55,679 --> 00:23:59,600
to do this

673
00:23:57,200 --> 00:24:01,600
moving to ed required a huge amount of

674
00:23:59,600 --> 00:24:04,000
time and effort from allison and a

675
00:24:01,600 --> 00:24:04,880
handful of other people

676
00:24:04,000 --> 00:24:07,760
and

677
00:24:04,880 --> 00:24:10,320
not only effort but actual expertise in

678
00:24:07,760 --> 00:24:12,880
designing and actually

679
00:24:10,320 --> 00:24:14,720
using designing exercises and actually

680
00:24:12,880 --> 00:24:16,240
writing tests

681
00:24:14,720 --> 00:24:18,480
so for example for each exercise you

682
00:24:16,240 --> 00:24:20,400
need to come up with a description

683
00:24:18,480 --> 00:24:22,480
hints examples so students don't get

684
00:24:20,400 --> 00:24:24,320
stuck you need a scaffold or a starting

685
00:24:22,480 --> 00:24:25,760
point for those students

686
00:24:24,320 --> 00:24:27,440
you need to collect any data or whatever

687
00:24:25,760 --> 00:24:29,600
the students need and make it available

688
00:24:27,440 --> 00:24:30,559
to them you need a solution for that

689
00:24:29,600 --> 00:24:32,080
normally

690
00:24:30,559 --> 00:24:33,919
and you also need like i said a suite of

691
00:24:32,080 --> 00:24:35,279
tests to actually do the automatic

692
00:24:33,919 --> 00:24:37,200
marking

693
00:24:35,279 --> 00:24:40,080
so over the past few semesters we've

694
00:24:37,200 --> 00:24:43,279
built up a huge library of programming

695
00:24:40,080 --> 00:24:46,159
challenges so we kind of have over 200

696
00:24:43,279 --> 00:24:48,000
it's probably closer uh

697
00:24:46,159 --> 00:24:50,720
it's i think well over 200 programming

698
00:24:48,000 --> 00:24:52,320
challenges built as a result of this

699
00:24:50,720 --> 00:24:54,960
but it's not for free you have to do the

700
00:24:52,320 --> 00:24:54,960
work yourself

701
00:24:55,279 --> 00:24:59,200
here's an example of what such a test

702
00:24:56,640 --> 00:25:01,200
would look like

703
00:24:59,200 --> 00:25:02,799
so it might be a little bit small to sit

704
00:25:01,200 --> 00:25:04,320
and hard to read on your screen but this

705
00:25:02,799 --> 00:25:06,880
is just the to give you an idea of the

706
00:25:04,320 --> 00:25:07,760
scope of a test uh that's required for

707
00:25:06,880 --> 00:25:10,640
one

708
00:25:07,760 --> 00:25:13,919
exercise so this is the marking code or

709
00:25:10,640 --> 00:25:16,240
smart code that's similar to the um the

710
00:25:13,919 --> 00:25:17,760
middle earth census graph that i showed

711
00:25:16,240 --> 00:25:19,120
earlier

712
00:25:17,760 --> 00:25:22,480
so there's a lot of work to be done you

713
00:25:19,120 --> 00:25:22,480
have to do this for every exercise

714
00:25:22,960 --> 00:25:25,679
and

715
00:25:24,159 --> 00:25:27,679
i think you also need to do this as a

716
00:25:25,679 --> 00:25:29,440
team it's very difficult to do this on

717
00:25:27,679 --> 00:25:31,679
your own it would be very difficult to

718
00:25:29,440 --> 00:25:33,919
do this on your own so to give you an

719
00:25:31,679 --> 00:25:36,080
idea we have one of the largest teams of

720
00:25:33,919 --> 00:25:39,200
instructors in the business school we

721
00:25:36,080 --> 00:25:42,799
had 20 instructors in semester 2 last

722
00:25:39,200 --> 00:25:44,400
year and over 25 in semester 1 this year

723
00:25:42,799 --> 00:25:48,080
and it really wouldn't have been as

724
00:25:44,400 --> 00:25:49,840
successful without such a big team

725
00:25:48,080 --> 00:25:53,200
we had people writing

726
00:25:49,840 --> 00:25:54,880
material like alison and priyanka and

727
00:25:53,200 --> 00:25:56,400
daniel and david also wrote a small

728
00:25:54,880 --> 00:25:58,799
amount of material

729
00:25:56,400 --> 00:26:00,240
people did qa on our exercises and

730
00:25:58,799 --> 00:26:01,760
material each week to make sure

731
00:26:00,240 --> 00:26:04,640
everything was working as expected ahead

732
00:26:01,760 --> 00:26:05,760
of time and then everyone or the rest of

733
00:26:04,640 --> 00:26:07,919
the instructors would always give

734
00:26:05,760 --> 00:26:09,679
thoughtful and feedback and deliver the

735
00:26:07,919 --> 00:26:11,360
material every week so

736
00:26:09,679 --> 00:26:12,559
on the scalability side of things you do

737
00:26:11,360 --> 00:26:15,840
need a large number of people when

738
00:26:12,559 --> 00:26:16,799
dealing with a problem of our scale but

739
00:26:15,840 --> 00:26:18,480
um

740
00:26:16,799 --> 00:26:20,000
i think you can uh

741
00:26:18,480 --> 00:26:21,440
get away with a much smaller team if

742
00:26:20,000 --> 00:26:23,120
you're delivering this to it to a maybe

743
00:26:21,440 --> 00:26:25,120
like a in a primary audience or a

744
00:26:23,120 --> 00:26:26,880
secondary audience so a special thank

745
00:26:25,120 --> 00:26:28,960
you to all the people that i've listed

746
00:26:26,880 --> 00:26:32,880
here who went the extra mile and

747
00:26:28,960 --> 00:26:32,880
contributed heavily to bus 6002

748
00:26:34,320 --> 00:26:40,080
so that brings us to the end of our talk

749
00:26:37,440 --> 00:26:42,720
we'd really like to extend a sincere

750
00:26:40,080 --> 00:26:45,200
thanks to the pycon organizers and

751
00:26:42,720 --> 00:26:47,760
volunteers who make pycon possible every

752
00:26:45,200 --> 00:26:48,960
year so thank you everyone

753
00:26:47,760 --> 00:26:52,679
and

754
00:26:48,960 --> 00:26:52,679
that's it from us

755
00:26:55,840 --> 00:26:59,600
thank you so much when i return remember

756
00:26:57,600 --> 00:27:02,000
to turn my mic back on um that was

757
00:26:59,600 --> 00:27:03,919
fantastic we are almost out of time but

758
00:27:02,000 --> 00:27:07,039
we can see if we can sneak in

759
00:27:03,919 --> 00:27:07,919
um one or two questions uh let me just

760
00:27:07,039 --> 00:27:10,960
get it

761
00:27:07,919 --> 00:27:13,760
um there was the one question on can the

762
00:27:10,960 --> 00:27:13,760
students collaborate

763
00:27:14,880 --> 00:27:19,039
yes they can collaborate

764
00:27:17,039 --> 00:27:21,360
do you want to take this allison ah sure

765
00:27:19,039 --> 00:27:23,279
do you want to pull up the workspaces

766
00:27:21,360 --> 00:27:25,840
uh yeah let me just grab that link so

767
00:27:23,279 --> 00:27:27,279
we've been demonstrating the lessons but

768
00:27:25,840 --> 00:27:28,640
there's also a feature on ed called the

769
00:27:27,279 --> 00:27:31,039
workspaces

770
00:27:28,640 --> 00:27:33,360
and uh it'll be in the same kind of

771
00:27:31,039 --> 00:27:35,600
course so here you can see

772
00:27:33,360 --> 00:27:36,720
we have a workspace here stephen can

773
00:27:35,600 --> 00:27:38,640
type

774
00:27:36,720 --> 00:27:40,720
and if you click the kind of little

775
00:27:38,640 --> 00:27:42,799
button that looks like a speech bubble

776
00:27:40,720 --> 00:27:45,679
next to the share that will also bring

777
00:27:42,799 --> 00:27:47,520
up a chat so students can also come in

778
00:27:45,679 --> 00:27:50,799
and talk here

779
00:27:47,520 --> 00:27:53,279
or come in and say hi and photo can come

780
00:27:50,799 --> 00:27:56,720
in and say hi so students can kind of

781
00:27:53,279 --> 00:27:58,240
work on larger kind of code

782
00:27:56,720 --> 00:27:59,440
problems together

783
00:27:58,240 --> 00:28:01,760
there's also

784
00:27:59,440 --> 00:28:03,520
another example where they can code on

785
00:28:01,760 --> 00:28:06,799
like a jupiter sort of environment

786
00:28:03,520 --> 00:28:06,799
collaboratively as well

787
00:28:07,360 --> 00:28:11,520
fantastic and that actually also

788
00:28:09,600 --> 00:28:13,760
partially answers this other question is

789
00:28:11,520 --> 00:28:15,440
there a chat function in a workspace so

790
00:28:13,760 --> 00:28:18,960
fantastic excellent

791
00:28:15,440 --> 00:28:20,880
um let's see if we've got another second

792
00:28:18,960 --> 00:28:21,760
but yeah we've got another we can feed

793
00:28:20,880 --> 00:28:24,399
um

794
00:28:21,760 --> 00:28:26,480
so this question is in my class the

795
00:28:24,399 --> 00:28:28,159
difference between the completion rate

796
00:28:26,480 --> 00:28:33,279
of the front and the back of the class

797
00:28:28,159 --> 00:28:33,279
is huge how did you manage that

798
00:28:34,720 --> 00:28:38,720
we have the same problem i think um

799
00:28:37,360 --> 00:28:41,200
yeah same sort of distribution of

800
00:28:38,720 --> 00:28:42,640
completion rates um i don't know whether

801
00:28:41,200 --> 00:28:44,240
i don't think we have the answer to that

802
00:28:42,640 --> 00:28:45,840
or at least i don't can you think of

803
00:28:44,240 --> 00:28:48,799
anything alison

804
00:28:45,840 --> 00:28:50,399
uh no but it's kind of nice at least

805
00:28:48,799 --> 00:28:52,720
that even if you're not completing

806
00:28:50,399 --> 00:28:55,039
things in the tutorial um like in the

807
00:28:52,720 --> 00:28:56,559
live one-hour time slot you have our

808
00:28:55,039 --> 00:28:57,760
students can kind of keep completing

809
00:28:56,559 --> 00:29:00,080
things at their own pace whether they're

810
00:28:57,760 --> 00:29:01,919
fast or slow so that's kind of a nice

811
00:29:00,080 --> 00:29:04,159
thing about the way we've set up the

812
00:29:01,919 --> 00:29:06,480
lessons

813
00:29:04,159 --> 00:29:08,880
that that certainly is a

814
00:29:06,480 --> 00:29:11,440
a pressing challenge for anyone who is

815
00:29:08,880 --> 00:29:13,360
teaching coding and computer science in

816
00:29:11,440 --> 00:29:14,880
general um

817
00:29:13,360 --> 00:29:16,480
yeah it's

818
00:29:14,880 --> 00:29:17,360
uh that that was a fantastic talk thank

819
00:29:16,480 --> 00:29:20,159
you so much thank you for answering

820
00:29:17,360 --> 00:29:24,000
questions there are a few more questions

821
00:29:20,159 --> 00:29:27,279
um that i will invite you to answer in

822
00:29:24,000 --> 00:29:30,159
the uh education hallway chat and i'll

823
00:29:27,279 --> 00:29:32,720
copy them in in just a minute but we are

824
00:29:30,159 --> 00:29:33,520
out of time here for now so we all have

825
00:29:32,720 --> 00:29:36,000
a

826
00:29:33,520 --> 00:29:39,360
15 minute break um

827
00:29:36,000 --> 00:29:41,679
before our next talk um which is katie

828
00:29:39,360 --> 00:29:44,399
bell talking about real python made of

829
00:29:41,679 --> 00:29:46,080
blocks so thank you ali steven thank you

830
00:29:44,399 --> 00:29:48,240
so much for joining us um thank you

831
00:29:46,080 --> 00:29:49,120
fantastic talk what an epic journey it

832
00:29:48,240 --> 00:29:51,039
was

833
00:29:49,120 --> 00:29:53,279
less walking than i remember there being

834
00:29:51,039 --> 00:29:55,440
um uh and

835
00:29:53,279 --> 00:29:58,480
thank you thank you very much uh we hope

836
00:29:55,440 --> 00:29:59,600
you uh we'll see you all in 15 minutes

837
00:29:58,480 --> 00:30:00,720
cheers

838
00:29:59,600 --> 00:30:03,720
thanks guys

839
00:30:00,720 --> 00:30:03,720
bye

840
00:30:11,440 --> 00:30:13,520
you