1
00:00:05,239 --> 00:00:07,259
[music]

2
00:00:09,825 --> 00:00:11,845
[music]

3
00:00:15,040 --> 00:00:16,960
Hey, welcome back to the education

4
00:00:16,960 --> 00:00:19,119
track. So, our next speaker we have the

5
00:00:19,119 --> 00:00:22,160
lovely Thea Katsugas. Um, and please

6
00:00:22,160 --> 00:00:25,545
welcome her to stage.

7
00:00:25,545 --> 00:00:27,565
[applause]

8
00:00:30,720 --> 00:00:33,520
All right. Uh, it's great to have Pyon

9
00:00:33,520 --> 00:00:35,280
AU back in Brizzy where the first

10
00:00:35,280 --> 00:00:38,480
education track took place. Um, that was

11
00:00:38,480 --> 00:00:42,320
my first Pyon AU 2015. Uh, and I gave a

12
00:00:42,320 --> 00:00:45,440
pretty terrible talk about PyCharm edu.

13
00:00:45,440 --> 00:00:48,399
So, apologies if anyone from Jet Brains

14
00:00:48,399 --> 00:00:52,120
um, ever saw that.

15
00:00:52,559 --> 00:00:54,719
Um, there are many names for Brisbane.

16
00:00:54,719 --> 00:00:57,199
Thankfully, I think Brisveus is dying

17
00:00:57,199 --> 00:00:59,440
out. Uh, but one of the traditional

18
00:00:59,440 --> 00:01:02,320
names is Megan, which is the Yagura word

19
00:01:02,320 --> 00:01:05,119
for area of the Chulip tree, which you

20
00:01:05,119 --> 00:01:06,960
can see at the Botanic Gardens, just 20

21
00:01:06,960 --> 00:01:09,439
minutes walk from here. Um, credit

22
00:01:09,439 --> 00:01:12,240
credit to Tatiana from Wikipedia for

23
00:01:12,240 --> 00:01:14,400
that image. Um, so I'd like to

24
00:01:14,400 --> 00:01:16,240
acknowledge both the Turble and Yagura

25
00:01:16,240 --> 00:01:18,159
people who've looked after this country

26
00:01:18,159 --> 00:01:20,720
for thousands of years.

27
00:01:20,720 --> 00:01:23,520
Also, hi. Uh, I'm Thea. Uh I'm a

28
00:01:23,520 --> 00:01:25,840
teacher, baker, and software maker. I

29
00:01:25,840 --> 00:01:28,159
say software maker more like little

30
00:01:28,159 --> 00:01:30,400
Python project maker. [laughter]

31
00:01:30,400 --> 00:01:32,799
Uh I started learning Python back in my

32
00:01:32,799 --> 00:01:35,200
second year of an undergraduate AI and

33
00:01:35,200 --> 00:01:37,360
computer science degree. I believe it

34
00:01:37,360 --> 00:01:39,119
was in a natural language processing

35
00:01:39,119 --> 00:01:42,799
course where we used the NLTK library,

36
00:01:42,799 --> 00:01:45,600
old school language AI.

37
00:01:45,600 --> 00:01:49,439
Um, when I got into education working at

38
00:01:49,439 --> 00:01:53,040
a large state school, um, I immediately

39
00:01:53,040 --> 00:01:55,200
went about changing our entire program

40
00:01:55,200 --> 00:01:57,520
to be in Python because I was selfish

41
00:01:57,520 --> 00:02:00,240
and didn't want to learn Visual Basic.

42
00:02:00,240 --> 00:02:02,479
Uh, but I regret nothing. [laughter]

43
00:02:02,479 --> 00:02:05,200
Uh, I taught year 6 to year 12 for 8

44
00:02:05,200 --> 00:02:07,280
years before deciding at the beginning

45
00:02:07,280 --> 00:02:10,000
of this year to make a change. I'm

46
00:02:10,000 --> 00:02:11,760
currently working at the University of

47
00:02:11,760 --> 00:02:14,720
Queensland, UKQ, uh, as a tutor in some

48
00:02:14,720 --> 00:02:18,080
undergrad courses and as the IT teacher

49
00:02:18,080 --> 00:02:20,959
and convenor at Yuk College. For

50
00:02:20,959 --> 00:02:23,520
context, Yuk College is a tertiary

51
00:02:23,520 --> 00:02:26,160
preparation institute. We predominantly

52
00:02:26,160 --> 00:02:28,239
teach students who don't quite meet the

53
00:02:28,239 --> 00:02:31,120
requirements to get into university yet.

54
00:02:31,120 --> 00:02:33,440
Um, perhaps they didn't get NATAR or are

55
00:02:33,440 --> 00:02:35,360
missing prerequisites or they're

56
00:02:35,360 --> 00:02:36,959
international and need to improve their

57
00:02:36,959 --> 00:02:39,920
academic English. I almost entirely

58
00:02:39,920 --> 00:02:42,239
teach beginners and most of them have

59
00:02:42,239 --> 00:02:43,920
significant challenges to learning

60
00:02:43,920 --> 00:02:46,480
programming. But just to be clear, none

61
00:02:46,480 --> 00:02:48,000
of what I'm saying today is the opinion

62
00:02:48,000 --> 00:02:52,000
of my employers, just my own.

63
00:02:52,000 --> 00:02:54,879
And with that, my strongest opinion. Um,

64
00:02:54,879 --> 00:02:56,480
I want to give a disclaimer before I

65
00:02:56,480 --> 00:02:58,720
begin the content of this talk. I hate

66
00:02:58,720 --> 00:03:01,519
generative AI. It's built from large

67
00:03:01,519 --> 00:03:04,080
language models created by stealing data

68
00:03:04,080 --> 00:03:06,800
without consent. It is polluting both

69
00:03:06,800 --> 00:03:09,360
the physical and digital world. I don't

70
00:03:09,360 --> 00:03:11,200
want to use it. I don't want my students

71
00:03:11,200 --> 00:03:13,280
to use it. It makes me ashamed to be in

72
00:03:13,280 --> 00:03:15,840
it. I keep getting told that it is

73
00:03:15,840 --> 00:03:17,840
inevitable, but in my heart, I don't

74
00:03:17,840 --> 00:03:20,480
want to believe that's true. So, if

75
00:03:20,480 --> 00:03:22,000
you've come to this talk hoping for

76
00:03:22,000 --> 00:03:24,400
ideas on how to prompt co-pilot into

77
00:03:24,400 --> 00:03:26,319
giving the perfect task sheet or how to

78
00:03:26,319 --> 00:03:28,239
write guard rails for Claude to help

79
00:03:28,239 --> 00:03:30,239
your students with their senior project,

80
00:03:30,239 --> 00:03:32,799
this is not that talk. Um, but I will

81
00:03:32,799 --> 00:03:34,799
talk about how other people are using it

82
00:03:34,799 --> 00:03:36,480
for assessment despite it not being

83
00:03:36,480 --> 00:03:39,519
something I'll be doing myself.

84
00:03:39,519 --> 00:03:41,760
So, who are these other people? To

85
00:03:41,760 --> 00:03:44,560
research for this talk, I spoke to 30 IT

86
00:03:44,560 --> 00:03:46,799
educators across state and private

87
00:03:46,799 --> 00:03:50,000
schools, universities, and TA, some in

88
00:03:50,000 --> 00:03:52,239
person, some online, and some through an

89
00:03:52,239 --> 00:03:54,080
open survey posted in digital

90
00:03:54,080 --> 00:03:57,040
technologies teacher groups. I asked

91
00:03:57,040 --> 00:03:58,799
whether their institution had an

92
00:03:58,799 --> 00:04:01,200
explicit AI policy, whether they've

93
00:04:01,200 --> 00:04:03,040
changed their assessment in reaction to

94
00:04:03,040 --> 00:04:05,439
AI, and what their general thoughts are

95
00:04:05,439 --> 00:04:08,902
on the use of LLMs in IT education.

96
00:04:08,902 --> 00:04:10,959
[gasps] Also, for the past 2 years, I've

97
00:04:10,959 --> 00:04:12,879
been collecting research papers,

98
00:04:12,879 --> 00:04:16,560
articles, rants on Masttodon um about

99
00:04:16,560 --> 00:04:19,359
AI. So, these have also informed the

100
00:04:19,359 --> 00:04:21,840
talk. I've got a long list of sources

101
00:04:21,840 --> 00:04:24,160
but couldn't work out how to put in a

102
00:04:24,160 --> 00:04:26,240
bibliography in slides without it being

103
00:04:26,240 --> 00:04:28,479
unreadable. So ask me afterwards if

104
00:04:28,479 --> 00:04:31,720
you're interested.

105
00:04:32,160 --> 00:04:33,919
Okay. So you know I think it's legally

106
00:04:33,919 --> 00:04:35,759
part of my registration with the

107
00:04:35,759 --> 00:04:37,600
Queensland College of Teachers that

108
00:04:37,600 --> 00:04:40,560
every lesson includes success criteria.

109
00:04:40,560 --> 00:04:43,120
So by the end of this talk you should be

110
00:04:43,120 --> 00:04:46,800
able to identify examples of secure open

111
00:04:46,800 --> 00:04:49,040
and mixed assessment.

112
00:04:49,040 --> 00:04:51,040
evaluate the strengths and weaknesses of

113
00:04:51,040 --> 00:04:54,080
different AI assessment policies and

114
00:04:54,080 --> 00:04:56,240
understand the potential pedagogical

115
00:04:56,240 --> 00:05:00,360
impacts of using AI.

116
00:05:02,960 --> 00:05:05,919
Let's start with some definitions. AI.

117
00:05:05,919 --> 00:05:08,080
As the excellent professor Emily Bender

118
00:05:08,080 --> 00:05:10,720
states in her article on AI for an

119
00:05:10,720 --> 00:05:13,280
Oxford research encyclopedia, the term

120
00:05:13,280 --> 00:05:15,120
artificial intelligence is typically

121
00:05:15,120 --> 00:05:17,840
used as if it refers to a coherent set

122
00:05:17,840 --> 00:05:20,800
of technologies, but in fact does not.

123
00:05:20,800 --> 00:05:23,039
For the purposes of this talk, when I

124
00:05:23,039 --> 00:05:25,600
say AI, I am referring to a typical

125
00:05:25,600 --> 00:05:29,120
LLMbased chatbot/coding agent, any tool

126
00:05:29,120 --> 00:05:31,840
that is generating code, text, or other

127
00:05:31,840 --> 00:05:34,720
media.

128
00:05:34,720 --> 00:05:37,440
Secure assessment. This is assessment in

129
00:05:37,440 --> 00:05:39,680
which students are directly observed

130
00:05:39,680 --> 00:05:41,919
completing a task where you have high

131
00:05:41,919 --> 00:05:43,680
confidence that the rules of the

132
00:05:43,680 --> 00:05:46,000
assessment were followed. The most

133
00:05:46,000 --> 00:05:48,880
common is obviously the exam or

134
00:05:48,880 --> 00:05:51,840
supervised in-class activities. It also

135
00:05:51,840 --> 00:05:53,600
includes interviews with unseen

136
00:05:53,600 --> 00:05:56,400
questions.

137
00:05:56,400 --> 00:06:00,800
In contrast, uh open assessment is one

138
00:06:00,800 --> 00:06:03,039
in which you're not supervising your

139
00:06:03,039 --> 00:06:05,360
students. Rules may not be followed.

140
00:06:05,360 --> 00:06:07,120
Think you know your typical take-home

141
00:06:07,120 --> 00:06:08,720
project.

142
00:06:08,720 --> 00:06:10,479
Sometimes the lines between these are

143
00:06:10,479 --> 00:06:12,240
blurred like [clears throat] assessment

144
00:06:12,240 --> 00:06:14,240
in which students are given the

145
00:06:14,240 --> 00:06:17,039
questions in advance and can prepare but

146
00:06:17,039 --> 00:06:19,360
then must complete the task closed book

147
00:06:19,360 --> 00:06:21,520
under exam conditions

148
00:06:21,520 --> 00:06:24,240
and we can mix secure and open

149
00:06:24,240 --> 00:06:26,560
components in our assessment to help

150
00:06:26,560 --> 00:06:30,000
authenticate student work.

151
00:06:30,000 --> 00:06:32,880
Finally, academic integrity. Being

152
00:06:32,880 --> 00:06:35,440
honest about the work you submit.

153
00:06:35,440 --> 00:06:37,360
Adhering to agreed upon moral and

154
00:06:37,360 --> 00:06:39,600
ethical standards in order to maintain

155
00:06:39,600 --> 00:06:42,000
trust and be accurately compared against

156
00:06:42,000 --> 00:06:45,639
achievement standards.

157
00:06:46,960 --> 00:06:49,120
There's not a lot of data yet about how

158
00:06:49,120 --> 00:06:51,360
Australian students are using generative

159
00:06:51,360 --> 00:06:54,960
AI for their IT classes. The closest I

160
00:06:54,960 --> 00:06:58,960
think we've got is in 2025 um AARA took

161
00:06:58,960 --> 00:07:01,360
a representative sample of students from

162
00:07:01,360 --> 00:07:03,199
schools across Australia for their

163
00:07:03,199 --> 00:07:06,240
annual ICT literacy assessment and

164
00:07:06,240 --> 00:07:10,720
survey 4,700 year 10

165
00:07:10,720 --> 00:07:13,280
for the question at school I use

166
00:07:13,280 --> 00:07:15,919
artificial intelligence eg chat GBT

167
00:07:15,919 --> 00:07:17,840
Microsoft co-pilot to assist with school

168
00:07:17,840 --> 00:07:20,400
work 26%

169
00:07:20,400 --> 00:07:23,199
of year 10 students responded frequently

170
00:07:23,199 --> 00:07:28,240
ly 36 responded occasionally and 38%

171
00:07:28,240 --> 00:07:30,639
responded rarely.

172
00:07:30,639 --> 00:07:32,560
However,

173
00:07:32,560 --> 00:07:35,599
81% said they frequently use search

174
00:07:35,599 --> 00:07:38,080
engines and now that they almost all

175
00:07:38,080 --> 00:07:40,240
insist on putting generated answers at

176
00:07:40,240 --> 00:07:42,240
the top becomes harder to get a clear

177
00:07:42,240 --> 00:07:45,520
idea. Also, this is across all students,

178
00:07:45,520 --> 00:07:48,560
not just those taking IT subjects.

179
00:07:48,560 --> 00:07:51,199
Incidentally, in this assessment, only

180
00:07:51,199 --> 00:07:55,520
37% of year 10 students met or exceeded

181
00:07:55,520 --> 00:07:57,919
the proficient standard for ICT

182
00:07:57,919 --> 00:08:00,080
literacy. Um, so please take that into

183
00:08:00,080 --> 00:08:01,599
consideration when designing your

184
00:08:01,599 --> 00:08:04,599
courses.

185
00:08:04,960 --> 00:08:06,800
From the teachers I surveyed, the

186
00:08:06,800 --> 00:08:09,360
majority worked at institutions with a

187
00:08:09,360 --> 00:08:13,599
permissive AI policy, so allowed if the

188
00:08:13,599 --> 00:08:16,479
teacher chooses it for that assessment.

189
00:08:16,479 --> 00:08:19,680
Um, only three people had a blanket ban,

190
00:08:19,680 --> 00:08:23,440
including me at Yuk College.

191
00:08:23,440 --> 00:08:25,919
The Queensland Curriculum and Assessment

192
00:08:25,919 --> 00:08:28,720
Authority, QCAA, in their academic

193
00:08:28,720 --> 00:08:31,360
integrity toolkit, notes that if

194
00:08:31,360 --> 00:08:33,200
students are working on assignments with

195
00:08:33,200 --> 00:08:35,760
generative AI, it is important that they

196
00:08:35,760 --> 00:08:37,839
give credit and show what ideas are

197
00:08:37,839 --> 00:08:40,320
their own. They ask that schools

198
00:08:40,320 --> 00:08:43,519
explicitly teach the appropriate and

199
00:08:43,519 --> 00:08:46,480
responsible use of AI without giving

200
00:08:46,480 --> 00:08:48,240
guidance as to what they believe that

201
00:08:48,240 --> 00:08:52,839
is. It's a school-based decision.

202
00:08:53,680 --> 00:08:55,519
Interestingly, the majority of school

203
00:08:55,519 --> 00:08:57,440
teachers I spoke to have not made a

204
00:08:57,440 --> 00:08:59,360
shift towards having more secure

205
00:08:59,360 --> 00:09:01,760
assessment, but universities were the

206
00:09:01,760 --> 00:09:06,160
opposite. UKQ currently has a 30% secure

207
00:09:06,160 --> 00:09:08,720
assessment policy. Each course must

208
00:09:08,720 --> 00:09:10,720
contain secure assessment that counts

209
00:09:10,720 --> 00:09:13,360
for at least 30% of the grade. The

210
00:09:13,360 --> 00:09:15,040
University of Melbourne is even higher.

211
00:09:15,040 --> 00:09:17,279
They have a 50% secure assessment

212
00:09:17,279 --> 00:09:19,839
requirement. I was speaking to someone

213
00:09:19,839 --> 00:09:21,600
um that studies there in the hallway

214
00:09:21,600 --> 00:09:23,279
yesterday and he said all of his

215
00:09:23,279 --> 00:09:25,120
programming exams so far have been pen

216
00:09:25,120 --> 00:09:27,920
and paper. [snorts] There's also the

217
00:09:27,920 --> 00:09:30,800
concept of a hurdle. So a hurdle is a

218
00:09:30,800 --> 00:09:33,120
piece of assessment that must be passed

219
00:09:33,120 --> 00:09:35,279
in order to pass the course overall.

220
00:09:35,279 --> 00:09:38,399
typically a secure one. For example, in

221
00:09:38,399 --> 00:09:40,080
one of the courses I tutor, there are

222
00:09:40,080 --> 00:09:42,800
four pieces of assessment. Weekly tasks,

223
00:09:42,800 --> 00:09:44,399
two take-home projects, and an

224
00:09:44,399 --> 00:09:46,560
examination. If you do not pass the

225
00:09:46,560 --> 00:09:48,480
exam, no matter how well you did in

226
00:09:48,480 --> 00:09:50,080
those other three assessments, you fail

227
00:09:50,080 --> 00:09:53,320
the course.

228
00:09:53,920 --> 00:09:55,680
So, there are three general categories

229
00:09:55,680 --> 00:09:56,959
of rules you can set for your

230
00:09:56,959 --> 00:10:00,480
assessment. No AI allowed, AI strongly

231
00:10:00,480 --> 00:10:03,040
encouraged or even required, and AI

232
00:10:03,040 --> 00:10:05,519
permitted with attribution. So, let's go

233
00:10:05,519 --> 00:10:07,120
through some examples of assessment

234
00:10:07,120 --> 00:10:09,519
types and how those policies can play

235
00:10:09,519 --> 00:10:12,000
out.

236
00:10:12,000 --> 00:10:14,560
Examinations with no AI allowed are

237
00:10:14,560 --> 00:10:16,560
typically pen and paper with a time

238
00:10:16,560 --> 00:10:19,519
limit and direct supervision. In digital

239
00:10:19,519 --> 00:10:21,279
technologies education, they're often

240
00:10:21,279 --> 00:10:23,120
used for testing knowledge of theory

241
00:10:23,120 --> 00:10:26,000
rather than programming, but not always.

242
00:10:26,000 --> 00:10:29,279
Um, SQL is often assessed on paper and

243
00:10:29,279 --> 00:10:32,160
pseudo code is the bane of every year 12

244
00:10:32,160 --> 00:10:34,079
digital solutions students life here in

245
00:10:34,079 --> 00:10:35,920
Queensland.

246
00:10:35,920 --> 00:10:37,920
Um, but examinations don't have to be on

247
00:10:37,920 --> 00:10:40,399
paper. For a more authentic environment,

248
00:10:40,399 --> 00:10:42,720
lockdown browsers with an online IDE are

249
00:10:42,720 --> 00:10:44,880
popping up. I'm playing around with one

250
00:10:44,880 --> 00:10:46,640
called Code Grade at the moment, which

251
00:10:46,640 --> 00:10:48,640
seems like it could allow secure exams

252
00:10:48,640 --> 00:10:51,360
on a computer. If you're lucky enough to

253
00:10:51,360 --> 00:10:54,160
still have computer labs, there may uh

254
00:10:54,160 --> 00:10:56,160
there are plenty of examples of

255
00:10:56,160 --> 00:10:58,000
monitoring software that can allow the

256
00:10:58,000 --> 00:11:00,320
teacher to view student screens.

257
00:11:00,320 --> 00:11:01,839
Although, in my experience, when you've

258
00:11:01,839 --> 00:11:03,920
got a class of 28 and half of them don't

259
00:11:03,920 --> 00:11:06,720
know how to behave in exam conditions,

260
00:11:06,720 --> 00:11:10,800
uh any monitoring goes out the window.

261
00:11:10,800 --> 00:11:12,959
Interesting. Uh, interestingly, a

262
00:11:12,959 --> 00:11:15,519
literature review on the topic of paper

263
00:11:15,519 --> 00:11:17,279
verse computer-based testing in

264
00:11:17,279 --> 00:11:19,920
engineering and computing education

265
00:11:19,920 --> 00:11:21,839
um showed there's no clear evidence

266
00:11:21,839 --> 00:11:23,839
whether one is better or more accurate

267
00:11:23,839 --> 00:11:26,240
than the other. Uh, many studies

268
00:11:26,240 --> 00:11:27,839
reported that students preferred the

269
00:11:27,839 --> 00:11:30,800
computer-based exams, but not all did.

270
00:11:30,800 --> 00:11:33,200
In my last semester, I had to administer

271
00:11:33,200 --> 00:11:35,600
fortnite fortnightly low stakes

272
00:11:35,600 --> 00:11:37,519
paper-based exams to my beginning

273
00:11:37,519 --> 00:11:40,320
students. Although some students, hi

274
00:11:40,320 --> 00:11:42,880
Sophie, um were disappointed in the lack

275
00:11:42,880 --> 00:11:44,320
of actually programming things for

276
00:11:44,320 --> 00:11:46,160
assessment, many said they preferred

277
00:11:46,160 --> 00:11:48,079
exams, preferably if they're multiple

278
00:11:48,079 --> 00:11:50,959
choice, which did make me a bit sad. Uh

279
00:11:50,959 --> 00:11:54,240
and I'm going to change things anyway.

280
00:11:54,240 --> 00:11:56,720
Exams don't have to be no AI, though.

281
00:11:56,720 --> 00:11:58,880
Can use allow the use of a restrained

282
00:11:58,880 --> 00:12:01,760
chatbot in that code grade environment I

283
00:12:01,760 --> 00:12:03,760
was looking at. One option is you give

284
00:12:03,760 --> 00:12:06,320
students access to a clawed bot that you

285
00:12:06,320 --> 00:12:07,680
as the teacher give specific

286
00:12:07,680 --> 00:12:10,079
instructions to. Perhaps it's allowed to

287
00:12:10,079 --> 00:12:12,480
explain but not write code or it can

288
00:12:12,480 --> 00:12:15,440
show examples. Each interaction is then

289
00:12:15,440 --> 00:12:17,600
logged and submitted when the student

290
00:12:17,600 --> 00:12:20,240
hands in their exam.

291
00:12:20,240 --> 00:12:22,880
It is worth noting that exams are not

292
00:12:22,880 --> 00:12:26,079
the most accessible piece of assessment.

293
00:12:26,079 --> 00:12:27,519
There are students that struggle with

294
00:12:27,519 --> 00:12:30,560
exam anxiety and don't test well. those

295
00:12:30,560 --> 00:12:32,720
with difficulties handwriting or typing

296
00:12:32,720 --> 00:12:35,680
quickly. Although extra time and rest

297
00:12:35,680 --> 00:12:37,680
breaks can be given, it's typically

298
00:12:37,680 --> 00:12:39,920
quite an arduous process to get go

299
00:12:39,920 --> 00:12:41,760
through to get those approved and often

300
00:12:41,760 --> 00:12:44,399
requires formal medical documentation,

301
00:12:44,399 --> 00:12:48,120
especially in senior.

302
00:12:48,639 --> 00:12:50,560
Then we have the most open of assessment

303
00:12:50,560 --> 00:12:52,240
types, but the one that makes the most

304
00:12:52,240 --> 00:12:54,240
sense for a lot of our classes, the

305
00:12:54,240 --> 00:12:57,760
project. Whether it's just creating a

306
00:12:57,760 --> 00:13:00,480
single Python file or website in HTML or

307
00:13:00,480 --> 00:13:02,480
building a whole Flask application with

308
00:13:02,480 --> 00:13:05,440
APIs and a database like I love having

309
00:13:05,440 --> 00:13:08,399
students actually create something.

310
00:13:08,399 --> 00:13:11,120
So how can we make students care enough

311
00:13:11,120 --> 00:13:13,600
to not just put our project requirements

312
00:13:13,600 --> 00:13:16,880
into a prompt and stop there? The first

313
00:13:16,880 --> 00:13:19,600
thing I do is actually before students

314
00:13:19,600 --> 00:13:21,839
start the learning when we install

315
00:13:21,839 --> 00:13:24,800
Python and our IDE. I'm currently using

316
00:13:24,800 --> 00:13:27,839
VS Codium, which is the VS Code ID

317
00:13:27,839 --> 00:13:30,399
without Microsoft Telemetry. Um, I get

318
00:13:30,399 --> 00:13:31,760
students to turn off all the annoying

319
00:13:31,760 --> 00:13:34,240
hints. So, I provide a settings.json

320
00:13:34,240 --> 00:13:35,839
file that they all have to copy in

321
00:13:35,839 --> 00:13:37,360
before they know what it does. And it

322
00:13:37,360 --> 00:13:39,600
stops VS Code from autocompleting every

323
00:13:39,600 --> 00:13:41,279
file.

324
00:13:41,279 --> 00:13:42,959
But obviously, that's not an entire

325
00:13:42,959 --> 00:13:45,200
solution. The more I read and listen to

326
00:13:45,200 --> 00:13:47,040
educators, the more it's suggested that

327
00:13:47,040 --> 00:13:49,519
to combat plain copying and pasting, we

328
00:13:49,519 --> 00:13:51,279
need to make assessment real life and

329
00:13:51,279 --> 00:13:56,720
authentic. Um, I'm really uh excited to

330
00:13:56,720 --> 00:13:59,680
hear Stephen talk next about how he's

331
00:13:59,680 --> 00:14:01,279
improving project-based learning with

332
00:14:01,279 --> 00:14:04,079
continuous check-ins and peer feedback.

333
00:14:04,079 --> 00:14:08,639
Um, I spoke to Dr. Anna uh Odowska from

334
00:14:08,639 --> 00:14:10,560
ANU the other week and she was saying

335
00:14:10,560 --> 00:14:13,360
how in their student internship course

336
00:14:13,360 --> 00:14:15,600
the project pitch and final reflections

337
00:14:15,600 --> 00:14:17,839
were becoming so generic and clearly

338
00:14:17,839 --> 00:14:20,560
generated. So she flipped it so that the

339
00:14:20,560 --> 00:14:22,720
internship supervisors who could

340
00:14:22,720 --> 00:14:24,560
potentially offer them a job at the end

341
00:14:24,560 --> 00:14:27,040
had a say in marking their assessment.

342
00:14:27,040 --> 00:14:28,320
Suddenly they didn't want to make

343
00:14:28,320 --> 00:14:30,160
something generic and mediocre cuz they

344
00:14:30,160 --> 00:14:32,160
had a real stake in the outcome of the

345
00:14:32,160 --> 00:14:35,745
assessment. But this is so hard.

346
00:14:35,745 --> 00:14:37,360
[laughter] I've been trying to make

347
00:14:37,360 --> 00:14:39,279
authentic assessment that excites

348
00:14:39,279 --> 00:14:41,120
students my whole career and I've had

349
00:14:41,120 --> 00:14:43,440
more failures than successes. It's a lot

350
00:14:43,440 --> 00:14:45,360
of work and it requires buyin from

351
00:14:45,360 --> 00:14:48,560
management and your school community.

352
00:14:48,560 --> 00:14:50,800
Another strategy with projects is to set

353
00:14:50,800 --> 00:14:52,560
complex tasks with a lot of moving

354
00:14:52,560 --> 00:14:55,120
parts, including like robots, Raspberry

355
00:14:55,120 --> 00:14:57,920
Pies, having a web app with a set file

356
00:14:57,920 --> 00:15:00,079
structure and specific helper functions

357
00:15:00,079 --> 00:15:03,120
you've written that must be used. One

358
00:15:03,120 --> 00:15:05,360
teacher said, "I'm taking time to teach

359
00:15:05,360 --> 00:15:07,519
students the use of GitHub as a means to

360
00:15:07,519 --> 00:15:09,360
versioning their code so that when AI

361
00:15:09,360 --> 00:15:10,800
screws up, they always have something to

362
00:15:10,800 --> 00:15:13,120
fall back on. The year 12 assignments

363
00:15:13,120 --> 00:15:15,040
are of sufficient complexity that even

364
00:15:15,040 --> 00:15:17,199
with the use of AI, I know students have

365
00:15:17,199 --> 00:15:20,720
to work. Um, the environments consist of

366
00:15:20,720 --> 00:15:23,040
different code and system components

367
00:15:23,040 --> 00:15:25,279
that AI struggles to fully comprehend.

368
00:15:25,279 --> 00:15:27,120
So, even with the use of AI, students

369
00:15:27,120 --> 00:15:28,399
need to know what they're doing to

370
00:15:28,399 --> 00:15:31,519
complete their assignments.

371
00:15:31,519 --> 00:15:33,279
Something I'm liking in my own teaching

372
00:15:33,279 --> 00:15:35,839
is the interview. After students submit

373
00:15:35,839 --> 00:15:37,199
their project, they have an interview

374
00:15:37,199 --> 00:15:39,920
with me to check, do they understand the

375
00:15:39,920 --> 00:15:41,600
code? Can you show me where this

376
00:15:41,600 --> 00:15:43,519
function is defined? What does this bit

377
00:15:43,519 --> 00:15:47,839
of SQL do? To analyze if I change this,

378
00:15:47,839 --> 00:15:50,000
what would happen? Why did you make this

379
00:15:50,000 --> 00:15:52,880
decision over that one? And to evaluate,

380
00:15:52,880 --> 00:15:54,560
what are the weaknesses of your project?

381
00:15:54,560 --> 00:15:57,519
What would you change? If the interview

382
00:15:57,519 --> 00:15:59,279
is worth as many marks as the project

383
00:15:59,279 --> 00:16:01,680
itself or is a hurdle that must be

384
00:16:01,680 --> 00:16:03,920
passed, it can be a good proxy for

385
00:16:03,920 --> 00:16:06,560
detecting mindless AI use and checking

386
00:16:06,560 --> 00:16:08,639
understanding against the achievement

387
00:16:08,639 --> 00:16:10,160
standards.

388
00:16:10,160 --> 00:16:13,040
But it's not without challenges. Time.

389
00:16:13,040 --> 00:16:15,120
Even a five-minute interview with your

390
00:16:15,120 --> 00:16:17,759
entire year 8 class is a whole week of

391
00:16:17,759 --> 00:16:19,839
lessons just talking to students and

392
00:16:19,839 --> 00:16:21,440
hoping that the other students aren't

393
00:16:21,440 --> 00:16:23,040
killing each other whilst you're not

394
00:16:23,040 --> 00:16:26,240
paying attention to them. It also dis

395
00:16:26,240 --> 00:16:28,079
disadvantages English as an additional

396
00:16:28,079 --> 00:16:30,240
language. Students or anyone who spoken

397
00:16:30,240 --> 00:16:32,639
English may not allow them to explain

398
00:16:32,639 --> 00:16:36,120
complex concepts.

399
00:16:36,720 --> 00:16:38,959
If you are allowing the use of LLMs,

400
00:16:38,959 --> 00:16:41,440
I've seen a lot of uh courses use an

401
00:16:41,440 --> 00:16:43,839
acknowledgement statement. There's ones

402
00:16:43,839 --> 00:16:46,240
that require every prompt be written

403
00:16:46,240 --> 00:16:48,800
down or make students delve into the

404
00:16:48,800 --> 00:16:51,199
reasons for prompting, asking questions

405
00:16:51,199 --> 00:16:53,519
like, "How did you check the output?

406
00:16:53,519 --> 00:16:55,360
What key decisions did you make along

407
00:16:55,360 --> 00:16:57,680
the way? Why did you use the tools that

408
00:16:57,680 --> 00:17:00,839
you did?"

409
00:17:02,160 --> 00:17:05,120
I would warn about forcing the use of

410
00:17:05,120 --> 00:17:08,400
code generators around my workplace. I'm

411
00:17:08,400 --> 00:17:10,319
seeing posters pop up for students

412
00:17:10,319 --> 00:17:12,319
rejecting slop. And I think that

413
00:17:12,319 --> 00:17:14,079
captures the feeling of many students

414
00:17:14,079 --> 00:17:17,280
towards their teachers using AI. I tutor

415
00:17:17,280 --> 00:17:19,280
students who are veently against any

416
00:17:19,280 --> 00:17:21,520
sort of AI. Often students that are very

417
00:17:21,520 --> 00:17:23,280
creative and can see how it's harming

418
00:17:23,280 --> 00:17:25,760
the industries they care about. They are

419
00:17:25,760 --> 00:17:28,319
ruthless towards teachers they suspect

420
00:17:28,319 --> 00:17:30,960
of using AI for resources, for marking

421
00:17:30,960 --> 00:17:33,679
and feedback or for coding uh simple

422
00:17:33,679 --> 00:17:36,000
projects. You know, as Cam spoke about

423
00:17:36,000 --> 00:17:39,600
this morning, Australians are skeptical.

424
00:17:39,600 --> 00:17:41,600
A quote from a data science student I

425
00:17:41,600 --> 00:17:44,640
spoke to said, I find it incredibly

426
00:17:44,640 --> 00:17:46,480
frustrating when we're asked to do

427
00:17:46,480 --> 00:17:48,559
significantly harder assessments with

428
00:17:48,559 --> 00:17:50,960
the expectation that AI will be used

429
00:17:50,960 --> 00:17:52,720
because we'll fall behind if we're not

430
00:17:52,720 --> 00:17:55,600
proficient in its use. Being explicitly

431
00:17:55,600 --> 00:17:57,679
told to use AI to write all of our

432
00:17:57,679 --> 00:17:59,840
functional testing meant I didn't have

433
00:17:59,840 --> 00:18:01,760
the space to properly learn how test

434
00:18:01,760 --> 00:18:05,720
scripts were supposed to work.

435
00:18:07,120 --> 00:18:08,799
I also think it's important to talk

436
00:18:08,799 --> 00:18:11,120
about the pedagogical impacts of using

437
00:18:11,120 --> 00:18:14,320
AI assistance. There's not a whole lot

438
00:18:14,320 --> 00:18:17,840
of good quality research yet. Obviously,

439
00:18:17,840 --> 00:18:20,559
it's very new. Uh but I think we can

440
00:18:20,559 --> 00:18:23,760
learn from calculators. In the 1980s,

441
00:18:23,760 --> 00:18:26,559
maths teachers were terrified. Students

442
00:18:26,559 --> 00:18:28,799
now had easy access to this technology

443
00:18:28,799 --> 00:18:31,120
that would do the work for them. No

444
00:18:31,120 --> 00:18:33,120
longer would students bother memorizing

445
00:18:33,120 --> 00:18:36,960
times tables. Maths education is doomed.

446
00:18:36,960 --> 00:18:39,760
Um, we know that's not entirely true.

447
00:18:39,760 --> 00:18:42,000
Calculators actually allow students to

448
00:18:42,000 --> 00:18:43,440
solve problems that they wouldn't be

449
00:18:43,440 --> 00:18:45,600
able to otherwise. They can focus on

450
00:18:45,600 --> 00:18:47,200
real world problems where numbers are

451
00:18:47,200 --> 00:18:49,440
messy without getting bogged down in

452
00:18:49,440 --> 00:18:52,720
tedious calculations.

453
00:18:52,720 --> 00:18:55,679
However, we also know that calculators

454
00:18:55,679 --> 00:18:58,400
can become a crutch. I know as a math

455
00:18:58,400 --> 00:19:00,640
teacher, I've witnessed so many students

456
00:19:00,640 --> 00:19:02,559
just automatically putting equations

457
00:19:02,559 --> 00:19:06,400
into their calculator 2 * 5 without

458
00:19:06,400 --> 00:19:08,480
thinking. So, when they put the wrong

459
00:19:08,480 --> 00:19:11,120
inputs in, a plus instead of a multiply,

460
00:19:11,120 --> 00:19:12,880
a six instead of a five, they don't

461
00:19:12,880 --> 00:19:15,840
question the answer.

462
00:19:15,840 --> 00:19:19,760
A 2019 study with university students

463
00:19:19,760 --> 00:19:22,080
used calculators for simple maths

464
00:19:22,080 --> 00:19:24,880
problems, but some groups had a lying

465
00:19:24,880 --> 00:19:26,559
calculator that was programmed to

466
00:19:26,559 --> 00:19:29,200
sometimes give the wrong answer.

467
00:19:29,200 --> 00:19:31,039
They showed that most students didn't

468
00:19:31,039 --> 00:19:32,960
become suspicious and just believed

469
00:19:32,960 --> 00:19:34,880
whatever the calculator said, even when

470
00:19:34,880 --> 00:19:37,440
the answer really didn't make sense, and

471
00:19:37,440 --> 00:19:39,360
most reached for the calculator instead

472
00:19:39,360 --> 00:19:42,160
of doing it in their head.

473
00:19:42,160 --> 00:19:44,480
This is, I think, something to be really

474
00:19:44,480 --> 00:19:46,640
careful about when using coding

475
00:19:46,640 --> 00:19:48,240
generators because they are lying

476
00:19:48,240 --> 00:19:50,880
calculators. They don't always give the

477
00:19:50,880 --> 00:19:52,480
right answer. And even if the student

478
00:19:52,480 --> 00:19:54,799
knows that and knows, you know, that

479
00:19:54,799 --> 00:19:55,919
they're supposed to validate the

480
00:19:55,919 --> 00:19:58,559
solution without being prompted by you

481
00:19:58,559 --> 00:20:00,480
to check for mistakes, they may not

482
00:20:00,480 --> 00:20:02,880
think to question

483
00:20:02,880 --> 00:20:05,039
using a particular technology in your

484
00:20:05,039 --> 00:20:07,039
classroom. like it has to be done with

485
00:20:07,039 --> 00:20:08,960
teacher guidance to learn how to make

486
00:20:08,960 --> 00:20:11,280
decisions about when and how that

487
00:20:11,280 --> 00:20:13,760
technology is appropriate.

488
00:20:13,760 --> 00:20:17,039
You know, we don't give calculators to

489
00:20:17,039 --> 00:20:18,400
primary school students when they're

490
00:20:18,400 --> 00:20:21,520
first uh learning how to add, subtract,

491
00:20:21,520 --> 00:20:23,600
multiply.

492
00:20:23,600 --> 00:20:24,960
When we don't give students the

493
00:20:24,960 --> 00:20:27,200
opportunity to learn mental maths, we

494
00:20:27,200 --> 00:20:28,799
know it stunts them when they hit

495
00:20:28,799 --> 00:20:33,240
algebra later and have to abstract.

496
00:20:35,120 --> 00:20:37,679
Professor Joe Bowler, a mathematics

497
00:20:37,679 --> 00:20:40,080
educator out of Stanford University,

498
00:20:40,080 --> 00:20:42,320
published a paper about the rights of

499
00:20:42,320 --> 00:20:45,200
students in a classroom.

500
00:20:45,200 --> 00:20:46,799
Two of them were the right to be

501
00:20:46,799 --> 00:20:48,799
confused and the right to make a

502
00:20:48,799 --> 00:20:51,799
mistake.

503
00:20:54,400 --> 00:20:56,320
making mistakes,

504
00:20:56,320 --> 00:20:58,720
getting confused, and pushing through

505
00:20:58,720 --> 00:21:01,280
these challenges to find a solution is

506
00:21:01,280 --> 00:21:03,520
known to be important uh to the process

507
00:21:03,520 --> 00:21:05,440
of learning. We've got neuroscience to

508
00:21:05,440 --> 00:21:09,280
back it up. Like think of Pia, all my

509
00:21:09,280 --> 00:21:11,440
education [laughter]

510
00:21:11,440 --> 00:21:13,919
educators, learners don't just absorb

511
00:21:13,919 --> 00:21:15,440
knowledge from looking at the right

512
00:21:15,440 --> 00:21:18,240
answer. You know, anyone who's tried to

513
00:21:18,240 --> 00:21:19,840
study maths from just reading the

514
00:21:19,840 --> 00:21:21,840
textbook, like it does nothing. You need

515
00:21:21,840 --> 00:21:25,120
hands-on interaction. And we all know

516
00:21:25,120 --> 00:21:26,960
the feeling of being stuck on a

517
00:21:26,960 --> 00:21:28,720
programming problem and scouring

518
00:21:28,720 --> 00:21:31,039
documentation and it's not working, but

519
00:21:31,039 --> 00:21:32,640
then you try something and it does work

520
00:21:32,640 --> 00:21:35,120
and you feel like a god. It's the best

521
00:21:35,120 --> 00:21:37,520
feeling and it's why I grew to love

522
00:21:37,520 --> 00:21:42,400
being a programmer. So, I think uh it's

523
00:21:42,400 --> 00:21:44,960
it's important that we that we keep

524
00:21:44,960 --> 00:21:48,000
letting students struggle.

525
00:21:48,000 --> 00:21:51,840
I realize I've talked very fast and gone

526
00:21:51,840 --> 00:21:54,080
through uh all my slides in twice the

527
00:21:54,080 --> 00:21:57,679
time. Um but I would like to end by

528
00:21:57,679 --> 00:22:00,240
quoting Professor Roberto Serrano from

529
00:22:00,240 --> 00:22:02,799
Brown University. He's currently

530
00:22:02,799 --> 00:22:05,200
rewriting his economics assessment after

531
00:22:05,200 --> 00:22:09,120
mass AI fraud in take-home exams. He

532
00:22:09,120 --> 00:22:12,320
says academic integrity is a value worth

533
00:22:12,320 --> 00:22:14,080
defending.

534
00:22:14,080 --> 00:22:16,320
A common thread I've noticed in AI

535
00:22:16,320 --> 00:22:18,480
education discourse is that it's too

536
00:22:18,480 --> 00:22:21,679
hard to police AI usage. We can't stop

537
00:22:21,679 --> 00:22:24,080
it. So don't even bother disallowing its

538
00:22:24,080 --> 00:22:26,159
use.

539
00:22:26,159 --> 00:22:29,039
I want to challenge that and say that I

540
00:22:29,039 --> 00:22:31,679
believe academic integrity is still a

541
00:22:31,679 --> 00:22:34,240
value worth defending.

542
00:22:34,240 --> 00:22:36,559
It's hard to catch plagiarism and

543
00:22:36,559 --> 00:22:38,880
contract cheating and collusion, but

544
00:22:38,880 --> 00:22:40,640
that doesn't mean that we give up and

545
00:22:40,640 --> 00:22:43,120
just say, "Okay, I guess you can." We

546
00:22:43,120 --> 00:22:44,880
write assessment that prohibits their

547
00:22:44,880 --> 00:22:47,280
use, and we take steps to verify that

548
00:22:47,280 --> 00:22:50,000
students understand their work.

549
00:22:50,000 --> 00:22:52,880
Yes, we're going to miss cases, but I

550
00:22:52,880 --> 00:22:54,640
still think it's worth having something

551
00:22:54,640 --> 00:22:56,720
to point to when you do have students

552
00:22:56,720 --> 00:22:58,720
that you can just tell haven't genuinely

553
00:22:58,720 --> 00:23:00,159
demonstrated the skills or knowledge

554
00:23:00,159 --> 00:23:02,640
that you're assessing them on. As

555
00:23:02,640 --> 00:23:04,720
education becomes a business, schools

556
00:23:04,720 --> 00:23:06,799
need 100% of students graduating,

557
00:23:06,799 --> 00:23:09,360
universities need 90% satisfaction rate

558
00:23:09,360 --> 00:23:11,679
on surveys. It's hard as a teacher to

559
00:23:11,679 --> 00:23:13,360
feel like you have the power to enforce

560
00:23:13,360 --> 00:23:16,080
this. But I believe that as educators,

561
00:23:16,080 --> 00:23:18,640
we have to push our students to think,

562
00:23:18,640 --> 00:23:22,000
to create, and to be human, to push back

563
00:23:22,000 --> 00:23:23,679
against the fascist technology that

564
00:23:23,679 --> 00:23:27,679
wants you to pay a subscription to code.

565
00:23:27,679 --> 00:23:30,880
Um, thank you. You can find me on the

566
00:23:30,880 --> 00:23:33,600
Fettyiverse. uh at miss theossaurus uh

567
00:23:33,600 --> 00:23:36,159
atbn.social. Uh and I haven't put the

568
00:23:36,159 --> 00:23:37,520
slides up anywhere because I'm too

569
00:23:37,520 --> 00:23:39,360
afraid to host my own website anymore

570
00:23:39,360 --> 00:23:41,840
thanks to scrapers. A massive thanks to

571
00:23:41,840 --> 00:23:43,360
all the teachers and students that took

572
00:23:43,360 --> 00:23:44,960
the time to speak with me and fill out

573
00:23:44,960 --> 00:23:47,919
my survey. Credit to Momo for the

574
00:23:47,919 --> 00:23:49,520
adorable theossaurus drawing I

575
00:23:49,520 --> 00:23:51,919
commissioned. Uh and Manny and I run the

576
00:23:51,919 --> 00:23:53,840
Brisbane Python meetups. We meet every

577
00:23:53,840 --> 00:23:55,280
month. Last Thursday of the month at the

578
00:23:55,280 --> 00:23:57,280
precinct in the Valley. Find us on Luma.

579
00:23:57,280 --> 00:23:59,360
Our next event 24th of September at 6

580
00:23:59,360 --> 00:24:02,880
p.m. We still don't have a speaker.

581
00:24:02,981 --> 00:24:05,001
[applause]

582
00:24:08,640 --> 00:24:11,279
Thank you so much, Thea. Here is your

583
00:24:11,279 --> 00:24:12,799
speaker's gift. Can we give her another

584
00:24:12,799 --> 00:24:16,136
round of applause, please? [applause]

585
00:24:18,799 --> 00:24:20,640
Uh, just before we go into our next

586
00:24:20,640 --> 00:24:22,880
speaker, I'm going to remind people in

587
00:24:22,880 --> 00:24:24,559
the room that there is a competition

588
00:24:24,559 --> 00:24:27,120
open from our wonderful sponsors, GPN,

589
00:24:27,120 --> 00:24:28,640
and that is going to be drawn at the

590
00:24:28,640 --> 00:24:30,480
student showcase. So, if you want to get

591
00:24:30,480 --> 00:24:32,960
a chance to win yourself some fabulous

592
00:24:32,960 --> 00:24:35,120
Lego, you should definitely get on that

593
00:24:35,120 --> 00:24:36,960
competition right now. I don't believe

594
00:24:36,960 --> 00:24:40,720
we have any student entries.

595
00:24:40,720 --> 00:24:42,559
Okay. [laughter]

596
00:24:42,559 --> 00:24:45,557
All right. Thank you again, Theo.

597
00:24:45,557 --> 00:24:47,577
[applause]