1 00:00:04,960 --> 00:00:20,000 [Music] 2 00:00:20,600 --> 00:00:25,720 welcome back everybody and for our next 3 00:00:22,800 --> 00:00:27,960 speaker we have Dr Steven tiny who works 4 00:00:25,720 --> 00:00:30,000 for the University of Sydney with 5 00:00:27,960 --> 00:00:31,519 Allison Wong who got a place on the 6 00:00:30,000 --> 00:00:35,200 slide even though she's decided to stay 7 00:00:31,519 --> 00:00:37,280 in her seat today um yeah so uh he works 8 00:00:35,200 --> 00:00:39,600 in the School of Business uh teaches 9 00:00:37,280 --> 00:00:41,520 python staff he's going to talk bit 10 00:00:39,600 --> 00:00:43,719 about I don't actually know how to say 11 00:00:41,520 --> 00:00:45,440 it is it Carell or Carol it Carol I call 12 00:00:43,719 --> 00:00:48,320 it Carol too okay I'm right I'm right 13 00:00:45,440 --> 00:00:49,559 everybody it's fine um okay so he's 14 00:00:48,320 --> 00:00:53,079 going to talk about playful learning 15 00:00:49,559 --> 00:00:53,079 with Carol thank you thank you 16 00:00:54,800 --> 00:00:59,160 Stephen okay so our talk is enhancing 17 00:00:57,520 --> 00:01:02,320 programming ability with playful 18 00:00:59,160 --> 00:01:03,519 learning and Carol um as you can see 19 00:01:02,320 --> 00:01:05,479 there's already a little bit of a demo 20 00:01:03,519 --> 00:01:08,320 going on with Carol um and I'll show 21 00:01:05,479 --> 00:01:11,400 some more demos of Carol in action later 22 00:01:08,320 --> 00:01:13,640 uh this is a now turned into a larger 23 00:01:11,400 --> 00:01:15,479 scale research project and it's been 24 00:01:13,640 --> 00:01:17,040 something we've been working on uh since 25 00:01:15,479 --> 00:01:19,840 the start of the 26 00:01:17,040 --> 00:01:22,159 year um and to give you some context uh 27 00:01:19,840 --> 00:01:24,159 so I work in the business school uh in 28 00:01:22,159 --> 00:01:27,360 the department or discipline of business 29 00:01:24,159 --> 00:01:29,479 analytics uh so we focus on teaching 30 00:01:27,360 --> 00:01:32,119 statistics sta uh data science 31 00:01:29,479 --> 00:01:33,560 optimizing ation data visualization uh 32 00:01:32,119 --> 00:01:36,840 within a business 33 00:01:33,560 --> 00:01:39,240 context uh and our students um both 34 00:01:36,840 --> 00:01:41,000 Allison and mine uh are at quite 35 00:01:39,240 --> 00:01:43,320 different levels we teach undergraduate 36 00:01:41,000 --> 00:01:45,000 and postgraduate students however in 37 00:01:43,320 --> 00:01:47,200 both contexts most students don't come 38 00:01:45,000 --> 00:01:50,479 into these degree programs with any 39 00:01:47,200 --> 00:01:52,159 python or programming experience uh and 40 00:01:50,479 --> 00:01:54,079 we also have a bit of a time crunch 41 00:01:52,159 --> 00:01:56,240 where teaching we usually only have the 42 00:01:54,079 --> 00:01:58,880 students for 13 weeks and we only have 43 00:01:56,240 --> 00:02:00,439 them a couple of hours each week so we 44 00:01:58,880 --> 00:02:02,880 have very limited time to teach these 45 00:02:00,439 --> 00:02:04,280 students programming alongside all the 46 00:02:02,880 --> 00:02:06,159 other content that we have to try and 47 00:02:04,280 --> 00:02:09,399 teach them about the statistics and the 48 00:02:06,159 --> 00:02:10,759 data science too the theory behind it so 49 00:02:09,399 --> 00:02:12,920 we have a limited time to teach and 50 00:02:10,759 --> 00:02:15,120 learn skills which are slowly acquired 51 00:02:12,920 --> 00:02:16,840 so we know that for most people 52 00:02:15,120 --> 00:02:19,800 acquiring programming as a skill is 53 00:02:16,840 --> 00:02:21,360 something that takes time and practice 54 00:02:19,800 --> 00:02:22,800 so things that might be slowly acquired 55 00:02:21,360 --> 00:02:25,080 are understanding syntax or learning a 56 00:02:22,800 --> 00:02:26,920 language is is hard understanding how 57 00:02:25,080 --> 00:02:28,920 program execution and how to trace 58 00:02:26,920 --> 00:02:31,599 through a program that's a skill that 59 00:02:28,920 --> 00:02:33,720 can also take time to develop and then 60 00:02:31,599 --> 00:02:35,680 there's the third stage of understanding 61 00:02:33,720 --> 00:02:38,800 and developing a model of program state 62 00:02:35,680 --> 00:02:40,120 in your mind so those things I really 63 00:02:38,800 --> 00:02:41,519 want to enforce that they they take a 64 00:02:40,120 --> 00:02:43,120 long time to learn and we have a very 65 00:02:41,519 --> 00:02:45,560 limited time to teach 66 00:02:43,120 --> 00:02:47,400 those uh and the result of this is that 67 00:02:45,560 --> 00:02:49,319 we end up with students with relatively 68 00:02:47,400 --> 00:02:51,120 weak computational thinking skills so we 69 00:02:49,319 --> 00:02:53,239 want them to improve or do better than 70 00:02:51,120 --> 00:02:55,000 what they're currently doing um often 71 00:02:53,239 --> 00:02:56,879 they are unable to write code 72 00:02:55,000 --> 00:02:58,120 independently uh even though they are 73 00:02:56,879 --> 00:02:59,879 exposed to and see a lot of code 74 00:02:58,120 --> 00:03:02,080 throughout a semester some of them 75 00:02:59,879 --> 00:03:03,799 struggle to do this by themselves and 76 00:03:02,080 --> 00:03:06,080 often they're unable to debug their own 77 00:03:03,799 --> 00:03:09,080 code that they've 78 00:03:06,080 --> 00:03:10,840 written so what we are uh trying to do 79 00:03:09,080 --> 00:03:13,080 is encourage greater participation with 80 00:03:10,840 --> 00:03:14,840 programming outside of the classroom as 81 00:03:13,080 --> 00:03:16,720 an extracurricular activity and make 82 00:03:14,840 --> 00:03:19,159 things more fun and engaging for the 83 00:03:16,720 --> 00:03:20,879 students um so that they can see that 84 00:03:19,159 --> 00:03:23,120 okay programming is not just something 85 00:03:20,879 --> 00:03:24,920 that they do to uh help them complete 86 00:03:23,120 --> 00:03:26,480 their homework and their assignments but 87 00:03:24,920 --> 00:03:28,159 is actually an enjoyable activity that 88 00:03:26,480 --> 00:03:29,879 has broader applications beyond the 89 00:03:28,159 --> 00:03:32,040 classroom 90 00:03:29,879 --> 00:03:33,879 uh and we've decided to um sort of 91 00:03:32,040 --> 00:03:35,799 promote this engagement with programming 92 00:03:33,879 --> 00:03:38,280 by introducing an extracurricular 93 00:03:35,799 --> 00:03:41,000 activity with Carol and Carol is a 94 00:03:38,280 --> 00:03:44,280 simplified programming environment it we 95 00:03:41,000 --> 00:03:46,560 use a it uses a subset of python uh and 96 00:03:44,280 --> 00:03:48,959 the environment of Carol is a 2d grid 97 00:03:46,560 --> 00:03:51,120 world and you move Carol around this 98 00:03:48,959 --> 00:03:53,079 grid world uh and interact with the grid 99 00:03:51,120 --> 00:03:55,519 World um through 100 00:03:53,079 --> 00:03:59,000 Carol and it's all code based so 101 00:03:55,519 --> 00:04:01,239 students control Carol only with code uh 102 00:03:59,000 --> 00:04:03,040 and the point of this is that students 103 00:04:01,239 --> 00:04:05,879 can develop their computational thinking 104 00:04:03,040 --> 00:04:07,840 skills by understanding the concepts of 105 00:04:05,879 --> 00:04:09,760 a program rather than trying to 106 00:04:07,840 --> 00:04:11,720 understand really complicated 107 00:04:09,760 --> 00:04:13,920 syntax there's also immediate 108 00:04:11,720 --> 00:04:15,280 visualization of the program state so 109 00:04:13,920 --> 00:04:17,160 all parts of the program state are 110 00:04:15,280 --> 00:04:18,880 actually in the the Carol world there's 111 00:04:17,160 --> 00:04:20,959 no way to create things like variables 112 00:04:18,880 --> 00:04:23,560 so everything actually is visible in the 113 00:04:20,959 --> 00:04:25,639 2D space uh and as a result there's 114 00:04:23,560 --> 00:04:27,759 reduced need to maintain a mental model 115 00:04:25,639 --> 00:04:29,199 so students can sort of defer or 116 00:04:27,759 --> 00:04:31,759 Outsource their mental model to the 117 00:04:29,199 --> 00:04:33,759 Carol world and they don't need to sort 118 00:04:31,759 --> 00:04:34,919 of as they're writing code think about 119 00:04:33,759 --> 00:04:36,600 or as they're actually testing their 120 00:04:34,919 --> 00:04:39,039 code think about what's happening inside 121 00:04:36,600 --> 00:04:42,280 their program because they can just see 122 00:04:39,039 --> 00:04:43,919 it um okay so I've told you what Carol 123 00:04:42,280 --> 00:04:48,080 is in theory so let's take a look at 124 00:04:43,919 --> 00:04:51,000 some examples of Carol hopefully this is 125 00:04:48,080 --> 00:04:54,080 working let's see if I can zoom this in 126 00:04:51,000 --> 00:04:55,280 so we teach primarily on Ed let's see if 127 00:04:54,080 --> 00:04:56,240 I can make that font a bit bigger for 128 00:04:55,280 --> 00:05:00,120 you 129 00:04:56,240 --> 00:05:01,759 all uh and we have a set of I'll provide 130 00:05:00,120 --> 00:05:04,880 the students with a set of information 131 00:05:01,759 --> 00:05:06,320 uh on Ed about what Carol is and how to 132 00:05:04,880 --> 00:05:08,720 use it through these slides you can see 133 00:05:06,320 --> 00:05:10,639 on the left hand side I'll start off 134 00:05:08,720 --> 00:05:12,320 with some basics of Carol uh we give 135 00:05:10,639 --> 00:05:14,639 them instructions about how they can 136 00:05:12,320 --> 00:05:16,639 move Carol around the world and a nice 137 00:05:14,639 --> 00:05:20,160 thing about Ed is that you can have 138 00:05:16,639 --> 00:05:22,280 these uh integrated uh and runnable code 139 00:05:20,160 --> 00:05:24,680 Snippets so for example I can run this 140 00:05:22,280 --> 00:05:26,600 code here um you define your Carol 141 00:05:24,680 --> 00:05:27,840 program by defining a main function and 142 00:05:26,600 --> 00:05:29,360 then just put all your instructions in 143 00:05:27,840 --> 00:05:31,080 there and later on you can Define your 144 00:05:29,360 --> 00:05:34,880 own other functions which you can call 145 00:05:31,080 --> 00:05:37,960 so that will just move Carol uh in two 146 00:05:34,880 --> 00:05:39,960 steps to the right uh and then Carol has 147 00:05:37,960 --> 00:05:41,960 a very limited subset of instru set of 148 00:05:39,960 --> 00:05:46,360 instructions that you can do so here we 149 00:05:41,960 --> 00:05:48,400 can turn left and then move and move uh 150 00:05:46,360 --> 00:05:50,360 and so we step the students through 151 00:05:48,400 --> 00:05:51,479 these basic instructions to hopefully 152 00:05:50,360 --> 00:05:52,240 get to a point where we can do something 153 00:05:51,479 --> 00:05:54,360 more 154 00:05:52,240 --> 00:05:55,560 interesting Carol can also interact with 155 00:05:54,360 --> 00:05:57,400 these things called beepers which are 156 00:05:55,560 --> 00:05:59,680 little objects that can be placed on 157 00:05:57,400 --> 00:06:01,280 each of the tiles so we can place one 158 00:05:59,680 --> 00:06:02,800 beeper down put another beeper down and 159 00:06:01,280 --> 00:06:05,840 we can see that there are two beepers 160 00:06:02,800 --> 00:06:07,680 now placed in the world um and so 161 00:06:05,840 --> 00:06:10,319 basically we can represent rather 162 00:06:07,680 --> 00:06:13,120 complex program state in these 2D grid 163 00:06:10,319 --> 00:06:16,440 environments with only beepers and also 164 00:06:13,120 --> 00:06:20,800 sometimes by coloring some of these 165 00:06:16,440 --> 00:06:23,919 environments um okay so now I'll move on 166 00:06:20,800 --> 00:06:26,639 to uh a more complicated example which 167 00:06:23,919 --> 00:06:29,759 we'll do try and do live I 168 00:06:26,639 --> 00:06:32,240 think have we hidden the Carol exercises 169 00:06:29,759 --> 00:06:34,039 wrong wrong Oh wrong I'm on 170 00:06:32,240 --> 00:06:37,720 challenges yep all right so let's try 171 00:06:34,039 --> 00:06:39,280 the Beeper drop exercise so this is a a 172 00:06:37,720 --> 00:06:41,360 an exercise that we give the students 173 00:06:39,280 --> 00:06:42,560 then as part of these this 174 00:06:41,360 --> 00:06:45,319 extracurricular 175 00:06:42,560 --> 00:06:47,800 activity uh we give them a program sort 176 00:06:45,319 --> 00:06:50,520 of goal on the left hand side and then 177 00:06:47,800 --> 00:06:52,120 give them a gift to to show them uh what 178 00:06:50,520 --> 00:06:54,520 this might look like when their program 179 00:06:52,120 --> 00:06:56,160 actually runs and so for the challenge 180 00:06:54,520 --> 00:06:57,840 for the students is then to write their 181 00:06:56,160 --> 00:07:00,039 code over on the right hand side in the 182 00:06:57,840 --> 00:07:02,400 code editor and then they can try and 183 00:07:00,039 --> 00:07:06,440 run their code if they like so if we 184 00:07:02,400 --> 00:07:10,720 want to do that we can then say 185 00:07:06,440 --> 00:07:13,680 move uh and then we can probably turn 186 00:07:10,720 --> 00:07:17,280 left and then move 187 00:07:13,680 --> 00:07:19,280 again and so on and I click the Run 188 00:07:17,280 --> 00:07:21,199 button and it will very quickly execute 189 00:07:19,280 --> 00:07:24,319 I have set the speed way too fast for 190 00:07:21,199 --> 00:07:25,599 Carol um that's a useful feature on Ed 191 00:07:24,319 --> 00:07:26,479 you can actually speed up car running so 192 00:07:25,599 --> 00:07:28,599 when you have a really long running 193 00:07:26,479 --> 00:07:30,080 program you can make it move really fast 194 00:07:28,599 --> 00:07:31,080 uh and then students don't get to see 195 00:07:30,080 --> 00:07:33,199 this but I'm just going to grab the 196 00:07:31,080 --> 00:07:34,160 solution over on the uh from the 197 00:07:33,199 --> 00:07:36,960 solution 198 00:07:34,160 --> 00:07:39,319 page and I will try running my code make 199 00:07:36,960 --> 00:07:41,879 sure that it works we place the Beeper 200 00:07:39,319 --> 00:07:44,840 in the center as required by the 201 00:07:41,879 --> 00:07:46,680 program uh I'm just going to edit out 202 00:07:44,840 --> 00:07:48,319 the uh a comment out placing the Beeper 203 00:07:46,680 --> 00:07:50,919 and I will mark my 204 00:07:48,319 --> 00:07:53,000 program um so we have an automated 205 00:07:50,919 --> 00:07:55,879 marking setup so that we can compare the 206 00:07:53,000 --> 00:07:58,440 the Carol world at the final State um to 207 00:07:55,879 --> 00:08:00,000 what is expected they get um some 208 00:07:58,440 --> 00:08:02,960 information description 209 00:08:00,000 --> 00:08:04,840 about what actually went wrong uh and 210 00:08:02,960 --> 00:08:06,800 they also get a little asky art diagram 211 00:08:04,840 --> 00:08:09,319 of hey you didn't place a beeper in the 212 00:08:06,800 --> 00:08:10,879 right spot so um the one there indicates 213 00:08:09,319 --> 00:08:12,479 that one beeper should be placed in the 214 00:08:10,879 --> 00:08:15,080 middle and that Carol should be at the 215 00:08:12,479 --> 00:08:16,680 bottom left corner of the screen so 216 00:08:15,080 --> 00:08:19,599 hopefully if I uncomment down my code I 217 00:08:16,680 --> 00:08:22,879 can mark this and I should pass the test 218 00:08:19,599 --> 00:08:25,599 case Okay so that's the the basics of 219 00:08:22,879 --> 00:08:27,599 the Carol environment that we have and 220 00:08:25,599 --> 00:08:31,440 the sorts of things that we can do with 221 00:08:27,599 --> 00:08:31,440 Carol or students can do with Carol 222 00:08:34,760 --> 00:08:38,360 okay so now that we know about the 223 00:08:36,880 --> 00:08:40,320 basics what did we actually do over 224 00:08:38,360 --> 00:08:42,560 these past two semesters so we developed 225 00:08:40,320 --> 00:08:44,240 a playful inquiry and inquiry-driven set 226 00:08:42,560 --> 00:08:47,360 of extracurricular programming 227 00:08:44,240 --> 00:08:48,600 activities uh and challenges using Carol 228 00:08:47,360 --> 00:08:51,040 we did this in two subjects an 229 00:08:48,600 --> 00:08:54,800 undergraduate unit and a postgraduate uh 230 00:08:51,040 --> 00:08:58,839 unit both are uh at the early stages of 231 00:08:54,800 --> 00:09:01,279 the students uh course or course work 232 00:08:58,839 --> 00:09:03,640 and we tried to encourage participation 233 00:09:01,279 --> 00:09:06,600 in these challenges by creating a 234 00:09:03,640 --> 00:09:08,800 competition so we created um a 235 00:09:06,600 --> 00:09:10,880 leaderboard system students were awarded 236 00:09:08,800 --> 00:09:12,200 points for completing uh the challenges 237 00:09:10,880 --> 00:09:15,640 that we set for 238 00:09:12,200 --> 00:09:17,240 them uh and uh at the end we rewarded 239 00:09:15,640 --> 00:09:18,440 those students that did well and 240 00:09:17,240 --> 00:09:20,519 actually rewarded a lot of students who 241 00:09:18,440 --> 00:09:22,279 even for just participating the 242 00:09:20,519 --> 00:09:24,160 competition runs over four or five weeks 243 00:09:22,279 --> 00:09:26,519 depending on the unit uh because of some 244 00:09:24,160 --> 00:09:28,800 scheduling issues and each week we 245 00:09:26,519 --> 00:09:33,079 released a new set of instructional 246 00:09:28,800 --> 00:09:35,560 slides on Ed and uh we release then five 247 00:09:33,079 --> 00:09:38,560 challenges on that week's topic so each 248 00:09:35,560 --> 00:09:39,920 week five new challenges for them to try 249 00:09:38,560 --> 00:09:41,800 questions were worth some maximum number 250 00:09:39,920 --> 00:09:43,720 of points and incorrect submissions were 251 00:09:41,800 --> 00:09:46,000 given a small penalty so what we wanted 252 00:09:43,720 --> 00:09:49,079 to try to encourage by imposing a 253 00:09:46,000 --> 00:09:51,000 penalty uh is getting students to test 254 00:09:49,079 --> 00:09:52,560 and evaluate their own code before they 255 00:09:51,000 --> 00:09:54,800 hit that Mark button so one of the 256 00:09:52,560 --> 00:09:57,519 issues that we find uh with our students 257 00:09:54,800 --> 00:09:59,399 is that on systems like Ed when they're 258 00:09:57,519 --> 00:10:01,440 writing uh code for something that's in 259 00:09:59,399 --> 00:10:03,480 a tutorial or maybe it's their homework 260 00:10:01,440 --> 00:10:05,279 they will just Spam The Mark button 261 00:10:03,480 --> 00:10:06,920 until they get it right rather than 262 00:10:05,279 --> 00:10:09,079 trying to actually test and evaluate 263 00:10:06,920 --> 00:10:10,880 their own code so we wanted to encourage 264 00:10:09,079 --> 00:10:13,399 this behavior of thinking about their 265 00:10:10,880 --> 00:10:15,920 own code uh and being able to test and 266 00:10:13,399 --> 00:10:18,399 evaluate it 267 00:10:15,920 --> 00:10:20,360 themselves okay and in terms of 268 00:10:18,399 --> 00:10:22,839 difficulty I can show you what the range 269 00:10:20,360 --> 00:10:24,959 of difficulty is uh that we we pose to 270 00:10:22,839 --> 00:10:26,279 our students so we saw the first 271 00:10:24,959 --> 00:10:28,120 exercise which was the Beeper drop you 272 00:10:26,279 --> 00:10:30,880 just go into the middle of the room and 273 00:10:28,120 --> 00:10:33,680 place a a beeper and then return to the 274 00:10:30,880 --> 00:10:36,639 corner uh we had some more difficult 275 00:10:33,680 --> 00:10:39,320 ones like filling in the potholes where 276 00:10:36,639 --> 00:10:41,079 students were given this random uh sort 277 00:10:39,320 --> 00:10:43,440 of almost one-dimensional world where 278 00:10:41,079 --> 00:10:45,399 they had to go in and find the locations 279 00:10:43,440 --> 00:10:46,880 where a beeper was present and then 280 00:10:45,399 --> 00:10:48,200 replace that beeper oh sorry where a 281 00:10:46,880 --> 00:10:51,079 beeper wasn't present in these holes and 282 00:10:48,200 --> 00:10:53,000 place a beeper in so that's a relatively 283 00:10:51,079 --> 00:10:55,200 sort of intermediate difficult 284 00:10:53,000 --> 00:10:58,639 difficulty U exercise and then we really 285 00:10:55,200 --> 00:11:01,760 ramped up the difficulty um later on so 286 00:10:58,639 --> 00:11:03,360 we did exercises like uh maze solving 287 00:11:01,760 --> 00:11:06,200 within 288 00:11:03,360 --> 00:11:08,040 Carol um so students implemented sort of 289 00:11:06,200 --> 00:11:11,279 like a a wall holding or or War 290 00:11:08,040 --> 00:11:15,000 following algorithm there um we made 291 00:11:11,279 --> 00:11:17,560 them do uh QR codes which we called KR 292 00:11:15,000 --> 00:11:20,839 codes um and we embedded a nice Rick 293 00:11:17,560 --> 00:11:23,839 Roll in there for the students um and 294 00:11:20,839 --> 00:11:27,120 then other we created some really quite 295 00:11:23,839 --> 00:11:30,240 intricate and very challenging exercises 296 00:11:27,120 --> 00:11:32,760 like coloring in these squares in a very 297 00:11:30,240 --> 00:11:34,360 um you know predetermined pattern or 298 00:11:32,760 --> 00:11:36,279 order which is actually really 299 00:11:34,360 --> 00:11:38,440 challenging because you are unable to in 300 00:11:36,279 --> 00:11:40,519 this carol World Define variables and 301 00:11:38,440 --> 00:11:42,200 keep track of where you are other than 302 00:11:40,519 --> 00:11:44,519 sort of by manually stepping through the 303 00:11:42,200 --> 00:11:46,760 world placing beepers and changing 304 00:11:44,519 --> 00:11:48,240 colors um so the students that were able 305 00:11:46,760 --> 00:11:51,360 to do this we think were really quite 306 00:11:48,240 --> 00:11:54,000 smart um and we tried as well to add 307 00:11:51,360 --> 00:11:55,279 some things which are sort of related to 308 00:11:54,000 --> 00:11:56,880 uh the work that we do in business 309 00:11:55,279 --> 00:11:58,639 analytics like optimization so this was 310 00:11:56,880 --> 00:12:01,480 sort of finding the minimum number of 311 00:11:58,639 --> 00:12:03,920 beepers in in a random World um and this 312 00:12:01,480 --> 00:12:06,200 was also surprisingly deviously 313 00:12:03,920 --> 00:12:08,360 difficult to do given the constraints of 314 00:12:06,200 --> 00:12:08,360 of 315 00:12:12,639 --> 00:12:16,920 Carol 316 00:12:14,880 --> 00:12:18,600 okay so this is an example of the 317 00:12:16,920 --> 00:12:21,120 leaderboard I apologize it's a bit small 318 00:12:18,600 --> 00:12:22,639 but uh we were able to keep track of and 319 00:12:21,120 --> 00:12:25,120 students were able to see in real time 320 00:12:22,639 --> 00:12:26,600 their position on the leaderboard um 321 00:12:25,120 --> 00:12:28,519 they got their name on the leaderboard 322 00:12:26,600 --> 00:12:30,240 the number of points uh and also which 323 00:12:28,519 --> 00:12:33,480 tutorial they belong 324 00:12:30,240 --> 00:12:35,360 to uh this is met with uh some students 325 00:12:33,480 --> 00:12:37,920 like this some students don't like this 326 00:12:35,360 --> 00:12:39,600 we'll get to that later um so we had 327 00:12:37,920 --> 00:12:41,800 some quite good success in our first 328 00:12:39,600 --> 00:12:45,560 semester um semester one these are some 329 00:12:41,800 --> 00:12:46,760 of the the winners of awards um so I 330 00:12:45,560 --> 00:12:49,639 think on the left these are the sort of 331 00:12:46,760 --> 00:12:51,399 the top five or 10% of students in the 332 00:12:49,639 --> 00:12:53,399 unit that I coordinated and then on the 333 00:12:51,399 --> 00:12:57,079 right Allison is presenting the award to 334 00:12:53,399 --> 00:12:58,560 the winner from uh her unit cubus 1040 335 00:12:57,079 --> 00:13:00,360 and then these are the similar uh 336 00:12:58,560 --> 00:13:02,519 cohorts in semester 337 00:13:00,360 --> 00:13:03,880 2 so we had quite a large number of 338 00:13:02,519 --> 00:13:05,720 students participating these are people 339 00:13:03,880 --> 00:13:08,199 who we are sort of at the top of the 340 00:13:05,720 --> 00:13:11,800 cohort and were winning the larger 341 00:13:08,199 --> 00:13:14,839 Awards we also a rewarded participation 342 00:13:11,800 --> 00:13:17,040 at different levels so um I've got some 343 00:13:14,839 --> 00:13:18,720 trophies here as an example and if I can 344 00:13:17,040 --> 00:13:21,079 get Nikki or someone to pass these 345 00:13:18,720 --> 00:13:23,959 around as a little bit of a prop These 346 00:13:21,079 --> 00:13:26,440 are 3D printed trophies um that we put 347 00:13:23,959 --> 00:13:27,680 together um for in recognition of the 348 00:13:26,440 --> 00:13:29,880 students 349 00:13:27,680 --> 00:13:32,760 achievement um so so we also had 350 00:13:29,880 --> 00:13:35,720 personalized certificates uh and also 351 00:13:32,760 --> 00:13:37,000 smaller 3D printed keychains maybe if 352 00:13:35,720 --> 00:13:40,320 you've got a good question we can hand 353 00:13:37,000 --> 00:13:41,680 out some of those um and we did was a 354 00:13:40,320 --> 00:13:44,519 little bit laborious making these but I 355 00:13:41,680 --> 00:13:47,680 think the students appreciated 356 00:13:44,519 --> 00:13:50,680 it uh and we did see quite a bit of 357 00:13:47,680 --> 00:13:52,759 evidence of this uh so students uh 358 00:13:50,680 --> 00:13:54,279 posted on in particular Chinese social 359 00:13:52,759 --> 00:13:56,759 media because a lot of our students are 360 00:13:54,279 --> 00:13:58,680 are Chinese International students U and 361 00:13:56,759 --> 00:13:59,839 in the translations um they said they 362 00:13:58,680 --> 00:14:02,720 were really happy and proud of their 363 00:13:59,839 --> 00:14:03,759 achievement which is really great to see 364 00:14:02,720 --> 00:14:05,079 and one of these students on the right 365 00:14:03,759 --> 00:14:06,279 hand side was quite upset because she 366 00:14:05,079 --> 00:14:07,480 actually missed out on the presentation 367 00:14:06,279 --> 00:14:10,759 ceremony but we were able to get a 368 00:14:07,480 --> 00:14:14,000 reward to a later um so in terms of 369 00:14:10,759 --> 00:14:15,240 participation and impact uh we've had we 370 00:14:14,000 --> 00:14:17,839 had sort of a thousand students in the 371 00:14:15,240 --> 00:14:19,839 first semester and uh about 900 students 372 00:14:17,839 --> 00:14:22,759 in the second semester and 200 200 and 373 00:14:19,839 --> 00:14:25,079 350 students participating so we had a a 374 00:14:22,759 --> 00:14:27,120 k degre of 375 00:14:25,079 --> 00:14:30,440 participation uh generally the feedback 376 00:14:27,120 --> 00:14:34,839 has been positive um so students uh have 377 00:14:30,440 --> 00:14:38,040 reported that they felt that Carol a FL 378 00:14:34,839 --> 00:14:39,880 over me um have helps them understand 379 00:14:38,040 --> 00:14:42,079 things Concepts like if El statements 380 00:14:39,880 --> 00:14:43,199 and Loops in particular those are two 381 00:14:42,079 --> 00:14:46,000 things that our students really have 382 00:14:43,199 --> 00:14:48,399 struggled with in the past and also uh 383 00:14:46,000 --> 00:14:50,040 students have reported that they felt uh 384 00:14:48,399 --> 00:14:52,040 that it made them think that programming 385 00:14:50,040 --> 00:14:53,920 was more fun and developed things like 386 00:14:52,040 --> 00:14:56,320 critical thinking and problem solving 387 00:14:53,920 --> 00:14:57,560 skills and they reported that part of 388 00:14:56,320 --> 00:14:58,880 the reason for that is that they can 389 00:14:57,560 --> 00:15:01,600 visualize exactly what's happening in 390 00:14:58,880 --> 00:15:05,000 their program at any 391 00:15:01,600 --> 00:15:06,839 time um in terms of uh we have done some 392 00:15:05,000 --> 00:15:09,399 informal surveys from semester one and 393 00:15:06,839 --> 00:15:11,199 now we've we've done a a formal survey 394 00:15:09,399 --> 00:15:13,279 in in semester 2 but I can't discuss the 395 00:15:11,199 --> 00:15:15,639 results of those just yet so I'm going 396 00:15:13,279 --> 00:15:18,560 to report the results from semester one 397 00:15:15,639 --> 00:15:20,399 um generally students thought that Carol 398 00:15:18,560 --> 00:15:22,199 uh had a positive impact on the way they 399 00:15:20,399 --> 00:15:23,399 thought about programming and improved 400 00:15:22,199 --> 00:15:26,120 their understanding of programing 401 00:15:23,399 --> 00:15:27,839 Concepts and they also found that it was 402 00:15:26,120 --> 00:15:31,160 fun which is part of our a broader goal 403 00:15:27,839 --> 00:15:31,160 to make programming fun for the 404 00:15:31,199 --> 00:15:35,240 students uh students felt the sort of 405 00:15:33,839 --> 00:15:38,279 comfortable with this process which is 406 00:15:35,240 --> 00:15:40,680 really great to see uh and in general 407 00:15:38,279 --> 00:15:44,240 there was an uplift in responses to 408 00:15:40,680 --> 00:15:48,240 things like being able to uh being able 409 00:15:44,240 --> 00:15:50,399 to use uh python to solve uh problems 410 00:15:48,240 --> 00:15:52,199 and that they they indicated to us that 411 00:15:50,399 --> 00:15:53,759 they would continue to undertake uh 412 00:15:52,199 --> 00:15:56,120 courses which required programming which 413 00:15:53,759 --> 00:15:58,240 is the majority of courses in our 414 00:15:56,120 --> 00:16:00,319 field we also asked them why they didn't 415 00:15:58,240 --> 00:16:02,680 participate most of the time it was 416 00:16:00,319 --> 00:16:04,279 because they didn't have time uh or some 417 00:16:02,680 --> 00:16:05,440 of them reported they didn't have uh 418 00:16:04,279 --> 00:16:07,519 didn't know about it or weren't 419 00:16:05,440 --> 00:16:09,000 interested in it um so that was from 420 00:16:07,519 --> 00:16:11,360 semester one and we had a greater uplift 421 00:16:09,000 --> 00:16:13,399 in semester 2 so hopefully we can reduce 422 00:16:11,360 --> 00:16:14,880 this number or have a nicer looking 423 00:16:13,399 --> 00:16:18,240 graph for semester 424 00:16:14,880 --> 00:16:20,600 2 now um one of the things I wanted to 425 00:16:18,240 --> 00:16:23,920 say is that what we're doing here is not 426 00:16:20,600 --> 00:16:26,759 really about Carol itself or using Ed um 427 00:16:23,920 --> 00:16:28,360 but what we're trying to do is create a 428 00:16:26,759 --> 00:16:29,839 playful learning environment for our 429 00:16:28,360 --> 00:16:32,920 students where they can learn and 430 00:16:29,839 --> 00:16:35,800 experiment and build confidence with 431 00:16:32,920 --> 00:16:38,839 programming um so our checklist for 432 00:16:35,800 --> 00:16:40,800 Success um does not really involve Carol 433 00:16:38,839 --> 00:16:42,600 at all um if you wanted to implement 434 00:16:40,800 --> 00:16:44,800 something like this in your own teaching 435 00:16:42,600 --> 00:16:46,839 context this is what we would recommend 436 00:16:44,800 --> 00:16:48,880 so we would recommend aligning with your 437 00:16:46,839 --> 00:16:50,560 learning outcome so for us we chose 438 00:16:48,880 --> 00:16:52,560 Carol specifically because it's a subset 439 00:16:50,560 --> 00:16:53,920 of python so if we were using a 440 00:16:52,560 --> 00:16:55,639 different programming language we might 441 00:16:53,920 --> 00:16:57,959 have gone with some other 442 00:16:55,639 --> 00:16:59,519 tool um it also specifically addressed 443 00:16:57,959 --> 00:17:01,560 student weaknesses because it 444 00:16:59,519 --> 00:17:03,039 specifically targeted use use of 445 00:17:01,560 --> 00:17:04,799 conditionals and 446 00:17:03,039 --> 00:17:06,079 Loops uh we wanted to keep it 447 00:17:04,799 --> 00:17:07,360 interesting and fun and we recommend 448 00:17:06,079 --> 00:17:09,000 that this is well we think this is 449 00:17:07,360 --> 00:17:11,319 really part of the success of this 450 00:17:09,000 --> 00:17:12,640 program so we design challenges with a 451 00:17:11,319 --> 00:17:14,199 range of difficulty levels and 452 00:17:12,640 --> 00:17:16,919 seriousness so try to put some 453 00:17:14,199 --> 00:17:18,559 playfulness in your exercises um don't 454 00:17:16,919 --> 00:17:20,400 make them sort of super boring move 455 00:17:18,559 --> 00:17:21,839 Carol to this position and return try to 456 00:17:20,400 --> 00:17:23,439 get them to do something fun like make a 457 00:17:21,839 --> 00:17:25,839 QR code or solve a 458 00:17:23,439 --> 00:17:28,120 maze try to create a safe space for 459 00:17:25,839 --> 00:17:29,679 mistakes so this had no impact on the 460 00:17:28,120 --> 00:17:32,240 student marks students were were 461 00:17:29,679 --> 00:17:34,440 participating completely voluntarily so 462 00:17:32,240 --> 00:17:36,880 if students did didn't perform very well 463 00:17:34,440 --> 00:17:38,440 it was there's no harm no fou and 464 00:17:36,880 --> 00:17:39,799 students were sort of free to make as 465 00:17:38,440 --> 00:17:41,799 many mistakes as they 466 00:17:39,799 --> 00:17:46,000 wanted we also think you should give 467 00:17:41,799 --> 00:17:47,600 Fast feedback uh so we gave automatic 468 00:17:46,000 --> 00:17:50,440 feedback and immediate feedback through 469 00:17:47,600 --> 00:17:52,360 an automatic marking process so if 470 00:17:50,440 --> 00:17:54,080 students aren't getting that um fast 471 00:17:52,360 --> 00:17:56,200 feedback from something like this 472 00:17:54,080 --> 00:17:59,120 program I we feel like they're not 473 00:17:56,200 --> 00:18:01,679 likely to continue doing it um and that 474 00:17:59,120 --> 00:18:03,280 feedback could be sort of a yes no youve 475 00:18:01,679 --> 00:18:05,159 passed a test case or not and it can 476 00:18:03,280 --> 00:18:07,320 also be part of the actual Carol 477 00:18:05,159 --> 00:18:09,520 visualization process too so seeing that 478 00:18:07,320 --> 00:18:12,640 2D grid world is also part of the 479 00:18:09,520 --> 00:18:14,720 feedback uh motivate so we we motivated 480 00:18:12,640 --> 00:18:16,280 our students through a live leaderboard 481 00:18:14,720 --> 00:18:18,520 with mixed success some students liked 482 00:18:16,280 --> 00:18:22,039 it some students didn't and we awarded 483 00:18:18,520 --> 00:18:24,520 students prizes and we awarded prizes um 484 00:18:22,039 --> 00:18:26,679 for a medium level of participation all 485 00:18:24,520 --> 00:18:29,600 the way up to doing really well like 486 00:18:26,679 --> 00:18:31,799 scoring the the highest points so we 487 00:18:29,600 --> 00:18:34,360 think you should actually not just award 488 00:18:31,799 --> 00:18:35,960 those um who did really well but go back 489 00:18:34,360 --> 00:18:37,520 down and try to encourage participation 490 00:18:35,960 --> 00:18:39,000 ahead of time and what we did as well 491 00:18:37,520 --> 00:18:40,600 with our students is we said hey if you 492 00:18:39,000 --> 00:18:42,400 meet this minimum Benchmark which was 493 00:18:40,600 --> 00:18:44,760 sort of doing half the challenges you 494 00:18:42,400 --> 00:18:46,520 would still get some sort of um 495 00:18:44,760 --> 00:18:47,480 recognition of your achievement in this 496 00:18:46,520 --> 00:18:49,760 case a 497 00:18:47,480 --> 00:18:50,880 certificate we also encouraged student 498 00:18:49,760 --> 00:18:52,440 communication through use of a 499 00:18:50,880 --> 00:18:54,760 discussion board and we tried to limit 500 00:18:52,440 --> 00:18:56,440 staff intervention on the uh discussion 501 00:18:54,760 --> 00:18:57,640 board for these specific questions 502 00:18:56,440 --> 00:19:00,280 because we wanted students to build a 503 00:18:57,640 --> 00:19:01,320 bit of a community around Carol and 504 00:19:00,280 --> 00:19:02,760 again I think I've already touched on 505 00:19:01,320 --> 00:19:05,320 this point of recognizing achievement at 506 00:19:02,760 --> 00:19:08,000 all levels um don't try to don't just 507 00:19:05,320 --> 00:19:10,400 limit it to the top performing 508 00:19:08,000 --> 00:19:12,520 students so what's going to happen in 509 00:19:10,400 --> 00:19:14,240 the future so I already mentioned this 510 00:19:12,520 --> 00:19:15,960 but in semester 2 we actually launched 511 00:19:14,240 --> 00:19:18,280 this as a formal research project within 512 00:19:15,960 --> 00:19:20,840 the university um this involved larger 513 00:19:18,280 --> 00:19:22,360 scale surveys so we have much uh higher 514 00:19:20,840 --> 00:19:24,400 uh response rates to these surveys and 515 00:19:22,360 --> 00:19:26,360 we've also already run focus groups 516 00:19:24,400 --> 00:19:28,880 we're waiting waiting for final exams to 517 00:19:26,360 --> 00:19:31,960 be completed so that when we can match 518 00:19:28,880 --> 00:19:34,240 Carol activity to assessment scores like 519 00:19:31,960 --> 00:19:35,720 exam questions that Target programming 520 00:19:34,240 --> 00:19:37,400 and we're also going to be able to 521 00:19:35,720 --> 00:19:39,159 control somewhat for predis 522 00:19:37,400 --> 00:19:41,320 predisposition and self- selection 523 00:19:39,159 --> 00:19:43,320 because we have access to past student 524 00:19:41,320 --> 00:19:46,480 performance data through their entry 525 00:19:43,320 --> 00:19:48,880 scores um scores in other units and 526 00:19:46,480 --> 00:19:52,480 things like that so we'll hopefully be 527 00:19:48,880 --> 00:19:55,720 able to um accurately model whether the 528 00:19:52,480 --> 00:19:57,400 effect is due to Carol itself um or just 529 00:19:55,720 --> 00:19:59,840 the students's predisposition to 530 00:19:57,400 --> 00:20:02,919 participation and we expect to publish 531 00:19:59,840 --> 00:20:04,280 these results in 2025 sometime and our 532 00:20:02,919 --> 00:20:07,240 plan is actually to kind of keep 533 00:20:04,280 --> 00:20:10,640 repeating this and improving on uh our 534 00:20:07,240 --> 00:20:12,840 initiative into 2026 uh as a part of a 535 00:20:10,640 --> 00:20:15,520 longer term 536 00:20:12,840 --> 00:20:16,960 study uh and lastly I'd like to wrap up 537 00:20:15,520 --> 00:20:19,400 by saying thank you to the university 538 00:20:16,960 --> 00:20:21,559 for funding us to do this we did this as 539 00:20:19,400 --> 00:20:24,679 part of a playful Innovations Grant 540 00:20:21,559 --> 00:20:26,440 within the uh within the business school 541 00:20:24,679 --> 00:20:30,159 uh and we spent most of our money on a 542 00:20:26,440 --> 00:20:32,039 3D printer which is really nice 543 00:20:30,159 --> 00:20:33,799 um but yeah we'd like to thank the 544 00:20:32,039 --> 00:20:36,360 university for supporting us uh in and 545 00:20:33,799 --> 00:20:37,610 letting us do this um and that's all I 546 00:20:36,360 --> 00:20:44,720 have to 547 00:20:37,610 --> 00:20:44,720 [Applause] 548 00:20:46,559 --> 00:20:53,520 say okay thank you Stephen uh we have 549 00:20:49,720 --> 00:20:56,159 already got hands up all over the 550 00:20:53,520 --> 00:20:58,080 place that was really awesome thanks um 551 00:20:56,159 --> 00:20:59,559 do you have a sense of whether there's a 552 00:20:58,080 --> 00:21:01,080 point at which the Simplicity of the 553 00:20:59,559 --> 00:21:03,480 environment starts limiting what people 554 00:21:01,080 --> 00:21:05,600 can learn or is the Simplicity good all 555 00:21:03,480 --> 00:21:07,799 the way through yeah there that's a good 556 00:21:05,600 --> 00:21:11,159 point there is a limit to it so we 557 00:21:07,799 --> 00:21:12,799 introduced this in the first half of the 558 00:21:11,159 --> 00:21:14,640 semester where students are learning 559 00:21:12,799 --> 00:21:15,840 those fundamental concepts like 560 00:21:14,640 --> 00:21:17,559 condition we tried to match it to the 561 00:21:15,840 --> 00:21:18,760 times when they're Lo uh learning 562 00:21:17,559 --> 00:21:22,919 conditionals and loops and things like 563 00:21:18,760 --> 00:21:24,919 that and yeah I think beyond that um it 564 00:21:22,919 --> 00:21:26,039 has less of an impact on those learning 565 00:21:24,919 --> 00:21:27,919 outcomes that you're probably trying to 566 00:21:26,039 --> 00:21:29,240 Target towards the end of a course where 567 00:21:27,919 --> 00:21:30,520 concepts are much more difficult like 568 00:21:29,240 --> 00:21:32,960 objectoriented programming things like 569 00:21:30,520 --> 00:21:35,400 that there's no real way to sort of uh 570 00:21:32,960 --> 00:21:37,320 mirror that with Carol so yes there is a 571 00:21:35,400 --> 00:21:38,720 limit a point which we probably suggest 572 00:21:37,320 --> 00:21:40,200 that you maybe move on to something a 573 00:21:38,720 --> 00:21:42,960 bit more real or different than than 574 00:21:40,200 --> 00:21:45,400 Carol make people angry or frustrated 575 00:21:42,960 --> 00:21:46,799 just oh the student seem to understand 576 00:21:45,400 --> 00:21:47,960 they and in their focus groups they've 577 00:21:46,799 --> 00:21:50,520 reported that they can see the clear 578 00:21:47,960 --> 00:21:53,840 connections between the the unit content 579 00:21:50,520 --> 00:21:55,600 and what is happening in Carol um but 580 00:21:53,840 --> 00:21:58,840 yeah they they appreciate that also it's 581 00:21:55,600 --> 00:22:02,279 kind of just a bit of fun that has has a 582 00:21:58,840 --> 00:22:02,279 limited lifespan yeah for 583 00:22:03,720 --> 00:22:10,000 them I was wondering do you combine the 584 00:22:07,720 --> 00:22:11,640 like you said give rapid feedback but 585 00:22:10,000 --> 00:22:13,919 you also said you know give prizes 586 00:22:11,640 --> 00:22:16,200 rewards that kind of thing do you 587 00:22:13,919 --> 00:22:18,760 combine those in any way and give prizes 588 00:22:16,200 --> 00:22:20,200 sort of throughout the use of this like 589 00:22:18,760 --> 00:22:21,840 you know each week or you know at the 590 00:22:20,200 --> 00:22:24,520 midpoint or whatever or is it just the 591 00:22:21,840 --> 00:22:26,520 prizes at the end as the like main goal 592 00:22:24,520 --> 00:22:29,400 so we've limited it to prizes at the end 593 00:22:26,520 --> 00:22:31,440 mostly I think for uh simp licity of 594 00:22:29,400 --> 00:22:33,039 administration um we're trying to do 595 00:22:31,440 --> 00:22:35,400 this alongside teaching the normal stuff 596 00:22:33,039 --> 00:22:37,799 so yeah I think if if we could we could 597 00:22:35,400 --> 00:22:39,559 we have well in the past actually the 598 00:22:37,799 --> 00:22:41,520 first semester I ran this I in my 599 00:22:39,559 --> 00:22:44,240 version of the unit I had quite a 600 00:22:41,520 --> 00:22:45,760 complex Awards uh sort of structure much 601 00:22:44,240 --> 00:22:47,919 more complicated than Allison's and I 602 00:22:45,760 --> 00:22:49,559 gave out uh prizes for things like first 603 00:22:47,919 --> 00:22:52,679 to complete all the exercises in that 604 00:22:49,559 --> 00:22:54,559 week and then uh or first or person who 605 00:22:52,679 --> 00:22:55,840 scored the most within a week but it was 606 00:22:54,559 --> 00:22:57,480 still only at the end because I didn't 607 00:22:55,840 --> 00:22:58,919 have time to keep up with it throughout 608 00:22:57,480 --> 00:23:01,840 throughout the semester but yeah I think 609 00:22:58,919 --> 00:23:03,559 it that would definitely help uh improve 610 00:23:01,840 --> 00:23:04,559 participation and cons consistent 611 00:23:03,559 --> 00:23:05,679 participation because one of the things 612 00:23:04,559 --> 00:23:08,279 that students have reported in focus 613 00:23:05,679 --> 00:23:09,679 groups is that they really want like the 614 00:23:08,279 --> 00:23:11,000 the prize and the award at the end but 615 00:23:09,679 --> 00:23:12,520 they will leave things to the last 616 00:23:11,000 --> 00:23:14,320 minute as classic student Behavior 617 00:23:12,520 --> 00:23:16,440 they'll try to do all the challenges a 618 00:23:14,320 --> 00:23:18,400 day or two before the due date closing 619 00:23:16,440 --> 00:23:21,279 date 620 00:23:18,400 --> 00:23:23,400 yeah yeah thank you um is your course 621 00:23:21,279 --> 00:23:25,559 workk open sourced it feels like it 622 00:23:23,400 --> 00:23:28,240 could be useful to schools as well uh 623 00:23:25,559 --> 00:23:29,720 our coursework is not open sourced um 624 00:23:28,240 --> 00:23:31,400 where Alice and I working on a different 625 00:23:29,720 --> 00:23:34,240 project that's not affiliated with the 626 00:23:31,400 --> 00:23:37,919 university to do something like that um 627 00:23:34,240 --> 00:23:40,559 but uh yeah not not this specific unit 628 00:23:37,919 --> 00:23:42,600 course Stanford thing right yeah so the 629 00:23:40,559 --> 00:23:44,880 Carol uh thing came from came from 630 00:23:42,600 --> 00:23:46,559 Stanford um you don't need to use Ed or 631 00:23:44,880 --> 00:23:49,080 anything like that there are open source 632 00:23:46,559 --> 00:23:51,279 Carol packages so that you can um just 633 00:23:49,080 --> 00:23:52,720 pip install a Carol package and run 634 00:23:51,279 --> 00:23:55,080 Carol locally on your machine or some 635 00:23:52,720 --> 00:23:57,480 other environment 636 00:23:55,080 --> 00:23:59,200 yeah hi thank you so much for your 637 00:23:57,480 --> 00:24:02,159 presentation I really enjoy enjoyed that 638 00:23:59,200 --> 00:24:05,360 I was just wondering did you have any 639 00:24:02,159 --> 00:24:07,360 data from prior to the Carol being used 640 00:24:05,360 --> 00:24:10,679 in the courses to Baseline student 641 00:24:07,360 --> 00:24:14,200 engagement and participation uh we do 642 00:24:10,679 --> 00:24:17,240 have data on that um so we have data 643 00:24:14,200 --> 00:24:19,320 through Ed of um all of our tutorial 644 00:24:17,240 --> 00:24:21,320 material is hosted on Ed and has a quite 645 00:24:19,320 --> 00:24:23,840 a com comprehensive analytics system so 646 00:24:21,320 --> 00:24:25,399 we can see which exercise students have 647 00:24:23,840 --> 00:24:27,080 uh participated in and completed and 648 00:24:25,399 --> 00:24:30,919 things like that number of attempts and 649 00:24:27,080 --> 00:24:32,399 so on was there much of a difference we 650 00:24:30,919 --> 00:24:33,760 haven't got to got to that point in the 651 00:24:32,399 --> 00:24:36,000 analysis the other thing is that our 652 00:24:33,760 --> 00:24:38,679 ethics approval for studying student 653 00:24:36,000 --> 00:24:40,480 data is limited to from semester 2 654 00:24:38,679 --> 00:24:42,080 onward so we're not actually allowed 655 00:24:40,480 --> 00:24:45,279 within the university to include student 656 00:24:42,080 --> 00:24:47,760 data from prior to that 657 00:24:45,279 --> 00:24:50,679 unfortunately um from sorry that was 658 00:24:47,760 --> 00:24:52,360 really cool but from like after students 659 00:24:50,679 --> 00:24:53,799 were finished with Carol did you notice 660 00:24:52,360 --> 00:24:57,080 a difference in sort of like onboarding 661 00:24:53,799 --> 00:24:58,799 them into uh more widely used uh 662 00:24:57,080 --> 00:25:01,159 programming languages and that sort of 663 00:24:58,799 --> 00:25:03,320 thing for us we stay with python 664 00:25:01,159 --> 00:25:05,159 throughout the entire course and most of 665 00:25:03,320 --> 00:25:07,480 their degree is done in Python some 666 00:25:05,159 --> 00:25:10,279 units will touch on are so I can't 667 00:25:07,480 --> 00:25:14,080 really comment on that yet uh hopefully 668 00:25:10,279 --> 00:25:16,000 we can get some insight into uh Pro 669 00:25:14,080 --> 00:25:17,679 improvements in programming performance 670 00:25:16,000 --> 00:25:19,480 um once we get to analyzing their sort 671 00:25:17,679 --> 00:25:20,720 of exam scores and things like that at 672 00:25:19,480 --> 00:25:23,880 the end of the 673 00:25:20,720 --> 00:25:25,360 semester um I noticed in the graph of 674 00:25:23,880 --> 00:25:28,000 like people participating or not 675 00:25:25,360 --> 00:25:30,679 participating uh that a lot of them said 676 00:25:28,000 --> 00:25:32,240 that it looked too hard um what exposure 677 00:25:30,679 --> 00:25:34,520 do they get to like what it's going to 678 00:25:32,240 --> 00:25:35,960 be like before they get started on it 679 00:25:34,520 --> 00:25:38,880 and then like what are you doing to help 680 00:25:35,960 --> 00:25:42,200 them like get over that hurdle yeah so I 681 00:25:38,880 --> 00:25:44,480 mean the they get to see exactly uh some 682 00:25:42,200 --> 00:25:47,240 of what I've shown you before um so we 683 00:25:44,480 --> 00:25:49,760 have the set of introductory slides um 684 00:25:47,240 --> 00:25:51,960 and in some of our tutorials as well we 685 00:25:49,760 --> 00:25:55,039 also um demonstrate this and get them to 686 00:25:51,960 --> 00:25:57,360 work through the first exercise um so 687 00:25:55,039 --> 00:25:59,799 that is was imp the in putting it into 688 00:25:57,360 --> 00:26:03,200 tutorials was implement in semester 2 so 689 00:25:59,799 --> 00:26:05,760 those results were from semester one 690 00:26:03,200 --> 00:26:07,440 yeah so so this is a question but I have 691 00:26:05,760 --> 00:26:10,080 to give you a premise I have to give 692 00:26:07,440 --> 00:26:13,080 people a premise I um I used some of 693 00:26:10,080 --> 00:26:16,039 this stuff with my year n students and I 694 00:26:13,080 --> 00:26:18,200 found that I was I was 695 00:26:16,039 --> 00:26:19,919 blindsided about halfway through I went 696 00:26:18,200 --> 00:26:22,960 to check how people were 697 00:26:19,919 --> 00:26:24,600 progressing and a quarter of my class 698 00:26:22,960 --> 00:26:27,880 had not used a single Loop and just 699 00:26:24,600 --> 00:26:30,600 copied and pasted move and turn left a 700 00:26:27,880 --> 00:26:33,159 billion times y 701 00:26:30,600 --> 00:26:36,320 um uh this is not a judgment of me of 702 00:26:33,159 --> 00:26:38,679 course um so I'm wondering if you've 703 00:26:36,320 --> 00:26:40,840 come across that and if you have thought 704 00:26:38,679 --> 00:26:44,080 about how to guide people to like 705 00:26:40,840 --> 00:26:46,720 getting their code a little you know 706 00:26:44,080 --> 00:26:47,960 more yeah we definitely have encountered 707 00:26:46,720 --> 00:26:49,679 that uh one of the problems that 708 00:26:47,960 --> 00:26:51,679 students report in their focus group is 709 00:26:49,679 --> 00:26:53,880 that sometimes their Carol programs took 710 00:26:51,679 --> 00:26:56,480 too long to run so even though you can 711 00:26:53,880 --> 00:26:57,679 run it at 100 times speed some students 712 00:26:56,480 --> 00:26:59,880 report to take some of those programs 713 00:26:57,679 --> 00:27:02,080 running for half an hour with 100 time 714 00:26:59,880 --> 00:27:06,320 speed um so they are definitely not 715 00:27:02,080 --> 00:27:08,679 writing things well a lot of the time 716 00:27:06,320 --> 00:27:09,960 um in terms of actually addressing that 717 00:27:08,679 --> 00:27:11,799 problem we haven't really come up with 718 00:27:09,960 --> 00:27:13,000 any solutions for that the best thing I 719 00:27:11,799 --> 00:27:14,679 can come off the top of my head is 720 00:27:13,000 --> 00:27:16,120 probably having some sort of dedicated 721 00:27:14,679 --> 00:27:18,080 out of class creating sort of like a 722 00:27:16,120 --> 00:27:20,360 peer learning group or sort of like a 723 00:27:18,080 --> 00:27:24,039 study session for Carol to help and 724 00:27:20,360 --> 00:27:24,039 guide those those students 725 00:27:24,840 --> 00:27:31,159 um change a scrip link a 726 00:27:29,200 --> 00:27:32,679 we we could do those sorts of things uh 727 00:27:31,159 --> 00:27:34,399 we we wanted to make it fun and we 728 00:27:32,679 --> 00:27:36,320 didn't really we wanted to kind of 729 00:27:34,399 --> 00:27:37,840 encourage independent problem solving we 730 00:27:36,320 --> 00:27:40,120 kind of didn't really care how they they 731 00:27:37,840 --> 00:27:42,480 did it too much because part of the goal 732 00:27:40,120 --> 00:27:45,320 is encourage participation don't force 733 00:27:42,480 --> 00:27:47,559 them down a narrow path 734 00:27:45,320 --> 00:27:48,559 yeah great thank you very much Stephen 735 00:27:47,559 --> 00:27:51,960 can we give him another round of 736 00:27:48,559 --> 00:27:51,960 applause please