1
00:00:06,320 --> 00:00:11,499
[Music]

2
00:00:16,240 --> 00:00:20,160
hello everyone and welcome to the kiara

3
00:00:18,240 --> 00:00:23,359
uh theater at linux conference australia

4
00:00:20,160 --> 00:00:25,519
2022. uh first up we've got nicola de

5
00:00:23,359 --> 00:00:27,760
fran who is a principal multimedia

6
00:00:25,519 --> 00:00:29,279
engineer at collabora

7
00:00:27,760 --> 00:00:31,119
and is actively involved with both the

8
00:00:29,279 --> 00:00:32,960
g-streamer and linux media communities

9
00:00:31,119 --> 00:00:34,079
to help create a solid support for codex

10
00:00:32,960 --> 00:00:36,160
on linux

11
00:00:34,079 --> 00:00:38,160
and he's here to present bring webm

12
00:00:36,160 --> 00:00:39,840
alpha support to gstreamer introducing

13
00:00:38,160 --> 00:00:41,840
video transition retro alpha plane

14
00:00:39,840 --> 00:00:43,200
support a workaround and we'll dive into

15
00:00:41,840 --> 00:00:45,039
the architecture he's implemented to

16
00:00:43,200 --> 00:00:46,399
support this kind of video now we are

17
00:00:45,039 --> 00:00:47,120
going to have time for questions at the

18
00:00:46,399 --> 00:00:48,960
end

19
00:00:47,120 --> 00:00:52,399
uh so put your questions into the

20
00:00:48,960 --> 00:00:53,680
questions tab and venueless as we go

21
00:00:52,399 --> 00:00:56,640
and try to put them in with plenty of

22
00:00:53,680 --> 00:00:58,800
time because we are operating on a delay

23
00:00:56,640 --> 00:01:01,520
but yeah take it away nicola

24
00:00:58,800 --> 00:01:03,760
thank you caitlyn

25
00:01:01,520 --> 00:01:04,960
so hello thanks uh thank you very much

26
00:01:03,760 --> 00:01:07,680
lca

27
00:01:04,960 --> 00:01:09,680
team for organizing this uh virtual

28
00:01:07,680 --> 00:01:12,479
conference i understand how difficult it

29
00:01:09,680 --> 00:01:16,320
can be it's amazing organization we have

30
00:01:12,479 --> 00:01:19,280
a lot of support so a big thanks

31
00:01:16,320 --> 00:01:22,240
and it's also part of our effort to not

32
00:01:19,280 --> 00:01:23,119
be the problem and solve the problem

33
00:01:22,240 --> 00:01:25,520
so

34
00:01:23,119 --> 00:01:27,600
as i gotta introduce i'm nicolas dufrein

35
00:01:25,520 --> 00:01:29,360
i'm just drummer contributor

36
00:01:27,600 --> 00:01:30,720
actually i've been contributing for over

37
00:01:29,360 --> 00:01:31,600
10 years

38
00:01:30,720 --> 00:01:34,159
um

39
00:01:31,600 --> 00:01:35,840
i'm quite lucky

40
00:01:34,159 --> 00:01:38,079
because i'm allowed to actually

41
00:01:35,840 --> 00:01:41,360
contribute to this project in my

42
00:01:38,079 --> 00:01:43,759
day-to-day job at collabora i've been at

43
00:01:41,360 --> 00:01:45,680
calabra for 11 years this

44
00:01:43,759 --> 00:01:47,920
now and

45
00:01:45,680 --> 00:01:48,799
i've been able to learn and contribute

46
00:01:47,920 --> 00:01:51,119
to

47
00:01:48,799 --> 00:01:53,360
so many things especially nice for

48
00:01:51,119 --> 00:01:55,520
someone like me that likes to learn new

49
00:01:53,360 --> 00:01:57,520
things and being a consultant is just

50
00:01:55,520 --> 00:01:59,280
about that jumping into a new project

51
00:01:57,520 --> 00:02:01,119
solving new problems

52
00:01:59,280 --> 00:02:04,079
and getting the work

53
00:02:01,119 --> 00:02:05,439
the job done and speaking of a job

54
00:02:04,079 --> 00:02:07,520
uh the

55
00:02:05,439 --> 00:02:09,440
implementation of webm alpha inside

56
00:02:07,520 --> 00:02:11,760
gstreamer was actually

57
00:02:09,440 --> 00:02:13,200
mandated by one of our customers in the

58
00:02:11,760 --> 00:02:15,599
streaming industry

59
00:02:13,200 --> 00:02:16,720
so they wanted to use g streamer on some

60
00:02:15,599 --> 00:02:19,599
sort of a

61
00:02:16,720 --> 00:02:20,640
pro streamer handheld that would

62
00:02:19,599 --> 00:02:22,879
basically

63
00:02:20,640 --> 00:02:24,720
allow on the road

64
00:02:22,879 --> 00:02:26,959
let's see more professional streaming

65
00:02:24,720 --> 00:02:28,879
capabilities but one of the big missing

66
00:02:26,959 --> 00:02:31,840
features in this rumor for them was the

67
00:02:28,879 --> 00:02:33,200
ability to play back 10 years

68
00:02:31,840 --> 00:02:34,080
and then

69
00:02:33,200 --> 00:02:38,160
yeah

70
00:02:34,080 --> 00:02:41,840
but to be honest i had just no idea what

71
00:02:38,160 --> 00:02:45,200
his tenure was so first thing i did is

72
00:02:41,840 --> 00:02:47,200
look at wikipedia which says um well a

73
00:02:45,200 --> 00:02:49,280
stinger is an opaque

74
00:02:47,200 --> 00:02:50,959
organ or body part found in various

75
00:02:49,280 --> 00:02:53,599
animals typically

76
00:02:50,959 --> 00:02:57,280
arthropods that usually deliver some

77
00:02:53,599 --> 00:03:00,720
kind of venom it wasn't very useful

78
00:02:57,280 --> 00:03:02,319
but after some more research

79
00:03:00,720 --> 00:03:04,480
because i really wanted to know where

80
00:03:02,319 --> 00:03:06,720
that word came from

81
00:03:04,480 --> 00:03:08,000
i actually found that the words tenures

82
00:03:06,720 --> 00:03:10,400
is used in

83
00:03:08,000 --> 00:03:12,159
american radio station or was used in

84
00:03:10,400 --> 00:03:15,200
american radio station

85
00:03:12,159 --> 00:03:16,879
as a little sound that would be fairly

86
00:03:15,200 --> 00:03:18,879
unique to the station and that would

87
00:03:16,879 --> 00:03:21,120
allow the audience to actually recognize

88
00:03:18,879 --> 00:03:23,440
the the channel that they're that they

89
00:03:21,120 --> 00:03:26,159
are listening to

90
00:03:23,440 --> 00:03:28,080
so and that definition actually well

91
00:03:26,159 --> 00:03:30,560
it's not really a definition but that

92
00:03:28,080 --> 00:03:33,280
definition actually fits very well to

93
00:03:30,560 --> 00:03:35,519
what streamers today actually uses as an

94
00:03:33,280 --> 00:03:37,680
audio videos tenure

95
00:03:35,519 --> 00:03:39,920
which is a way to personalize their

96
00:03:37,680 --> 00:03:41,599
streaming and their streaming transition

97
00:03:39,920 --> 00:03:44,640
or and it could be an entrance a

98
00:03:41,599 --> 00:03:46,959
transition or an ending in order to have

99
00:03:44,640 --> 00:03:48,560
a very specific signature

100
00:03:46,959 --> 00:03:50,879
so what they are they are personal

101
00:03:48,560 --> 00:03:53,760
animation

102
00:03:50,879 --> 00:03:57,439
for transition they usually start with a

103
00:03:53,760 --> 00:03:58,640
transparent video that transit toward an

104
00:03:57,439 --> 00:04:01,040
opaque

105
00:03:58,640 --> 00:04:02,720
middle point where the transition take

106
00:04:01,040 --> 00:04:05,599
place and then it would go back to

107
00:04:02,720 --> 00:04:07,760
transparent and then it would go away in

108
00:04:05,599 --> 00:04:09,040
the montage

109
00:04:07,760 --> 00:04:11,120
and

110
00:04:09,040 --> 00:04:13,599
these are while typically altered with

111
00:04:11,120 --> 00:04:16,400
tools like after effects but it looks

112
00:04:13,599 --> 00:04:18,160
like it's feasible these days to to make

113
00:04:16,400 --> 00:04:20,079
them actually using open source tool

114
00:04:18,160 --> 00:04:21,519
there's a demonstration i've seen

115
00:04:20,079 --> 00:04:24,400
recently about

116
00:04:21,519 --> 00:04:27,440
producing a stinger with [ __ ] enscape

117
00:04:24,400 --> 00:04:28,960
and also with shotgun video editor which

118
00:04:27,440 --> 00:04:32,000
natively support

119
00:04:28,960 --> 00:04:35,360
with with ffmpeg the ability to to

120
00:04:32,000 --> 00:04:37,680
encode and export uh those video

121
00:04:35,360 --> 00:04:40,720
uh test tenures actually

122
00:04:37,680 --> 00:04:44,080
could have been in other format web than

123
00:04:40,720 --> 00:04:48,400
webm you could have actually used uh one

124
00:04:44,080 --> 00:04:50,320
of the pro codec like a pro res 444 that

125
00:04:48,400 --> 00:04:53,840
has native support for

126
00:04:50,320 --> 00:04:56,000
for that or some higher profile in hevc

127
00:04:53,840 --> 00:04:58,479
but the truth is that it's rarely

128
00:04:56,000 --> 00:05:01,440
supported by your hardware it's not

129
00:04:58,479 --> 00:05:04,479
accelerated on everybody's computer

130
00:05:01,440 --> 00:05:05,520
so you need a more expensive setup to

131
00:05:04,479 --> 00:05:07,520
use them

132
00:05:05,520 --> 00:05:09,520
and that's where the wet em the webm

133
00:05:07,520 --> 00:05:11,919
team which is a google team actually

134
00:05:09,520 --> 00:05:13,759
came up with a great idea which is webm

135
00:05:11,919 --> 00:05:15,120
alpha

136
00:05:13,759 --> 00:05:16,960
and they wanted and they actually

137
00:05:15,120 --> 00:05:18,400
managed to do that using

138
00:05:16,960 --> 00:05:22,840
their

139
00:05:18,400 --> 00:05:25,840
royalty free codec vp8 and vp9

140
00:05:22,840 --> 00:05:28,320
but that there's a but there's a

141
00:05:25,840 --> 00:05:31,759
there's a problem because neither vp8 or

142
00:05:28,320 --> 00:05:34,240
vp9 have support for an alpha channel

143
00:05:31,759 --> 00:05:35,440
they only support three channels the

144
00:05:34,240 --> 00:05:37,919
colors

145
00:05:35,440 --> 00:05:40,560
the crew and the two chromas the two

146
00:05:37,919 --> 00:05:40,560
color plane

147
00:05:42,000 --> 00:05:46,639
the idea they had is that what if we use

148
00:05:44,960 --> 00:05:49,440
two streams

149
00:05:46,639 --> 00:05:50,720
and we map the alpha from the second

150
00:05:49,440 --> 00:05:51,520
stream

151
00:05:50,720 --> 00:05:53,919
as

152
00:05:51,520 --> 00:05:56,240
we map the luma as from the second

153
00:05:53,919 --> 00:05:59,520
screen the the brightness

154
00:05:56,240 --> 00:06:01,520
as the alpha and then it just ignore

155
00:05:59,520 --> 00:06:03,520
the uv plane

156
00:06:01,520 --> 00:06:05,600
and that's exactly what they did so

157
00:06:03,520 --> 00:06:08,000
typically that's the simplest form and

158
00:06:05,600 --> 00:06:08,960
that's what the software live vpx would

159
00:06:08,000 --> 00:06:09,919
produce

160
00:06:08,960 --> 00:06:12,960
uh

161
00:06:09,919 --> 00:06:15,520
vp8 or vp9 decode to

162
00:06:12,960 --> 00:06:17,199
three planes the y the u and the v and

163
00:06:15,520 --> 00:06:20,560
then they would decode twice and then

164
00:06:17,199 --> 00:06:23,680
they would map the y as a and the

165
00:06:20,560 --> 00:06:26,800
the x's there are the u and the v and

166
00:06:23,680 --> 00:06:27,600
they are actually set to something

167
00:06:26,800 --> 00:06:29,919
uh

168
00:06:27,600 --> 00:06:32,479
uniform so always the same value with

169
00:06:29,919 --> 00:06:34,160
the idea that uh during compression

170
00:06:32,479 --> 00:06:36,400
because it's always the same value it's

171
00:06:34,160 --> 00:06:39,280
going to compress very well so it's not

172
00:06:36,400 --> 00:06:41,440
going to introduce as much overhead

173
00:06:39,280 --> 00:06:43,840
um this is also a bit unlike if you were

174
00:06:41,440 --> 00:06:44,960
to do that with h.264 where you can

175
00:06:43,840 --> 00:06:46,160
actually

176
00:06:44,960 --> 00:06:48,560
have

177
00:06:46,160 --> 00:06:51,120
monochrome encoding you could have used

178
00:06:48,560 --> 00:06:54,479
monochrome encoding instead of

179
00:06:51,120 --> 00:06:55,759
of this workaround

180
00:06:54,479 --> 00:06:57,840
now that

181
00:06:55,759 --> 00:07:00,400
that by itself

182
00:06:57,840 --> 00:07:03,759
is not sufficient so

183
00:07:00,400 --> 00:07:06,160
uh they actually needed to

184
00:07:03,759 --> 00:07:09,120
store and attach the the two frames

185
00:07:06,160 --> 00:07:11,680
together somewhere and with bit stream

186
00:07:09,120 --> 00:07:13,440
like vp8 and vp9 which are our turn i

187
00:07:11,680 --> 00:07:16,319
mean they're done you can't actually

188
00:07:13,440 --> 00:07:18,319
extend them there's no way to add new

189
00:07:16,319 --> 00:07:22,080
new information to it

190
00:07:18,319 --> 00:07:24,720
so their idea was to use

191
00:07:22,080 --> 00:07:27,280
a container format instead so containers

192
00:07:24,720 --> 00:07:30,080
are the name we give to

193
00:07:27,280 --> 00:07:32,880
things like webm mp4

194
00:07:30,080 --> 00:07:35,680
uh you'll find transport stream in the

195
00:07:32,880 --> 00:07:37,199
stream in the the broadcast industry

196
00:07:35,680 --> 00:07:40,720
and

197
00:07:37,199 --> 00:07:43,199
they would extend webm which is a subset

198
00:07:40,720 --> 00:07:45,440
of matroska which is well known in the

199
00:07:43,199 --> 00:07:47,520
uh in the multimedia community

200
00:07:45,440 --> 00:07:49,360
in order to support this

201
00:07:47,520 --> 00:07:52,080
now what you need to know about webm to

202
00:07:49,360 --> 00:07:53,280
keep things simple it's implemented on

203
00:07:52,080 --> 00:07:54,319
top of

204
00:07:53,280 --> 00:07:57,199
uh

205
00:07:54,319 --> 00:07:58,960
next enable on ebml which is extendable

206
00:07:57,199 --> 00:08:00,080
binary markup language

207
00:07:58,960 --> 00:08:01,520
which

208
00:08:00,080 --> 00:08:04,160
what is nice is that you can actually

209
00:08:01,520 --> 00:08:05,360
express it in xml in order to represent

210
00:08:04,160 --> 00:08:08,160
the content

211
00:08:05,360 --> 00:08:10,240
so to make it short and simplify it a

212
00:08:08,160 --> 00:08:13,280
bit

213
00:08:10,240 --> 00:08:15,120
a webm file is split into

214
00:08:13,280 --> 00:08:18,160
two groups the segment

215
00:08:15,120 --> 00:08:20,160
is what describes the data that is

216
00:08:18,160 --> 00:08:22,560
stored in the container and then you get

217
00:08:20,160 --> 00:08:24,720
a cluster which has block group and

218
00:08:22,560 --> 00:08:27,919
blocks and block additions

219
00:08:24,720 --> 00:08:30,160
to to actually um carry the data that

220
00:08:27,919 --> 00:08:31,599
you're you're streaming

221
00:08:30,160 --> 00:08:32,880
and

222
00:08:31,599 --> 00:08:36,000
the way they

223
00:08:32,880 --> 00:08:38,479
decide they they manage to add

224
00:08:36,000 --> 00:08:40,719
this alpha channel this extra second

225
00:08:38,479 --> 00:08:43,760
string is through what we call the block

226
00:08:40,719 --> 00:08:46,240
addition which is an abstract

227
00:08:43,760 --> 00:08:48,640
piece of binary that you can add

228
00:08:46,240 --> 00:08:50,640
attached to a frame per frame and block

229
00:08:48,640 --> 00:08:53,120
addition have no meaning unless you

230
00:08:50,640 --> 00:08:55,040
actually add a tag in the track entry so

231
00:08:53,120 --> 00:08:56,720
that's what they did so they added a tag

232
00:08:55,040 --> 00:08:59,760
to the video track

233
00:08:56,720 --> 00:09:02,240
which says that we are in alpha mode one

234
00:08:59,760 --> 00:09:04,800
why one why not true or false because

235
00:09:02,240 --> 00:09:06,880
they probably intended to be able to add

236
00:09:04,800 --> 00:09:08,240
other modes in the future

237
00:09:06,880 --> 00:09:09,680
i don't see what it would be but they

238
00:09:08,240 --> 00:09:12,480
want to be safe

239
00:09:09,680 --> 00:09:14,640
so whenever alpha mode is set it means

240
00:09:12,480 --> 00:09:16,800
that there's going to be a block

241
00:09:14,640 --> 00:09:22,040
addition for each frame and that this

242
00:09:16,800 --> 00:09:22,040
block addition is an alpha channel

243
00:09:22,959 --> 00:09:28,320
so enough of the

244
00:09:25,200 --> 00:09:29,200
introduction here

245
00:09:28,320 --> 00:09:31,839
uh

246
00:09:29,200 --> 00:09:34,000
how do we map this into gstreamer and

247
00:09:31,839 --> 00:09:36,880
what i really want to show you is the

248
00:09:34,000 --> 00:09:39,279
journey of a gstreamer developer after

249
00:09:36,880 --> 00:09:40,399
learning a technology and trying to map

250
00:09:39,279 --> 00:09:43,760
it back

251
00:09:40,399 --> 00:09:46,560
into gstreamer technology

252
00:09:43,760 --> 00:09:49,120
just just read me well this is entirely

253
00:09:46,560 --> 00:09:51,360
supported in ffmpeg it has been there

254
00:09:49,120 --> 00:09:53,839
for a while you can both encode and

255
00:09:51,360 --> 00:09:55,920
decode in ffmpeg so we're a bit in the

256
00:09:53,839 --> 00:09:58,080
catch-up mode in the digital world in

257
00:09:55,920 --> 00:10:00,560
this regard because we didn't have any

258
00:09:58,080 --> 00:10:01,680
use case so far

259
00:10:00,560 --> 00:10:03,040
actually

260
00:10:01,680 --> 00:10:05,120
that being said

261
00:10:03,040 --> 00:10:08,399
there was prior to we'll see later that

262
00:10:05,120 --> 00:10:10,959
there was already some some work done

263
00:10:08,399 --> 00:10:12,560
we also had to keep in consideration we

264
00:10:10,959 --> 00:10:15,200
had to know actually what we needed to

265
00:10:12,560 --> 00:10:17,839
consider before we started so

266
00:10:15,200 --> 00:10:20,480
um one of the thing i noticed is that

267
00:10:17,839 --> 00:10:23,120
non-alpha decoder the existing vp8 and

268
00:10:20,480 --> 00:10:25,680
vp9 decoder would work just fine

269
00:10:23,120 --> 00:10:29,360
ignoring alpha with the black background

270
00:10:25,680 --> 00:10:30,720
and that should keep working for sure

271
00:10:29,360 --> 00:10:32,560
another thing we didn't want to

272
00:10:30,720 --> 00:10:34,720
introduce

273
00:10:32,560 --> 00:10:36,560
any wrapper for the case where you

274
00:10:34,720 --> 00:10:39,519
actually decode uh

275
00:10:36,560 --> 00:10:42,800
vp9 data because this can be

276
00:10:39,519 --> 00:10:44,399
pretty big 4k 60fps with hdr and

277
00:10:42,800 --> 00:10:47,120
everything these days

278
00:10:44,399 --> 00:10:48,079
with netflix and and not netflix but

279
00:10:47,120 --> 00:10:50,000
with uh

280
00:10:48,079 --> 00:10:52,240
with with google youtube actually using

281
00:10:50,000 --> 00:10:54,959
that a lot

282
00:10:52,240 --> 00:10:56,720
uh it should also work with auto plugger

283
00:10:54,959 --> 00:10:58,880
that what it means basically it should

284
00:10:56,720 --> 00:11:01,920
work in your favorite media player

285
00:10:58,880 --> 00:11:04,959
without having to code something

286
00:11:01,920 --> 00:11:07,519
and of course it would it should not

287
00:11:04,959 --> 00:11:09,760
break the ability to to do to actually

288
00:11:07,519 --> 00:11:12,000
disable stuff to disable hardware

289
00:11:09,760 --> 00:11:13,200
decoding which is a relatively new

290
00:11:12,000 --> 00:11:15,440
feature

291
00:11:13,200 --> 00:11:16,800
and that would have been very easy to to

292
00:11:15,440 --> 00:11:18,800
ignore

293
00:11:16,800 --> 00:11:20,800
now about the prior arts so before i

294
00:11:18,800 --> 00:11:23,279
started um

295
00:11:20,800 --> 00:11:24,880
just like what i i do usually and i i

296
00:11:23,279 --> 00:11:27,040
would strongly recommend if you're part

297
00:11:24,880 --> 00:11:28,800
of a community

298
00:11:27,040 --> 00:11:29,680
i've talked about the project i wanted

299
00:11:28,800 --> 00:11:32,959
to do

300
00:11:29,680 --> 00:11:34,480
uh over the irc channel of g streamer

301
00:11:32,959 --> 00:11:35,279
and

302
00:11:34,480 --> 00:11:38,480
and

303
00:11:35,279 --> 00:11:40,720
and basically give some layout of the

304
00:11:38,480 --> 00:11:43,279
the design i would like to to to use to

305
00:11:40,720 --> 00:11:45,040
make sure that i'm not totally off track

306
00:11:43,279 --> 00:11:46,320
that i'm not doing something that would

307
00:11:45,040 --> 00:11:48,800
never work

308
00:11:46,320 --> 00:11:51,440
and uh that was really good because uh

309
00:11:48,800 --> 00:11:53,360
tim philip mirrar and matthias poncher

310
00:11:51,440 --> 00:11:56,959
actually had an old branch which was

311
00:11:53,360 --> 00:11:59,839
implementing that for software decoding

312
00:11:56,959 --> 00:12:02,320
there was a bunch of little issues that

313
00:11:59,839 --> 00:12:04,560
actually led to not them making it

314
00:12:02,320 --> 00:12:08,399
upstream

315
00:12:04,560 --> 00:12:08,399
basically it was only software decoder

316
00:12:08,639 --> 00:12:12,959
and anytime you would have actually a

317
00:12:10,800 --> 00:12:14,480
hardware decoder in your system

318
00:12:12,959 --> 00:12:16,639
it would actually prefer the hardware

319
00:12:14,480 --> 00:12:18,560
decoder over the the software decoder

320
00:12:16,639 --> 00:12:21,279
but the other decoder might not have

321
00:12:18,560 --> 00:12:23,440
alpha support so that the auto plugging

322
00:12:21,279 --> 00:12:24,800
decision was not correct and was quite

323
00:12:23,440 --> 00:12:27,200
surprising

324
00:12:24,800 --> 00:12:29,200
they were also using a feature called q

325
00:12:27,200 --> 00:12:31,600
data which is a

326
00:12:29,200 --> 00:12:33,200
basically string identifier for extra

327
00:12:31,600 --> 00:12:35,120
data on on

328
00:12:33,200 --> 00:12:36,800
on buffers in gcstreamer

329
00:12:35,120 --> 00:12:38,800
to store the block audition i thought it

330
00:12:36,800 --> 00:12:41,040
was not very elegant but it was not a

331
00:12:38,800 --> 00:12:43,120
bug it was it was fine but it to me that

332
00:12:41,040 --> 00:12:44,880
was not the most elegant way so i've

333
00:12:43,120 --> 00:12:45,839
decided to use a different approach

334
00:12:44,880 --> 00:12:46,800
there

335
00:12:45,839 --> 00:12:49,519
um

336
00:12:46,800 --> 00:12:51,600
but in the end it was a big time saver

337
00:12:49,519 --> 00:12:53,519
because they had solved all the the

338
00:12:51,600 --> 00:12:56,560
things that i knew the last which was

339
00:12:53,519 --> 00:12:58,800
ebml uh parsing and

340
00:12:56,560 --> 00:13:01,760
and and handling so they had the metro

341
00:12:58,800 --> 00:13:04,000
rosca de max part in place all i had to

342
00:13:01,760 --> 00:13:06,880
do was to map

343
00:13:04,000 --> 00:13:09,279
so speaking of mapping so

344
00:13:06,880 --> 00:13:12,000
um in order to make the

345
00:13:09,279 --> 00:13:15,040
codec alpha stream identifiable i

346
00:13:12,000 --> 00:13:16,639
decided to add a new field to the to the

347
00:13:15,040 --> 00:13:19,680
caps that would be exposed by the

348
00:13:16,639 --> 00:13:22,880
demuxers so i would map the alpha mode

349
00:13:19,680 --> 00:13:24,959
from the ebml to a gstreamer caps codec

350
00:13:22,880 --> 00:13:26,399
alpha it's a boolean because i don't

351
00:13:24,959 --> 00:13:28,320
believe we'll have a new mode in the

352
00:13:26,399 --> 00:13:29,519
future

353
00:13:28,320 --> 00:13:32,000
and

354
00:13:29,519 --> 00:13:34,959
for the older codecs they would remain

355
00:13:32,000 --> 00:13:36,720
with no fields this is something very

356
00:13:34,959 --> 00:13:39,199
that you you would have to know if you

357
00:13:36,720 --> 00:13:41,440
were to do development in gstreamer uh

358
00:13:39,199 --> 00:13:45,279
especially regarding the caps

359
00:13:41,440 --> 00:13:46,480
that if a field is not in the caps it

360
00:13:45,279 --> 00:13:49,360
basically

361
00:13:46,480 --> 00:13:51,920
mean that it supports any value for

362
00:13:49,360 --> 00:13:54,160
other field

363
00:13:51,920 --> 00:13:54,160
and

364
00:13:55,760 --> 00:14:00,079
so and it it's it's nice because in our

365
00:13:58,720 --> 00:14:03,680
case it's a way to not break

366
00:14:00,079 --> 00:14:05,920
compatibility it means that the old uh

367
00:14:03,680 --> 00:14:08,639
the oldy color will be compatible with

368
00:14:05,920 --> 00:14:11,040
the streams that have codec equal alpha

369
00:14:08,639 --> 00:14:13,360
set into as a field

370
00:14:11,040 --> 00:14:16,800
if i had created a new format a new

371
00:14:13,360 --> 00:14:19,040
media type actually like video x vp9

372
00:14:16,800 --> 00:14:21,760
dash alpha i would have broken this

373
00:14:19,040 --> 00:14:21,760
implementation

374
00:14:23,120 --> 00:14:27,440
the second part of the mapping was where

375
00:14:25,120 --> 00:14:29,199
do we store the block addition as i said

376
00:14:27,440 --> 00:14:31,600
it was attached to the gst buffer

377
00:14:29,199 --> 00:14:33,839
already so i just keep that but the way

378
00:14:31,600 --> 00:14:36,320
i decided to attach it was to

379
00:14:33,839 --> 00:14:39,279
provide in the video library

380
00:14:36,320 --> 00:14:41,279
what we call gst meta which is a little

381
00:14:39,279 --> 00:14:44,000
object that we can attach to every

382
00:14:41,279 --> 00:14:46,079
buffer in order to carry some extra data

383
00:14:44,000 --> 00:14:48,880
and that meta which is called gsd video

384
00:14:46,079 --> 00:14:50,880
codec alpha meta actually will holds a

385
00:14:48,880 --> 00:14:55,160
reference to the gst buffer that

386
00:14:50,880 --> 00:14:55,160
represents the alpha channel

387
00:14:59,360 --> 00:15:02,959
now

388
00:15:00,240 --> 00:15:03,920
we got the mapping and how do we process

389
00:15:02,959 --> 00:15:06,160
this

390
00:15:03,920 --> 00:15:07,279
uh in order to decode

391
00:15:06,160 --> 00:15:10,000
so

392
00:15:07,279 --> 00:15:13,440
my main idea was to avoid actually

393
00:15:10,000 --> 00:15:15,040
modifying the vp9 decoders to support

394
00:15:13,440 --> 00:15:17,519
alpha

395
00:15:15,040 --> 00:15:20,160
the reason is that well the prior art

396
00:15:17,519 --> 00:15:22,160
from tim and matthew was was done this

397
00:15:20,160 --> 00:15:25,519
way and it would

398
00:15:22,160 --> 00:15:28,000
highly increase the complexity of the

399
00:15:25,519 --> 00:15:28,800
of the decoders and i really didn't want

400
00:15:28,000 --> 00:15:31,040
that

401
00:15:28,800 --> 00:15:33,600
and just remember is a very powerful

402
00:15:31,040 --> 00:15:36,160
graph processing tool and i i thought

403
00:15:33,600 --> 00:15:39,120
that hey we should create a graph for it

404
00:15:36,160 --> 00:15:42,000
so that's what i end up that was my

405
00:15:39,120 --> 00:15:44,160
draft actually which i kept till the end

406
00:15:42,000 --> 00:15:46,320
except i renamed few things

407
00:15:44,160 --> 00:15:47,440
well that's the final name sorry

408
00:15:46,320 --> 00:15:50,240
so i

409
00:15:47,440 --> 00:15:53,519
basically to to support this new graph i

410
00:15:50,240 --> 00:15:56,720
basically need two new tools that would

411
00:15:53,519 --> 00:15:58,480
allow me to basically split the streams

412
00:15:56,720 --> 00:16:01,759
then i can decode and then i need to

413
00:15:58,480 --> 00:16:03,199
combine those frames to form

414
00:16:01,759 --> 00:16:06,079
the alpha

415
00:16:03,199 --> 00:16:08,480
so the first element i had to write

416
00:16:06,079 --> 00:16:09,519
is codec alpha demax

417
00:16:08,480 --> 00:16:13,680
it's

418
00:16:09,519 --> 00:16:16,480
it was a simple element it would have

419
00:16:13,680 --> 00:16:17,680
one input stream it would read for from

420
00:16:16,480 --> 00:16:20,399
the input

421
00:16:17,680 --> 00:16:22,480
the gst video codec alpha meta it would

422
00:16:20,399 --> 00:16:25,120
remove the codec alpha field from the

423
00:16:22,480 --> 00:16:28,399
caps and then it would push everything

424
00:16:25,120 --> 00:16:31,519
toward two pads so there's the color the

425
00:16:28,399 --> 00:16:34,320
the the base pad which is the colors

426
00:16:31,519 --> 00:16:37,279
and then it would push the the gc buffer

427
00:16:34,320 --> 00:16:39,360
from the codec alpha meta on the alpha

428
00:16:37,279 --> 00:16:42,079
pad and that's it you now you have two

429
00:16:39,360 --> 00:16:43,920
streams that you can put into cues run

430
00:16:42,079 --> 00:16:46,800
into decoders and get two parallel

431
00:16:43,920 --> 00:16:46,800
decoder running

432
00:16:47,440 --> 00:16:51,360
now out of the decoders

433
00:16:49,519 --> 00:16:53,199
and now we have 2d colors running on

434
00:16:51,360 --> 00:16:54,880
their own threads so there's a bit more

435
00:16:53,199 --> 00:16:57,199
of a challenge when it comes times to

436
00:16:54,880 --> 00:16:58,720
combine the alpha

437
00:16:57,199 --> 00:17:01,360
so what

438
00:16:58,720 --> 00:17:04,480
so i ended up implementing alpha combine

439
00:17:01,360 --> 00:17:06,559
as a fairly simple element in the sense

440
00:17:04,480 --> 00:17:07,760
that it doesn't do

441
00:17:06,559 --> 00:17:09,839
time zone

442
00:17:07,760 --> 00:17:12,079
synchronization or timeout like an

443
00:17:09,839 --> 00:17:13,760
aggregator would do because i thought it

444
00:17:12,079 --> 00:17:15,679
was a bit of an overhead to introduce

445
00:17:13,760 --> 00:17:17,679
yet another thread

446
00:17:15,679 --> 00:17:19,600
in this case

447
00:17:17,679 --> 00:17:22,720
and because we know that there's always

448
00:17:19,600 --> 00:17:25,439
going to be one frame uh

449
00:17:22,720 --> 00:17:28,319
one frame for for one alpha frame for

450
00:17:25,439 --> 00:17:30,080
all color frames and in fact if there's

451
00:17:28,319 --> 00:17:31,360
one missing we can actually signal it

452
00:17:30,080 --> 00:17:33,360
with the gap

453
00:17:31,360 --> 00:17:34,320
we have gap mechanism and gstreamer for

454
00:17:33,360 --> 00:17:36,080
that

455
00:17:34,320 --> 00:17:38,000
so we'll wait until the color and the

456
00:17:36,080 --> 00:17:40,480
alpha buffer are ready then we would

457
00:17:38,000 --> 00:17:42,080
append the memory

458
00:17:40,480 --> 00:17:44,080
together you have to know that gst

459
00:17:42,080 --> 00:17:46,960
buffers are our containers for gst

460
00:17:44,080 --> 00:17:50,480
murmur json memory are blocks of data

461
00:17:46,960 --> 00:17:52,400
and we basically represent a virtual

462
00:17:50,480 --> 00:17:54,400
single memory

463
00:17:52,400 --> 00:17:57,919
group

464
00:17:54,400 --> 00:18:00,480
so it doesn't matter if your data is one

465
00:17:57,919 --> 00:18:02,240
gst buffer or multiple gst buffer

466
00:18:00,480 --> 00:18:05,120
except there's some restriction you

467
00:18:02,240 --> 00:18:07,039
cannot have multiple gst memory for

468
00:18:05,120 --> 00:18:08,640
for the same plane in the streamer

469
00:18:07,039 --> 00:18:10,960
should be the same

470
00:18:08,640 --> 00:18:13,679
and then i can just happen them together

471
00:18:10,960 --> 00:18:15,840
and ignore the uv and then i get

472
00:18:13,679 --> 00:18:18,160
a new format uh fortunately for the

473
00:18:15,840 --> 00:18:21,039
software decoder uh the decoder produce

474
00:18:18,160 --> 00:18:23,520
i420 which is exactly which i described

475
00:18:21,039 --> 00:18:25,760
at the beginning three separate planes

476
00:18:23,520 --> 00:18:28,320
and then i would transform this in an

477
00:18:25,760 --> 00:18:30,880
already supported format called a420

478
00:18:28,320 --> 00:18:32,720
which is the same format but at the end

479
00:18:30,880 --> 00:18:35,600
there's an alpha plane that is added

480
00:18:32,720 --> 00:18:37,760
exactly what we needed

481
00:18:35,600 --> 00:18:40,559
the beauty of alpha combine

482
00:18:37,760 --> 00:18:42,480
is that despite its name it's actually

483
00:18:40,559 --> 00:18:45,280
not doing any

484
00:18:42,480 --> 00:18:47,360
binary operations so it's totally zero

485
00:18:45,280 --> 00:18:49,840
copy it's just a per symbol like

486
00:18:47,360 --> 00:18:53,120
manipulation of the the memory

487
00:18:49,840 --> 00:18:53,120
inside the gstreamer system

488
00:18:53,200 --> 00:18:58,480
now that's great but part of my work was

489
00:18:55,760 --> 00:19:01,360
to support hardware decoders and to be

490
00:18:58,480 --> 00:19:04,640
honest at today's hardware decoder they

491
00:19:01,360 --> 00:19:05,919
will all support nv12 or some variation

492
00:19:04,640 --> 00:19:06,960
of it

493
00:19:05,919 --> 00:19:08,840
so

494
00:19:06,960 --> 00:19:12,000
in nv12

495
00:19:08,840 --> 00:19:14,960
the the format is basically

496
00:19:12,000 --> 00:19:17,520
okay but the format of nv12 is not that

497
00:19:14,960 --> 00:19:18,880
different it keeps the luma the the

498
00:19:17,520 --> 00:19:22,160
brightness actually

499
00:19:18,880 --> 00:19:25,840
uh separate and it will interleave the u

500
00:19:22,160 --> 00:19:25,840
and the v in the same buffer

501
00:19:26,240 --> 00:19:32,240
so that basically means we could do

502
00:19:29,520 --> 00:19:34,080
exactly the same thing if such a format

503
00:19:32,240 --> 00:19:35,919
existed

504
00:19:34,080 --> 00:19:37,520
but the truth is that i didn't find any

505
00:19:35,919 --> 00:19:40,000
such a format

506
00:19:37,520 --> 00:19:41,840
but that didn't really block me so

507
00:19:40,000 --> 00:19:44,880
i mean first time in 10 years i decided

508
00:19:41,840 --> 00:19:46,640
to create my own format so i created a

509
00:19:44,880 --> 00:19:48,880
v12

510
00:19:46,640 --> 00:19:50,559
arguably i think i found some people

511
00:19:48,880 --> 00:19:52,960
that might be using that on some

512
00:19:50,559 --> 00:19:55,039
proprietary tools so i might not be the

513
00:19:52,960 --> 00:19:57,679
creator but anyway it's the first made

514
00:19:55,039 --> 00:19:58,480
up format i added to gstreamer

515
00:19:57,679 --> 00:20:00,320
and

516
00:19:58,480 --> 00:20:02,720
that serves very well the purpose so

517
00:20:00,320 --> 00:20:04,720
it's exactly that it's nv12 and then you

518
00:20:02,720 --> 00:20:07,679
can add an extra

519
00:20:04,720 --> 00:20:11,280
alpha format at the end in order to

520
00:20:07,679 --> 00:20:12,880
carry the alpha and you ignore the uv uh

521
00:20:11,280 --> 00:20:15,520
the uv plane and they have the same

522
00:20:12,880 --> 00:20:17,280
features that when you set the uv you

523
00:20:15,520 --> 00:20:19,679
want to set them to

524
00:20:17,280 --> 00:20:22,240
a uniform value in order to compress

525
00:20:19,679 --> 00:20:22,240
very well

526
00:20:26,640 --> 00:20:30,960
so in the end

527
00:20:28,320 --> 00:20:34,080
we added bunch of elements into

528
00:20:30,960 --> 00:20:36,559
gstreamer so the there's a new plugin

529
00:20:34,080 --> 00:20:38,559
called codec alpha which have the

530
00:20:36,559 --> 00:20:41,039
helpers

531
00:20:38,559 --> 00:20:43,280
um they're the helpers i mentioned and

532
00:20:41,039 --> 00:20:46,080
then to make all this works we actually

533
00:20:43,280 --> 00:20:48,640
need decoders so actually we need bins

534
00:20:46,080 --> 00:20:50,400
so we need a container that will

535
00:20:48,640 --> 00:20:51,520
actually abstract the graph i've showed

536
00:20:50,400 --> 00:20:54,400
earlier

537
00:20:51,520 --> 00:20:56,720
and i ended up exposing some decoders so

538
00:20:54,400 --> 00:20:59,360
there's vp8 alpha decode bin and vp9

539
00:20:56,720 --> 00:21:02,400
alpha decode bin and these are strictly

540
00:20:59,360 --> 00:21:06,720
used to decode webm alpha with software

541
00:21:02,400 --> 00:21:09,280
decoders and then for the v4 l2 sl vpl

542
00:21:06,720 --> 00:21:12,799
alpha in the code bin it's very long

543
00:21:09,280 --> 00:21:16,400
before l2 sl just just a side note sl is

544
00:21:12,799 --> 00:21:18,240
for stateless it's a new

545
00:21:16,400 --> 00:21:21,120
kernel api

546
00:21:18,240 --> 00:21:22,960
to support decoding with

547
00:21:21,120 --> 00:21:25,280
hardware that is called to be stateless

548
00:21:22,960 --> 00:21:27,440
which doesn't actually do the parsing

549
00:21:25,280 --> 00:21:29,120
and the state management for your codex

550
00:21:27,440 --> 00:21:32,080
this is something i've been working on

551
00:21:29,120 --> 00:21:34,640
outside of gstreamer a lot on these apis

552
00:21:32,080 --> 00:21:37,440
and reviewing this api upstream

553
00:21:34,640 --> 00:21:39,760
so this is also another subject i work

554
00:21:37,440 --> 00:21:43,120
on and we wanted to support these of

555
00:21:39,760 --> 00:21:45,840
course we can add as many wrappers as we

556
00:21:43,120 --> 00:21:48,880
need as long as we have the ability to

557
00:21:45,840 --> 00:21:50,000
do the zero copy alpha combine

558
00:21:48,880 --> 00:21:53,360
or

559
00:21:50,000 --> 00:21:55,120
another mean so some some

560
00:21:53,360 --> 00:21:58,120
some tool that would allow this in the

561
00:21:55,120 --> 00:21:58,120
context

562
00:21:58,320 --> 00:22:03,679
um i prepared couple of demos actually

563
00:22:00,799 --> 00:22:05,760
of this uh so this is basically i

564
00:22:03,679 --> 00:22:08,559
thought it was much more interesting to

565
00:22:05,760 --> 00:22:11,120
to see what these things these i just

566
00:22:08,559 --> 00:22:13,520
want to mention before

567
00:22:11,120 --> 00:22:15,440
we actually started demo that i'm

568
00:22:13,520 --> 00:22:18,480
streaming i'm doing

569
00:22:15,440 --> 00:22:20,640
some encoding over the internet and my

570
00:22:18,480 --> 00:22:21,919
little laptop may suffer a little bit

571
00:22:20,640 --> 00:22:24,480
it's very warm

572
00:22:21,919 --> 00:22:26,400
so some of the demo that worked great

573
00:22:24,480 --> 00:22:28,880
before the presentation actually can

574
00:22:26,400 --> 00:22:30,000
stutter a little bit i noticed this

575
00:22:28,880 --> 00:22:32,159
during my

576
00:22:30,000 --> 00:22:34,320
my tech check

577
00:22:32,159 --> 00:22:34,320
so

578
00:22:34,880 --> 00:22:38,240
let's start with the first evo

579
00:22:36,880 --> 00:22:41,360
how that works

580
00:22:38,240 --> 00:22:43,760
first thing let's check what we have new

581
00:22:41,360 --> 00:22:46,640
so i said we have a new

582
00:22:43,760 --> 00:22:46,640
codec alpha

583
00:22:46,799 --> 00:22:51,039
plugin

584
00:22:47,840 --> 00:22:53,520
so you can see that's gc inspect that

585
00:22:51,039 --> 00:22:55,840
we have the new alpha combine element

586
00:22:53,520 --> 00:22:58,080
the new codec alpha demux element and

587
00:22:55,840 --> 00:23:00,159
then we have the 2d code bin

588
00:22:58,080 --> 00:23:02,799
and then if

589
00:23:00,159 --> 00:23:05,039
yeah i won't see the v4l1 on my laptop

590
00:23:02,799 --> 00:23:08,400
because i don't have such hardware

591
00:23:05,039 --> 00:23:09,840
in there and if i look into let's say

592
00:23:08,400 --> 00:23:12,320
vp9

593
00:23:09,840 --> 00:23:12,320
of

594
00:23:12,640 --> 00:23:17,360
the code bin

595
00:23:15,200 --> 00:23:20,480
you can see that

596
00:23:17,360 --> 00:23:20,480
let's start from here

597
00:23:20,640 --> 00:23:24,799
you can see that we have a codec decoder

598
00:23:23,200 --> 00:23:27,679
that is for video

599
00:23:24,799 --> 00:23:30,320
it's a wrapper bin for vp9

600
00:23:27,679 --> 00:23:31,919
you can see is rank so

601
00:23:30,320 --> 00:23:35,200
one of the

602
00:23:31,919 --> 00:23:38,559
one of the issue we had with was that

603
00:23:35,200 --> 00:23:41,039
due to how the auto plugger compared the

604
00:23:38,559 --> 00:23:43,840
the capabilities in order to determine

605
00:23:41,039 --> 00:23:46,000
what is a fit and what is not a fit and

606
00:23:43,840 --> 00:23:48,480
the fact that we use the same media type

607
00:23:46,000 --> 00:23:50,720
the easiest way to make the alpha

608
00:23:48,480 --> 00:23:52,400
decoder work was basically to limit

609
00:23:50,720 --> 00:23:55,919
their use to alpha decoding so they

610
00:23:52,400 --> 00:23:58,799
don't support a standard vp9 and make

611
00:23:55,919 --> 00:24:00,880
their rank so rank is just to order

612
00:23:58,799 --> 00:24:02,960
basically the list of elements when it

613
00:24:00,880 --> 00:24:05,200
comes time still to plug to something

614
00:24:02,960 --> 00:24:06,000
higher so it's primary plus 10 in this

615
00:24:05,200 --> 00:24:08,480
case

616
00:24:06,000 --> 00:24:10,640
and then uh if it was a hardware wrapper

617
00:24:08,480 --> 00:24:13,520
bin it would be a little higher again so

618
00:24:10,640 --> 00:24:15,200
that's just made so it's ir any vp9

619
00:24:13,520 --> 00:24:16,000
implementation

620
00:24:15,200 --> 00:24:18,320
and it

621
00:24:16,000 --> 00:24:21,360
it would be looked at first or discarded

622
00:24:18,320 --> 00:24:23,919
by the uh the auto plugger

623
00:24:21,360 --> 00:24:25,600
and then there's not much um property

624
00:24:23,919 --> 00:24:28,320
it's all the default stuff but you can

625
00:24:25,600 --> 00:24:31,520
see all the child elements the the

626
00:24:28,320 --> 00:24:32,559
combiner the decoders the queues and

627
00:24:31,520 --> 00:24:35,760
the uh

628
00:24:32,559 --> 00:24:36,960
the codic alpha demox in there

629
00:24:35,760 --> 00:24:39,120
now

630
00:24:36,960 --> 00:24:42,400
how do you play this let's take a

631
00:24:39,120 --> 00:24:45,120
stinger because we we talk about stainer

632
00:24:42,400 --> 00:24:47,600
so here we're going to use a little

633
00:24:45,120 --> 00:24:49,200
terminal player command line player

634
00:24:47,600 --> 00:24:51,440
i'm going to use spleeventry because

635
00:24:49,200 --> 00:24:52,960
it's generally work better for this type

636
00:24:51,440 --> 00:24:54,320
of application

637
00:24:52,960 --> 00:24:56,880
and

638
00:24:54,320 --> 00:25:00,240
this is a stainer i've downloaded on the

639
00:24:56,880 --> 00:25:03,039
internet that was available for free i

640
00:25:00,240 --> 00:25:06,159
just forgot where it is from

641
00:25:03,039 --> 00:25:09,039
but thanks for this so let's

642
00:25:06,159 --> 00:25:09,039
play this demo

643
00:25:09,520 --> 00:25:13,200
let's play but it did play in the

644
00:25:12,000 --> 00:25:15,840
background

645
00:25:13,200 --> 00:25:15,840
there we go

646
00:25:16,480 --> 00:25:19,760
so when i'm screen sharing i discovered

647
00:25:18,960 --> 00:25:21,760
that

648
00:25:19,760 --> 00:25:24,000
uh window don't pops

649
00:25:21,760 --> 00:25:25,919
over they actually stay behind so i need

650
00:25:24,000 --> 00:25:29,919
to remember so that that's what it

651
00:25:25,919 --> 00:25:32,000
looked like you can you can see that

652
00:25:29,919 --> 00:25:35,279
oh i'm missing a bit of it you can see

653
00:25:32,000 --> 00:25:37,039
that initially it's uh it's transparent

654
00:25:35,279 --> 00:25:38,720
oops and then it goes away pretty hard

655
00:25:37,039 --> 00:25:41,279
to

656
00:25:38,720 --> 00:25:43,919
catch right

657
00:25:41,279 --> 00:25:46,240
let's seek to zero put the window in the

658
00:25:43,919 --> 00:25:46,240
front

659
00:25:47,600 --> 00:25:51,120
there we go

660
00:25:48,640 --> 00:25:52,240
hope you saw it so that's using

661
00:25:51,120 --> 00:25:54,159
basically

662
00:25:52,240 --> 00:25:55,919
um

663
00:25:54,159 --> 00:25:59,600
i'm basically using the player and i'm

664
00:25:55,919 --> 00:26:00,640
using a gl image sync and i'm telling

665
00:25:59,600 --> 00:26:03,600
the

666
00:26:00,640 --> 00:26:05,840
the display sync to avoid ignoring the

667
00:26:03,600 --> 00:26:07,120
alpha channel so you can see through the

668
00:26:05,840 --> 00:26:08,799
window

669
00:26:07,120 --> 00:26:10,400
now stinger is not the only thing we

670
00:26:08,799 --> 00:26:13,120
could play there's also

671
00:26:10,400 --> 00:26:15,360
this this uh this is about transparent

672
00:26:13,120 --> 00:26:17,440
videos in general so you can you can

673
00:26:15,360 --> 00:26:18,799
play all kind of content so i created

674
00:26:17,440 --> 00:26:20,799
another late

675
00:26:18,799 --> 00:26:23,279
demo which i found which i'll show you

676
00:26:20,799 --> 00:26:25,520
on the web also later

677
00:26:23,279 --> 00:26:28,799
and in this case we're just playing a

678
00:26:25,520 --> 00:26:30,400
different movie so let's run demo

679
00:26:28,799 --> 00:26:31,919
overlay

680
00:26:30,400 --> 00:26:34,960
there we go

681
00:26:31,919 --> 00:26:38,400
this one stutter a little bit

682
00:26:34,960 --> 00:26:40,799
but you got a phone actually that is uh

683
00:26:38,400 --> 00:26:44,480
going around on top of my desktop

684
00:26:40,799 --> 00:26:44,480
and it's all semi-transparent

685
00:26:45,120 --> 00:26:48,159
and

686
00:26:46,000 --> 00:26:50,400
one of the surprises i

687
00:26:48,159 --> 00:26:52,240
i kind of thought it would help but i

688
00:26:50,400 --> 00:26:55,279
didn't i didn't know it was going to

689
00:26:52,240 --> 00:26:57,600
just work so this effort also helped the

690
00:26:55,279 --> 00:27:00,240
webkit community in order to

691
00:26:57,600 --> 00:27:02,320
fill in some some of the gap and some of

692
00:27:00,240 --> 00:27:05,600
the missing feature because browser are

693
00:27:02,320 --> 00:27:07,600
mandated to support these streams but uh

694
00:27:05,600 --> 00:27:09,760
grimmer bay's webkit implementation

695
00:27:07,600 --> 00:27:13,039
didn't support that yet

696
00:27:09,760 --> 00:27:15,840
uh so that's helping uh webkit gtk and

697
00:27:13,039 --> 00:27:18,480
web webkit wp

698
00:27:15,840 --> 00:27:19,600
and i'll show you what it looked like

699
00:27:18,480 --> 00:27:20,880
before

700
00:27:19,600 --> 00:27:24,000
actually

701
00:27:20,880 --> 00:27:24,000
before my changes

702
00:27:27,600 --> 00:27:30,640
here we go

703
00:27:28,720 --> 00:27:32,720
so it's going to be a bit stuttery but

704
00:27:30,640 --> 00:27:34,720
basically you had the phone

705
00:27:32,720 --> 00:27:36,799
going around it's a website

706
00:27:34,720 --> 00:27:38,880
and you have the phone going around and

707
00:27:36,799 --> 00:27:41,600
a black background so it's clearly not

708
00:27:38,880 --> 00:27:43,679
the appropriate render for this web page

709
00:27:41,600 --> 00:27:45,840
so for them it was actually

710
00:27:43,679 --> 00:27:49,840
some test failure

711
00:27:45,840 --> 00:27:50,720
let me show you how i was running this

712
00:27:49,840 --> 00:27:53,679
so

713
00:27:50,720 --> 00:27:56,559
demo epiphany before

714
00:27:53,679 --> 00:27:58,399
so basically i've used one of the new

715
00:27:56,559 --> 00:28:00,960
features which allow you to adjust the

716
00:27:58,399 --> 00:28:03,520
rank of element using an environment and

717
00:28:00,960 --> 00:28:06,080
i've disabled vp9 alpha decode bin so it

718
00:28:03,520 --> 00:28:08,240
was no longer visible to epiphany which

719
00:28:06,080 --> 00:28:10,480
is another well to webkit which is using

720
00:28:08,240 --> 00:28:12,399
anatom plugger to choose the element and

721
00:28:10,480 --> 00:28:13,919
then i'm visiting this web page over

722
00:28:12,399 --> 00:28:17,039
here

723
00:28:13,919 --> 00:28:19,760
now let's do the same

724
00:28:17,039 --> 00:28:21,600
but we're not going to disable the alpha

725
00:28:19,760 --> 00:28:24,919
decoder

726
00:28:21,600 --> 00:28:24,919
the other one

727
00:28:37,120 --> 00:28:39,679
there we go

728
00:28:39,919 --> 00:28:42,480
so

729
00:28:40,640 --> 00:28:44,320
i was really surprised i was really

730
00:28:42,480 --> 00:28:46,000
impressed

731
00:28:44,320 --> 00:28:47,760
when i was done when i it was all

732
00:28:46,000 --> 00:28:50,799
debugging everything i just launched

733
00:28:47,760 --> 00:28:53,200
epiphany the chrome web browser and it

734
00:28:50,799 --> 00:28:56,240
literally just worked there was a little

735
00:28:53,200 --> 00:28:57,520
uh problem around the edge of that phone

736
00:28:56,240 --> 00:28:59,760
because there's a gradient of

737
00:28:57,520 --> 00:29:01,360
semi-transparency and they had a

738
00:28:59,760 --> 00:29:03,200
mishandling of the

739
00:29:01,360 --> 00:29:05,440
pre-multiplied alpha

740
00:29:03,200 --> 00:29:08,559
in this case because videos are straight

741
00:29:05,440 --> 00:29:11,279
we say straight so they they have a

742
00:29:08,559 --> 00:29:12,399
full color information and the alpha

743
00:29:11,279 --> 00:29:14,159
while in

744
00:29:12,399 --> 00:29:16,080
in rendering usually you want to

745
00:29:14,159 --> 00:29:17,760
pre-multiply the color as soon as

746
00:29:16,080 --> 00:29:19,760
possible because if you're going to

747
00:29:17,760 --> 00:29:20,799
render the same object it's it's much

748
00:29:19,760 --> 00:29:22,720
faster

749
00:29:20,799 --> 00:29:24,799
to do the blending it optimize the

750
00:29:22,720 --> 00:29:27,840
blending you also want to do this for

751
00:29:24,799 --> 00:29:29,840
scaling purpose uh but as they never

752
00:29:27,840 --> 00:29:31,919
tested it it was quite uh expected to

753
00:29:29,840 --> 00:29:34,159
have small small little bugs

754
00:29:31,919 --> 00:29:35,679
but they they they fixed it the same

755
00:29:34,159 --> 00:29:38,960
basically the same day i've published

756
00:29:35,679 --> 00:29:41,840
the code so thanks to egalia crew

757
00:29:38,960 --> 00:29:45,520
who did all this nice work

758
00:29:41,840 --> 00:29:47,440
and now as i said i needed to support a

759
00:29:45,520 --> 00:29:49,600
hardware acceleration

760
00:29:47,440 --> 00:29:51,200
and the hardware i'm going to show you

761
00:29:49,600 --> 00:29:52,880
today i'm not going to be able to show

762
00:29:51,200 --> 00:29:54,720
it to the camera there's too many wires

763
00:29:52,880 --> 00:29:55,600
there so i'm going to unplug it if i do

764
00:29:54,720 --> 00:29:58,480
that

765
00:29:55,600 --> 00:30:00,559
it's actually a

766
00:29:58,480 --> 00:30:02,679
renegade inlet from liber computer it

767
00:30:00,559 --> 00:30:06,000
actually hosts a rockchip

768
00:30:02,679 --> 00:30:10,159
rk3399 it's a chip that has been

769
00:30:06,000 --> 00:30:12,240
very popular from i would say 2015 2019

770
00:30:10,159 --> 00:30:14,320
on chromebooks arm chromebooks

771
00:30:12,240 --> 00:30:16,600
uh it's pretty powerful

772
00:30:14,320 --> 00:30:19,840
uh our target was rk

773
00:30:16,600 --> 00:30:22,399
3566 but that's two early samples and

774
00:30:19,840 --> 00:30:24,080
two unstable for me to show you so

775
00:30:22,399 --> 00:30:26,240
what i'm going to do to be able to show

776
00:30:24,080 --> 00:30:27,840
you that i'm going to use another

777
00:30:26,240 --> 00:30:31,440
distribu

778
00:30:27,840 --> 00:30:32,240
pipeline so it's the rk33 viewer

779
00:30:31,440 --> 00:30:34,960
here

780
00:30:32,240 --> 00:30:37,200
which actually runs a v4l2 source

781
00:30:34,960 --> 00:30:38,640
because it's a uvc camera

782
00:30:37,200 --> 00:30:40,960
and

783
00:30:38,640 --> 00:30:41,760
i just select the format because my

784
00:30:40,960 --> 00:30:44,480
little

785
00:30:41,760 --> 00:30:47,520
capture box likes to convert and then

786
00:30:44,480 --> 00:30:49,600
i'll send it to gle matching so this

787
00:30:47,520 --> 00:30:52,720
should if it works

788
00:30:49,600 --> 00:30:54,559
now we're on target display western oh

789
00:30:52,720 --> 00:30:57,840
it's in the background

790
00:30:54,559 --> 00:31:00,640
okay there we go so now we have weston

791
00:30:57,840 --> 00:31:01,840
i need a terminal to actually launch a

792
00:31:00,640 --> 00:31:04,640
demo there

793
00:31:01,840 --> 00:31:07,679
so i'm actually on the device i've

794
00:31:04,640 --> 00:31:11,039
cross-built g-streamer on my machine

795
00:31:07,679 --> 00:31:13,519
and i'm actually i can show you how but

796
00:31:11,039 --> 00:31:16,240
in my g-streamer 3 i'm running a script

797
00:31:13,519 --> 00:31:18,960
called gstm i just asked to strip off

798
00:31:16,240 --> 00:31:20,880
the path from my nfs export

799
00:31:18,960 --> 00:31:21,919
and this way i don't need to install g

800
00:31:20,880 --> 00:31:23,760
streamer every time i make a

801
00:31:21,919 --> 00:31:25,519
modification so it's my development

802
00:31:23,760 --> 00:31:27,840
environment on target

803
00:31:25,519 --> 00:31:30,159
it's a bit slow to load over nfs there's

804
00:31:27,840 --> 00:31:31,679
a lot of files to look at

805
00:31:30,159 --> 00:31:34,399
but it works

806
00:31:31,679 --> 00:31:36,720
then i'll go to the demo folder

807
00:31:34,399 --> 00:31:38,960
uh i wanted to show you the same demos

808
00:31:36,720 --> 00:31:41,440
but turns out my os on this target is

809
00:31:38,960 --> 00:31:44,399
too old so i got a two old mezza with

810
00:31:41,440 --> 00:31:47,600
two old pan frost uh to hold everything

811
00:31:44,399 --> 00:31:49,440
basically to show you the browser

812
00:31:47,600 --> 00:31:51,279
running but be aware this is all

813
00:31:49,440 --> 00:31:54,080
mainline linux

814
00:31:51,279 --> 00:31:55,840
it's a 516 rc something

815
00:31:54,080 --> 00:31:58,159
with

816
00:31:55,840 --> 00:32:00,720
stock mezza running

817
00:31:58,159 --> 00:32:02,399
from fedora and

818
00:32:00,720 --> 00:32:03,840
it's quite good it's quite good i'm

819
00:32:02,399 --> 00:32:04,799
quite impressed these days of what you

820
00:32:03,840 --> 00:32:07,440
can do

821
00:32:04,799 --> 00:32:08,480
with just just open source just mainline

822
00:32:07,440 --> 00:32:11,200
uh

823
00:32:08,480 --> 00:32:13,120
tool so the demo that we're history and

824
00:32:11,200 --> 00:32:16,720
that i'm going to show you is the

825
00:32:13,120 --> 00:32:18,320
overlay the little overlay phone

826
00:32:16,720 --> 00:32:21,919
that's been

827
00:32:18,320 --> 00:32:25,919
there we go so it's starting

828
00:32:21,919 --> 00:32:28,799
there we go so we're on rk3399 we got a

829
00:32:25,919 --> 00:32:30,480
hardware accelerated vp9

830
00:32:28,799 --> 00:32:32,240
alpha transform

831
00:32:30,480 --> 00:32:34,480
one of the one of the thing that you

832
00:32:32,240 --> 00:32:36,720
have to consider when you want to use

833
00:32:34,480 --> 00:32:38,720
hardware acceleration

834
00:32:36,720 --> 00:32:41,200
on these

835
00:32:38,720 --> 00:32:43,360
is that

836
00:32:41,200 --> 00:32:46,559
in order to decode this stream which is

837
00:32:43,360 --> 00:32:47,760
just a simple 30 fps vp9 video it's

838
00:32:46,559 --> 00:32:50,799
1080p

839
00:32:47,760 --> 00:32:52,080
i need 60 fps power in my hardware in

840
00:32:50,799 --> 00:32:54,880
order to do that because i need to

841
00:32:52,080 --> 00:32:57,039
decode two concurrent stream and my my

842
00:32:54,880 --> 00:32:59,919
system need to be well optimized if you

843
00:32:57,039 --> 00:33:04,880
want to go in um streaming standards and

844
00:32:59,919 --> 00:33:07,519
go to 60 fps you need a 120 fps

845
00:33:04,880 --> 00:33:09,760
decoder in alert to do that and if

846
00:33:07,519 --> 00:33:12,000
you're streaming in 4k that could

847
00:33:09,760 --> 00:33:14,240
effectively be a challenge to find

848
00:33:12,000 --> 00:33:17,200
hardware even laptops that supports that

849
00:33:14,240 --> 00:33:19,039
you probably need a big a big gpu to

850
00:33:17,200 --> 00:33:21,519
actually handle it

851
00:33:19,039 --> 00:33:22,799
but keeping that in mind there's always

852
00:33:21,519 --> 00:33:25,760
a way

853
00:33:22,799 --> 00:33:27,519
to do that

854
00:33:25,760 --> 00:33:29,360
there we go

855
00:33:27,519 --> 00:33:32,080
i'm quite impressed but the be aware

856
00:33:29,360 --> 00:33:36,720
that the vp9 decoder on that platform is

857
00:33:32,080 --> 00:33:39,840
a 4k 60 capable so having 60 fps 1080p

858
00:33:36,720 --> 00:33:39,840
1080p should be fine

859
00:33:40,399 --> 00:33:46,399
and voila that's it for my demos

860
00:33:44,880 --> 00:33:50,080
now uh

861
00:33:46,399 --> 00:33:52,799
okay let me keep my there we go so i

862
00:33:50,080 --> 00:33:54,559
like everything else um

863
00:33:52,799 --> 00:33:56,480
it's quite rare that we managed to do

864
00:33:54,559 --> 00:33:58,240
all the features we would like in a go

865
00:33:56,480 --> 00:34:00,000
especially when they're not sponsored

866
00:33:58,240 --> 00:34:02,960
but there's many addition that could be

867
00:34:00,000 --> 00:34:05,200
done so if you want to contribute or you

868
00:34:02,960 --> 00:34:06,720
want to encourage some contributions

869
00:34:05,200 --> 00:34:09,839
feel free to

870
00:34:06,720 --> 00:34:12,720
to to to express your your needs uh one

871
00:34:09,839 --> 00:34:14,879
of the thing i looked at is uh

872
00:34:12,720 --> 00:34:17,919
what other hardware accelerated

873
00:34:14,879 --> 00:34:19,119
accelerator could be used uh to do that

874
00:34:17,919 --> 00:34:21,679
um

875
00:34:19,119 --> 00:34:24,079
i got from uh

876
00:34:21,679 --> 00:34:27,839
okay so i got that it should be fairly

877
00:34:24,079 --> 00:34:29,599
easy with uh direct3d 11 uh decoders to

878
00:34:27,839 --> 00:34:31,599
do that and dark

879
00:34:29,599 --> 00:34:34,560
and version 12 also

880
00:34:31,599 --> 00:34:38,399
uh it's just about someone to do it um

881
00:34:34,560 --> 00:34:41,679
about va api it looks like we need some

882
00:34:38,399 --> 00:34:43,839
more infrastructure around drm pixel

883
00:34:41,679 --> 00:34:46,159
format with the modifiers and all these

884
00:34:43,839 --> 00:34:47,679
things in place before we can actually

885
00:34:46,159 --> 00:34:49,760
do something that is

886
00:34:47,679 --> 00:34:53,280
truly efficient because

887
00:34:49,760 --> 00:34:55,440
um even though they do nv12 they do a

888
00:34:53,280 --> 00:34:58,240
bit of a variation of of nv12 so the

889
00:34:55,440 --> 00:35:01,040
decoder would produce nv12 but the data

890
00:34:58,240 --> 00:35:03,119
is actually whether compressed tiled or

891
00:35:01,040 --> 00:35:04,560
something which is specific and

892
00:35:03,119 --> 00:35:06,240
undocumented well not really

893
00:35:04,560 --> 00:35:10,000
undocumented but it's specific let's say

894
00:35:06,240 --> 00:35:13,520
to intel or md and we need to carry that

895
00:35:10,000 --> 00:35:16,160
modifier that that extra representation

896
00:35:13,520 --> 00:35:19,200
until we get to the gpu in order to

897
00:35:16,160 --> 00:35:22,000
process and color convert the yu yuv to

898
00:35:19,200 --> 00:35:23,760
rgb the the data so until we have this

899
00:35:22,000 --> 00:35:25,680
infrastructure in place it's going to be

900
00:35:23,760 --> 00:35:27,839
hard but i think as soon as this is in

901
00:35:25,680 --> 00:35:29,520
place the wrapper shouldn't be much more

902
00:35:27,839 --> 00:35:32,240
complex than what we have for software

903
00:35:29,520 --> 00:35:34,800
decoder and the v4l decoder i forgot to

904
00:35:32,240 --> 00:35:36,320
list it but i also

905
00:35:34,800 --> 00:35:37,680
i basically forgot to do that i will

906
00:35:36,320 --> 00:35:39,520
probably do that in the next weeks

907
00:35:37,680 --> 00:35:42,640
before release

908
00:35:39,520 --> 00:35:45,440
but i forgot the v4l stateful decoders

909
00:35:42,640 --> 00:35:47,359
which also have vp8 and vp9 support

910
00:35:45,440 --> 00:35:51,280
we're also missing the apple codec

911
00:35:47,359 --> 00:35:53,440
support that would be used on ios and

912
00:35:51,280 --> 00:35:55,200
os x

913
00:35:53,440 --> 00:35:58,000
and we're entirely

914
00:35:55,200 --> 00:35:59,520
missing the encoding and maxing so in

915
00:35:58,000 --> 00:36:02,079
order to encode though there was a

916
00:35:59,520 --> 00:36:04,079
discussion ongoing on the issue tracker

917
00:36:02,079 --> 00:36:05,119
which has been very useful and we

918
00:36:04,079 --> 00:36:07,599
believe that

919
00:36:05,119 --> 00:36:09,839
we need basically the reverse

920
00:36:07,599 --> 00:36:11,040
tool so we need an alpha split that

921
00:36:09,839 --> 00:36:15,839
would actually

922
00:36:11,040 --> 00:36:19,839
take a an a yuv or an av12 buffer and

923
00:36:15,839 --> 00:36:21,920
make two nv12 or two i420 buffers

924
00:36:19,839 --> 00:36:25,839
and we would need the kodak alpha max

925
00:36:21,920 --> 00:36:28,160
which would basically attach the um

926
00:36:25,839 --> 00:36:30,400
the the alpha gsd buffer on the gst

927
00:36:28,160 --> 00:36:32,480
buffer of the colors and finally we

928
00:36:30,400 --> 00:36:34,560
would need to mux it i believe that

929
00:36:32,480 --> 00:36:36,960
maxing will be uh

930
00:36:34,560 --> 00:36:38,880
much easier basically uh we know from

931
00:36:36,960 --> 00:36:42,560
the we will know ahead of time from the

932
00:36:38,880 --> 00:36:45,200
caps that we need to add the alpha

933
00:36:42,560 --> 00:36:48,160
the the alpha tag in the segment

934
00:36:45,200 --> 00:36:50,079
so that's a very good fit and then we

935
00:36:48,160 --> 00:36:54,079
just need to pick the

936
00:36:50,079 --> 00:36:56,400
the the gst meta and wrap it as a block

937
00:36:54,079 --> 00:37:00,320
addition in the stream so that should be

938
00:36:56,400 --> 00:37:00,320
very straightforward i believe

939
00:37:00,880 --> 00:37:04,400
oops

940
00:37:02,000 --> 00:37:06,320
no focus

941
00:37:04,400 --> 00:37:09,280
voila um

942
00:37:06,320 --> 00:37:12,640
oh that's pretty good so thank you ah

943
00:37:09,280 --> 00:37:14,000
that was it and uh we're ready for the

944
00:37:12,640 --> 00:37:16,079
questions

945
00:37:14,000 --> 00:37:20,920
excellent hello i'm back oh i need to

946
00:37:16,079 --> 00:37:20,920
build my headset sorry oh yeah you don't

947
00:37:24,000 --> 00:37:27,040
um we've got some good questions that

948
00:37:25,839 --> 00:37:29,440
was a really interesting talk thank you

949
00:37:27,040 --> 00:37:30,800
so much for that nichola thanks

950
00:37:29,440 --> 00:37:33,520
um the first question that we're gonna

951
00:37:30,800 --> 00:37:35,520
go for is do the resolutions have to

952
00:37:33,520 --> 00:37:37,520
match on the main stream and the alpha

953
00:37:35,520 --> 00:37:40,480
stream could he use a lower resolution

954
00:37:37,520 --> 00:37:43,200
on the alpha stream and upscale

955
00:37:40,480 --> 00:37:45,119
right now they must match

956
00:37:43,200 --> 00:37:47,839
in this it's actually defined by the

957
00:37:45,119 --> 00:37:49,680
webml specification

958
00:37:47,839 --> 00:37:51,920
and it would also be a limitation

959
00:37:49,680 --> 00:37:54,240
because alpha combine

960
00:37:51,920 --> 00:37:56,560
the element that actually takes the two

961
00:37:54,240 --> 00:37:58,960
uh the two streams and combine them

962
00:37:56,560 --> 00:38:01,119
assume that they actually match

963
00:37:58,960 --> 00:38:03,920
so we don't have metadata to describe an

964
00:38:01,119 --> 00:38:06,160
alpha plane which would be um would be a

965
00:38:03,920 --> 00:38:09,520
bit some sample i would say if it was

966
00:38:06,160 --> 00:38:10,400
like a half of a fourth of the plane so

967
00:38:09,520 --> 00:38:12,240
yes

968
00:38:10,400 --> 00:38:13,680
they need to match

969
00:38:12,240 --> 00:38:15,920
okay and we've got a very similar

970
00:38:13,680 --> 00:38:17,280
question as well likewise do the frame

971
00:38:15,920 --> 00:38:18,720
rates have to match on the main and

972
00:38:17,280 --> 00:38:20,000
alpha streams could you use a lower

973
00:38:18,720 --> 00:38:22,320
frame rate for the alpha stream and

974
00:38:20,000 --> 00:38:24,640
interpolate at playback

975
00:38:22,320 --> 00:38:26,400
so the spec is incomplete it doesn't

976
00:38:24,640 --> 00:38:28,480
define everything

977
00:38:26,400 --> 00:38:29,599
but when you look at the streams being

978
00:38:28,480 --> 00:38:32,320
produced

979
00:38:29,599 --> 00:38:35,200
the rules are the following you need

980
00:38:32,320 --> 00:38:37,119
your first frame to have an alpha frame

981
00:38:35,200 --> 00:38:39,119
that's mandatory so when you start you

982
00:38:37,119 --> 00:38:40,640
need to provide the initial data for

983
00:38:39,119 --> 00:38:41,920
your alpha even if it's fully

984
00:38:40,640 --> 00:38:44,320
transparent

985
00:38:41,920 --> 00:38:45,920
and this is to satisfy hardware decoders

986
00:38:44,320 --> 00:38:48,240
basically because they need special

987
00:38:45,920 --> 00:38:50,800
buffers and crafting those buffer with a

988
00:38:48,240 --> 00:38:52,160
cpu is uh it's slow so it would just

989
00:38:50,800 --> 00:38:54,720
delay the start

990
00:38:52,160 --> 00:38:58,240
and increase your latency

991
00:38:54,720 --> 00:39:01,200
and then you're allowed to skip

992
00:38:58,240 --> 00:39:03,599
the alpha channel uh anytime actually

993
00:39:01,200 --> 00:39:06,960
the data does not change

994
00:39:03,599 --> 00:39:09,520
that being said it's a lot of effort uh

995
00:39:06,960 --> 00:39:10,560
for something for for very low gain

996
00:39:09,520 --> 00:39:11,440
because

997
00:39:10,560 --> 00:39:12,240
uh

998
00:39:11,440 --> 00:39:13,839
the

999
00:39:12,240 --> 00:39:16,640
the vp it's something you need to know

1000
00:39:13,839 --> 00:39:18,240
about the vpx bitstream which is from

1001
00:39:16,640 --> 00:39:20,079
on2

1002
00:39:18,240 --> 00:39:23,040
it was highly patented before it was

1003
00:39:20,079 --> 00:39:25,040
released by google so thanks for that

1004
00:39:23,040 --> 00:39:27,760
it's probably the best

1005
00:39:25,040 --> 00:39:31,200
the best compressed bitstream out there

1006
00:39:27,760 --> 00:39:33,359
it actually do some statistics uh on the

1007
00:39:31,200 --> 00:39:36,480
previous decoded frame in order to

1008
00:39:33,359 --> 00:39:38,400
better compress your your parameters so

1009
00:39:36,480 --> 00:39:40,560
if if a parameter is always the same

1010
00:39:38,400 --> 00:39:42,960
value it's going to use less bits

1011
00:39:40,560 --> 00:39:45,119
and if you have

1012
00:39:42,960 --> 00:39:47,839
like the same frame it actually can go

1013
00:39:45,119 --> 00:39:50,240
up to just couple of bytes to say please

1014
00:39:47,839 --> 00:39:53,119
just render a previous frame that's part

1015
00:39:50,240 --> 00:39:55,760
of the codex in vp8 and vp9 this is a

1016
00:39:53,119 --> 00:39:57,119
very powerful feature so usually it gets

1017
00:39:55,760 --> 00:39:58,720
so small

1018
00:39:57,119 --> 00:40:01,839
just a couple of bites that it's

1019
00:39:58,720 --> 00:40:04,000
probably too much effort to to do that

1020
00:40:01,839 --> 00:40:04,880
but you can yes there's no problem with

1021
00:40:04,000 --> 00:40:06,640
that

1022
00:40:04,880 --> 00:40:07,440
that's real cool

1023
00:40:06,640 --> 00:40:09,040
um

1024
00:40:07,440 --> 00:40:11,280
i think that is all the questions that

1025
00:40:09,040 --> 00:40:13,200
we've got in chat for now yep i must

1026
00:40:11,280 --> 00:40:14,160
admit this was an excellent question

1027
00:40:13,200 --> 00:40:15,040
thank you

1028
00:40:14,160 --> 00:40:16,800
yeah

1029
00:40:15,040 --> 00:40:18,960
that's really cool to know though the

1030
00:40:16,800 --> 00:40:20,079
encoding like that's really interesting

1031
00:40:18,960 --> 00:40:21,359
um

1032
00:40:20,079 --> 00:40:23,280
thank you so much for coming and talking

1033
00:40:21,359 --> 00:40:25,280
to us this was a really interesting talk

1034
00:40:23,280 --> 00:40:27,119
um about some stuff that i don't often

1035
00:40:25,280 --> 00:40:28,079
hear about which is really cool

1036
00:40:27,119 --> 00:40:30,079
um

1037
00:40:28,079 --> 00:40:31,040
i'll just mention one thing uh i had to

1038
00:40:30,079 --> 00:40:33,520
do

1039
00:40:31,040 --> 00:40:34,319
um there's job opening at collabora so

1040
00:40:33,520 --> 00:40:36,240
if

1041
00:40:34,319 --> 00:40:39,200
you're looking for a new opportunity

1042
00:40:36,240 --> 00:40:41,119
please visit call call.la slash carriers

1043
00:40:39,200 --> 00:40:42,720
if you find something that suits you

1044
00:40:41,119 --> 00:40:45,359
just you can contact

1045
00:40:42,720 --> 00:40:49,040
directly or one of us if you know one of

1046
00:40:45,359 --> 00:40:51,520
us we're everywhere on irc on

1047
00:40:49,040 --> 00:40:53,119
slack on whatever wherever we're pretty

1048
00:40:51,520 --> 00:40:54,720
much all over the place

1049
00:40:53,119 --> 00:40:56,480
thank you

1050
00:40:54,720 --> 00:40:57,440
cool thank you so much

1051
00:40:56,480 --> 00:40:59,920
um

1052
00:40:57,440 --> 00:41:01,760
our next talk in this room is the let's

1053
00:40:59,920 --> 00:41:05,359
make a game with paris spotfield addison

1054
00:41:01,760 --> 00:41:07,520
and that's 11 40 am adt so that's in

1055
00:41:05,359 --> 00:41:09,040
like 15 minutes if i can do maths

1056
00:41:07,520 --> 00:41:11,200
correctly

1057
00:41:09,040 --> 00:41:11,200
um

1058
00:41:11,760 --> 00:41:14,720
great

1059
00:41:13,040 --> 00:41:17,720
i'll see you later thank you very much

1060
00:41:14,720 --> 00:41:17,720
bye