welcome back everyone we hope you had a
great lunch and
got a nap in for some of you for joining
from the more fun time zone crossovers
uh we are joined now by lily uh lily is
a penetration tester digital security
consultant and public speaker who serves
on the board of digital rights watch
lilly specializes in web application
security privacy education and the
history of technology related issues
bringing these topics to an
international audience
she believes in the power of consumer
and tech worker action to help the
technology industry better serve the
people it affects
take it away lily
thank you
and hey everyone i hope you are all
doing well on this fine blurs day thank
you for being here with me
i'm lily i'm a historian and a hacker
and i'm currently a security specialist
at thoughtworks you can follow me on
twitter if you like retweets about
poison or you can email me if that's
more your speed and yeah for the next 20
odd minutes we're going to talk about
javascript and vampires
what's my deal with vampires i hear you
ask vampires are notoriously unable to
enter a house unless they're invited in
and despite your best intentions and
while you might have been told never to
let a vampire in there are plenty of
tricks that vampires can use in order to
let you get them to let it get you to
let them in so disguises please cunning
ploys and when they're in they generally
want to drink your blood which may or
may not be a thing you actually want
in the same way you may sometimes find
that the web application that you've
been building is serving javascript to
your users that you didn't intend to be
there
and what that unexpected code does could
be good or bad
there's plenty of good javascript and
plenty of good vampires
friendly vampires will hang out around
the house and uh
you know do your laundry do your dishes
do the cleaning at night
while everybody else is asleep
friendly javascript is all the stuff
that makes your site work the way that
you want it to it makes your page look
nice allows you to build games inside
your app or see how popular your latest
post is let you chat with other people
but there's also evil javascript and
there are certainly evil vampires in
this case when you find one of these
things inside without warning they can
drain your blood your money your user
base
your processing power sometimes your
entire life
just so we're clear what's at stake here
needless to say most of us want to
ensure that we're only letting in the
good ones and keeping the evil ones away
most javascript will end up being served
from your app in one of three ways it'll
be embedded in the html call from
another file that you're also serving or
called from another file that someone
else is serving
in the security advocacy world we spend
a lot of time educating folks about the
pitfalls of third-party js libraries
that you might be bundling up and
serving yourselves and about how to
sanitize input to prevent cross-site
scripting attacks from your users but
when it comes to the issues thrown by
calling javascript directly from some
other domain entirely we don't pay
nearly as much attention in my opinion
and maybe
you're all over this and you're thinking
yeah lily i know it's just how the
internet works but if that's the case
and you think you don't need this talk
um i don't know perhaps there's someone
on your team or in your professional
life who needs to hear this and who
needs convincing
please stick with me because i'm going
to give you tools information and
arguments that should help make these
conversations with those people a little
bit easier
we see lots of attacks in the wild that
happen as a result of sneaky third-party
js the majority of the popular ones are
done for one of two reasons
mining cryptocurrencies and stealing
credit card details because the bad guys
really have no imaginations like
seriously today's browsers can access
your camera and your microphone and your
screen i'm sharing my screen from my
browser right now with you and this is
how you can see and hear me far worse
things are possible
but no cryptocurrencies
crypto jacking by which i mean using
third-party js libraries to mine
cryptocurrencies and users browsers was
fairly popular up until relatively
recently thanks to services like
coinhive whose whose entire product was
letting you use browser power to mine
monero
it's estimated that the actual amount of
real world money mined by these scripts
i don't know it's in a couple of tens of
dollars i think and in several cases the
attackers actually messed up the attack
and have succeeded in printing the
javascript javascript code directly onto
the page instead of executing it so
that's about the level of cunning we're
dealing with here
it's become less of a problem now
because coinhive went offline a few
months ago and while there are still a
few competitors out there none of them
have the popularity coinhive did
still a bad experience for the users of
those apps though and an overall bad
experience for the planet
mage cart is the attack i've seen the
most often making headlines
it was this attack that was responsible
for the british airways and the
ticketmaster data breaches of credit
card numbers and login details for more
than
400 000 people in 2018
and it cost british airways 20 million
pounds in gdpr fines but it's still
circulating in lots of places
it's a credit card skimming attack in
which attackers modify existing shopping
cart checkout code to process the
payment as usual while also sending off
all the submitted details in a tidy post
request to the attacker's server where
they sell them on to whoever wants them
the reason these attacks are possible is
because of the way the modern web works
now here's a statement calculated to
make a third of you feel really old and
to make another third of you think that
i'm really old
back when i was first messing around
making websites in 2001 when i was about
12 years old
one of the first things i learned was
that the primary sin a web developer
could commit was to hotlink your assets
that is to source a picture or another
resource directly from someone else's
website in your code firstly because it
was like stealing from that other person
and it would cost them a lot of money to
have to keep serving their picture over
and over to your users
and secondly because if that person
figured out what you were doing they had
the power to change that picture to a
different picture that said this
webmaster is a stinky butt head and then
that's what all your users would see
so if you flash forward uh god 20 years
we have evolved to the practice of
extreme hotlinking otherwise known as
content delivery networks or cdns
a major point of cdns is to host stuff
so that you don't have to and being
really really good at serving things to
you fast
but it's still hot linking it's still
asking your website to fetch things from
someone else's and trust that they are
what they say they're going to be
most of the time this tends to work out
unless the cdn service themselves
decides that they don't like the thing
you're doing or because they get
compromised or because they were never
trustworthy to start with
scott helm puts it pretty well in my
opinion using a cdn is consensual xss
and we're trusting the cdn not to do
anything untoward
we put lots of trust into these things
how thoroughly do most of us evaluate
what we're using
there are lots of cdns
that will work directly with you to
supply the assets you provide them to
your users around the world but there
are also lots of other third-party
services often built on the backs of
those cdns that act as public javascript
repositories that will serve any version
of any javascript library or anything
else in anyone's github repo or npm
package directly to your site as
executable code
all you have to do is source their url
in your html
i'm talking about services like js
deliver and unpackage and raw git many
of these sites are run by passionate
members of the community who want to
give other developers like them the
tools they need to make great stuff
but a lot of these services are run by
like one person or very small teams who
cannot possibly keep up with every
single script that people use their
service to share
this presents security issues not just
in terms of file integrity which we'll
get to in a second but with availability
if you're relying on one person to keep
the site up and renew the domain your
website trusts to serve it with code
that's a hell of a buzz factor
let's look at githack one of these
third-party cdn services the site's aim
is stated right at the top
raw.githat.com
serves raw files directly from various
source code hosting services with proper
content type headers
it's true that sites like github will
serve javascript files with content type
headers that are of type text plain
so that you can't directly reference
someone's raw file in your own site
because the browser won't treat that
file as executable
github does this for a couple of reasons
one of which is because they themselves
aren't a cdn
but i'd also guess it's because they
cannot be liable for actually running
whatever random code any one of their
millions of users decides to push at any
one moment
so if you go and reference this file
straight from someone's repo
you're saving a bit of time in uploading
and serving that file yourself but
you're also trusting that the
functionality of the code won't
substantially change and also that that
developer
or the group of developers with commit
access to that repo have all turned on
mfa and don't reuse the same password
across all their accounts and that they
themselves aren't intending to do
anything malicious
that is a lot of trust to defer
then you're also deferring trust to the
person who cased that version of the
file to the cdn in the first place
git hack has a bit of feature that lets
anyone purge at purge
cased versions whenever they want so an
attacker could come along and make sure
that their malicious edit to the code
ended up being the one that was cashed
rawgit a third-party cdn service that
githack seems to have been based on was
actually officially sunset in 2018
precisely because people were using the
service to poison the cache with
cryptojacking scripts and the single
person who was running the service
couldn't keep up with it anymore
despite it having been in the state for
about three years now i have still seen
raw get used by sites as recently as
several months ago which is a sort of
damocles if i ever saw one
apart from developer libraries and cdns
the other reason that third-party
javascript ends up in apps is most often
because of advertising and marketing
tooling
i'm talking about google tag manager
analytics tools advertising network
panels to make back a bit of money
and while i'm the first to admit that i
know nothing about marketing there are
plenty of reasons why people use these
tools that aren't just about vanity
graphs
the main reason with this type of tool
and why i'm calling it out specifically
is because these kinds of things often
get added outside of the usual developer
pipeline
google tag manager is literally code
injection as a service and it can mean
that marketers are able to update those
code snippets directly onto the site
and those snippets often then call on
fourth party js and then it's kind of a
free-for-all
this is also the kind of place where
mage card attacks are especially
effective
this is why it's really important that
you know how to spot your vampires
as a developer you might think you know
what code is ending up in the html that
ends up getting served to your users
but you may end up being surprised
i know that supply chain problems are
really hard to fix and at some point we
really do have to make decisions about
who to trust or nobody would ever build
anything ever because the overhead would
just be way too difficult to manage
but in most cases we can control the
sources of the code that will end up in
front of our users
this is low hanging fruit and we should
pick it
so in brief
here is how you spot a vampire
look from the outside in
as a pen tester this is what i do all
the time
we often don't have access to the source
code we see the app the way that a user
will see it and work backwards from
there whereas most developers will see
their site from the inside out knowing
how they've built it up and what checks
exist along the way
i suggest it might be educational to
take a closer look at your app and the
way it's presented to your users
use a proxy capture the outgoing
requests when the page loads and see
exactly where you're asking your users
browsers to go when they visit you
is everything you're seeing there
subject to the dependency checkers and
volume scanners that i know you've got
built into your pipelines or is there
stuff there that surprises you
i know that i only really became aware
of this problem when i stopped being a
developer and started being a pen tester
and you might have laughed before about
things like git hack and raw git but i'm
talking about them today because i still
see them get used in live production
websites even now all the time mixed in
with other libraries that people are
serving themselves
so when you've seen what you're serving
you face the unenviable task of working
out what to do about it next
don't worry i'm here to help you with
that
the first part is the technical
solutions of which i'm pleased to inform
you that there are plenty
i only have like
what 30 minutes
which is not enough time to explain any
one of these things in great depth but i
want to touch on them so that you can
refer back to this later and dig into
these if you need any of them this is
the slide
probably the one you'll want to
screenshot and go research later if you
need to dig into any of these terms in
more depth
there are plenty of mechanisms to put in
place at the web app and browser level
that will help ensure that you're only
loading the javascript that you intend
to load
i'd recommend going through and reading
about all of these if you don't
recognize any of them but prioritize
learning about sub-resource integrity
and the content security policy most of
all because they're the ones that are
most adept at saving you
in brief sub-resource integrity is a
mechanism for checking hashes of the
files you're loading to ensure that they
are precisely the things you intend to
load and then blocking them if they're
not
the content security policy does a lot
of other stuff that isn't just about
third party js but it lets you define a
list of domains and subdomains from
which your site will accept assets
loading
it can be handy if you've already pruned
out sources that you don't trust or you
can't trust and i would recommend also
not allowing things to load from stuff
like wildcard aws s3 because
literally anyone can host things in that
domain
be as specific as you can
the real solution here like so many
things with tech is people based
to stop the vampires from getting in
you are going to have to talk to people
and come up with a systemic vampire
management plan
this is the place to make your case for
doing js right
talk about the worst case you can use
the notable attacks that i highlighted
earlier if you like or just google mage
cart because there's stuff happening all
the time
talk about the user experience you're
presenting to your customers
if you can't guarantee that the
javascript that you're calling on
is the thing that you intend it to be
and most importantly
work with your like-minded colleagues to
come up with a strategic approach for
tackling this because in lots of
established businesses it can be a
really
really hard case to make to anyone whose
sole concern is not security
which is most people
in our ideal amazing golden utopia state
we'd select each one of our vampire
guests with incredible care
that is to say we would use only the
specific javascript that we needed we'd
host it ourselves we'd have a lovely
maintained register of all the stuff we
were using and all the dependencies we'd
automate our updates and our patches
nothing at all would happen outside of
the normal developer flow and we'd all
live happily ever after
but we know that real life isn't like
that and it's not always going to happen
that easily so once you've done you're
looking from the outside in that we were
talking about earlier and track down
what kinds of javascript your web app is
actually serving to people start looking
for your short medium and long term
plans for getting to that ideal state
this is going to vary depending on the
challenges that you face at your
specific workplace so please adjust
these to suit you
you might never get to that ideal state
and that's okay you're just one person
we can't win all the fights but see
what's within your ability to control
so without caveat here are my
suggestions
in the short term find out exactly where
all of that outside in javascript is
coming from do you recognize it all is
it all covered off in your usual
deployment process or is there stuff
being added in that you weren't aware of
if so go and find out who added it where
they fit into the thing and what their
actual use cases are
empathizing with these folks is going to
be really important too they are just
doing their best at their jobs and they
may not have realized what risks they're
really introducing
do your best to understand what they're
trying to achieve
and if there's a more secure way you can
help them achieve it
does this mean for example that uh
developers and marketers should pair on
google tag manager work
does this mean that we need to start
looking at whether we can host more of
our own stuff
this is the period of time where you
should start trying to get folks on side
tldr
don't be a jerk
medium term
trial some of these process changes and
adjust them where needed
this is also a good time to start
working on how to roll out some of these
basic technical controls i mentioned
earlier although i know that content
security policy implementations in
particular can be a real pain in the
butt which is why i say start working on
and not roll it out
by now you should also have a clear list
of what you're serving to people and at
least be monitoring that for any changes
and incorporating as much of it as you
can into your usual pipeline
this is also the point to start
assessing the cdns that you're using and
asking yourself questions about their
security practices their reliability and
all of that kind of stuff
are there any that you see that you're
serving that you could pair back
or can you at least use one source if
you're using multiple ones
see what works for you see what's
practical
in the long term you want to work
towards getting everything using sub
resource integrity at a minimum and
content security policies if you can
manage that
hosting this stuff yourself is ideal
from the perspective of being able to
strictly control the domains from which
you will accept javascript tightly
maintaining a list of third-party
sources and evaluating new ones as you
go will also help to ensure that things
just can't get pushed out to the site
willy-nilly
by this stage you should hopefully have
as much as possible under the one
umbrella developers and marketers
working together to ensure that the
scripts that are being deployed are all
serving useful purposes and coming from
suppliers that can make reputable
guarantees about their own security and
you will ride off into the sunset and be
very happy
and once you've done all this you will
be totally one thousand percent
unhackable and safe from all vampires
forever
the end
okay no you won't
there are still lots of other things you
need to keep an eye out for and security
issues never end
the security issues involved with using
third-party javascript
are one of a really small facet of the
larger issues that can occur with making
companies talk or making computers sorry
talk to other computers and for better
or worse we're all in this never-ending
game of cat and mouse forever
all we can do is our best and keep on
top of the risks that we're exposing to
ourselves and to our users in the course
of our work
i will be putting up a list of resources
on my website if you want to refer to
any of this or dig into a lot of the
sources that i looked at while
researching this talk later on um i
haven't put it up yet as of the time of
me speaking but that will be happening
very soon
and that's it i wanted to leave you with
one of my favorite vampire tweets um
thank you for listening and i'm happy to
take questions if there are any
thank you
that was really excellent and chat's
been going wild and you're probably
lucky we're not an in-person conference
because you may get your glasses stolen
if we were
oh dear
no problem are there any questions in
the chats i have
not been looking
yeah i'm trying to quickly scan through
no problem
[Music]
[Laughter]
i suspect many will appear uh probably
just as we finish being live because
that's how the rules of these things
work
well uh which is probably going to be
the easiest way
i got my glasses in tokyo
so
i hope that helps
it probably doesn't at all
well soon
one day
flights will be a thing people can take
again
well thank you very much we will be back
in
uh about 15 minutes at quarter past two
australian eastern time
uh so we'll see you all shortly
great
you