foreign
we are delighted to be welcoming Amanda
remotely so Amanda are you there yes
fantastic hi Amanda can you hear me
I can thank you so much I'm delighted to
be here awesome and our audio levels are
great in the room to hear you as well so
without too much delay we're gonna hand
over Amanda take it away
thank you so much hello again and thank
you everyone for joining me today to
talk about navigating the murky areas
where ethical legal and policy
challenges present when measuring open
source
um so to start off some admin notes for
Those whom it helps you can find these
slides and very detailed speaker notes
at bitly that's bit.ly backslash OSS
Dash
uh
d-r-a-g-o-n-s dash
r-a-i-n-b-o-w-s-n-e-k-2023 dash lights
um I would also like to begin by
acknowledging and paying respect to the
traditional owners of the various lands
on which I am calling from today I'm
joining you from the land which is
served as a site of sustenance community
meeting and exchange among indigenous
peoples since time immemorial the
Western Abenaki are the traditional
stewards of these Forest lands and
Waters in which they call indakina or
Homeland we respect their spiritual and
lived connections to this region and
remember the hardships they've endured
both past and present including violence
and forced displacement at the hands of
colonizing peoples As We Gather for this
physically dispersed and virtually
constructed meeting as we share our own
knowledge teaching and learning
practices may we give thanks for the
opportunity to share in the joys of this
place and protect it
I would also like to let everyone know
that I will be speaking today both
directly and indirectly about exclusion
the potential and reality of harm
against marginalized groups and how
technology can be used to identify
people who have not chosen that if these
topics are best covered elsewhere in a
different format or not at all for you I
wish you all the warmth and best
community this space has to offer you
will have time before I get into these
topics to not be present anymore in the
space if that works best for you
so some context about myself I am a pale
white woman with light hair and eyes who
wears glasses uh I my hair is different
now
um I am a researcher and engineer at
Google where I'm currently leading a
team focused on research and education
and our open source programs office
I'm also one of Google's co-leads for
project ocean and across with the
indomitable Katie McLaughlin who I know
is also part of this lovely Community
external faculty for the Vermont complex
system center and I'm also a co-founder
for open source stories with Julia
farioli I sit on the board of directors
for the computational Democracy project
I once wrote a book with Alice Zang on
feature engineering I'm also queer a
very proud mama of two smaller humans a
U.S Navy veteran and I'm lucky enough to
live currently in the state of Vermont
in the United States
and while I am fortunate enough
to be here my state and the areas I love
continue to experience life-altering
flooding as the direct result of human
accelerated climate change if you want
to learn more about what Vermont is
experiencing and how you can help
Vermont Public Radio has consistently
the most accurate coverage and updates
for our local communities and those who
care about us I tell all of you this
because it's important for us to share
and understand where we come from and
since I cannot be there with you today
it is important for you to know where my
heart and my head is as we talk today
so to kick off the conversation
um I I wanted to talk about where I
started with working in the space in
2019 I was working in Google Cloud as a
developer relations engineering manager
with a wonderful team and a nagging
feeling about the general understanding
of how open source worked
um so basically I had this kind of
research idea that I couldn't figure out
where to put or what to do with and
fortunately many of my friends from
Google also frequent the half Bakery of
ideas and loved to introduce you to the
friends you haven't met yet and this is
how I met Julia farioli of the open
source programs office together we
developed and pitched the idea for what
would become project ocean a pilot for
the team and moving forward academic
industry and community research of Open
Source at a global scale
so despite launching a new initiative in
early 2020 we were still able to achieve
many of the goals we had and the work
continues with a range of internal
academic and Community Partners today
one of our most audacious goals was
mapping out the entire open source
ecosystem and then sharing it with
everyone and honestly at the time it
didn't feel that audacious because we're
Google we know how to organize
information and make it accessible and
useful or so I was told
uh but one of the first barriers we
continued to run against was defining
the problem space of Open Source with
our stakeholders and the communities we
work with there are many mental models
and analogies about open source that
exist
some of these models Aid our
understanding of a complex ecosystem
however we found that we don't challenge
these enough to truly investigate the
risks and the problems in open source
now rather than continually building for
how we've been burned previously we lack
sufficient baselines to gain a true
census of how many people are involved
in open source how many people are
holding up our technical world and where
we might be at risk of the whole modern
tech Sac simply falling to the ground
organizationally this means that we
can't move forward effectively and
sustainably until we break the popular
but over reductionist model of modern
digital infrastructure
so this XKCD comic is frequently shown
to demonstrate the gaps that exist in
both our understanding and our failure
to adequately respond to the challenge
of Shoring up open source more often
than not it's used to show that the
brittleness of the open source ecosystem
and why further large-scale investment
is critical
my counter to this comic as developed
with Nikki ringland is that the reality
of Open Source is that billions of
dollars are already spent globally on
sustaining open source so the initial
comic has been used effectively to
centralize buckets of money attention
and power and to build walls against
organizationals around organizational
spaces for collaboration
however it isn't that we aren't
investing is that we are making
localized decisions about investment and
support for open source while we lack a
resilient way to approach a global
decentralized and equitably resourced
ecosystem
black swans of Open Source past are
frequently cited as the risk we will see
happen again if we don't invest in a
centralized initiative now now and maybe
yesterday and this may be true or it may
be that despite any amount of investment
or organization the next problem will
present itself in a way that will also
only be clearly visible through
hindsight
so this brings us to the impetus behind
mapping out open source ecosystems as
Julie and I enventioned it for project
ocean quite simply we are looking for
that magic spot of any research which
allows us to have better data and better
models so that we can make better data
informed decisions we want to know where
we are so that we can work collectively
in our communities to figure out where
to go next
so along the way with my many
collaborities and colleagues as I said
including your very own Katie McLaughlin
who I believe We call glassent we have
found ourselves frequently in the spots
on the map that remain unclear where
there's no clear policy or approach to
navigating the data and Abstract
representation of Open Source so this is
where cartographers frequently Mark as
the RB dragons on their Maps the fuzzy
places which may hold treasure or danger
and so today I'd like to walk through a
few of these areas specifically around
the ethical legal and policy challenges
we continue to navigate and as much as
I'd like to have clear well-lit paths
for you to also navigate these Waters I
don't I can share what I've learned what
we should talk about as a community and
what questions researchers can ask
themselves When approaching the same
challenge My Hope Is that together we
continue to challenge assumptions avoid
the most harm and find the best way
forward each time we approach the home
of a dragon
so first let's explore some of the
ethical challenges which arise when
working within research about open
source
one of the key cultural tenets of Open
Source software and the communities that
build around it is working in the open
this place is much but not all of the
conversation about these communities
online in a way that is visible to
anyone who wants to join the effort
this is the intent be open to make work
community and Innovation transparent
there's a subset of the larger open
source movement which identifies as free
software one could argue it's the
opposite way open source is a subset of
free software we're not here to debate
that today but here free hinges on the
concept of Libre rather than tied to a
cost savings measure this is a very
different concept than open or free
meaning please scrape our data and tell
us about ourselves there's no Universal
consent form that any one person or
Community is signing up for when placing
their information online yet this is how
we've been advancing our knowledge of
the problems that exist in open source
and advocating for their resolution the
ethical challenge here when and where
can we assume consent for online content
to be analyzed how do we even find and
respect these barriers when they are so
unclear
this brings us to the next ethical
challenge which primarily applies to
folks working in institutions with irbs
for those not familiar IRB stands for
institutional review board
which review research studies to ensure
that they comply with applicable
regulations meet communally accepted
ethical standards follow institutional
policies and adequately protect research
participants
and as we just talked about the lack of
explicit consent to be a test subject in
aggregate there is an Institutional Gap
that many irbs do not require review of
analysis and research when it is
conducted in aggregate from metadata
such as the kind that you collect from
platform apis
so even if the markers of the metadata
could potentially be identifying such as
mailing list information where email
addresses and location data do not
adequately provide anonymization
individuals whose work is represented
are not considered to be research
participants for data which already
exists and is open to many
so this brings the ethical challenge for
us when and where do we add and how do
we identify when a research study should
be reviewed by IRB and how do we
advocate for that kind of oversight
within the research Community when is it
beneficial to the group being studied
when is it actually not a necessary step
towards framing ethical research
problems
this brings us to the next step of
problem framing humans are complex
creatures as much as it serves reducing
the problem space no one person readily
sits in a single dimensional cluster
when we fail to recognize this we are at
risk of problem framing becoming too
reductive we have to ask when and where
are we actively erasing people or
reducing their identities to vectors
which become harmful when applied to
policy one for all is not all for one we
must continue to be conscientious of
categorization and reductionism event of
identities to single categories
so continue on with the concept and
challenge of self-identification markers
we must always be aware that people from
vulnerable communities May and do
intentionally separate their identities
across multiple online communities and
spaces now more than ever we have to be
aware of how our aggregated work May
unintentionally Place someone from a
digital space into a place where
physically they may be subject to real
world consequences because of who they
are and how they exist in the world
so this brings us to a fundamental
technique of working within Acro open
source research we're combining
information across multiple Trace
systems to gain a more complete picture
of a community is a common technique for
example
if we want to see how software changes
over time or Community changes over time
we may combine data from working
software repositories issue trackers and
maintainer mailing lists
since not every system identifier is the
same we can use anti-aliasing method
methods to consolidate work to
individuals
here I refer to anti-aliasing as
sampling techniques to combine
information to single identifiers
the ethical challenge
question this brings us is does
anti-aliasing across data sets
potentially create opportunities for
harm for members of Open Source
communities
we talked before about how individuals
may separate themselves online across
digital spaces one of these specific
separations is when a trans person who
works in open source moves their work
and digital identity as they come out as
trans they may not change over all their
professional Community work from their
dead name to their present identity when
this is uncovered through inferred
methods how do we honor the person and
their life who we have unintentionally
outed beyond their original trusted
community
the impetus and the work is not on them
it is on us as researchers to make sure
that we are honoring the people in the
spaces we are working with
so we're going to pause on ethical
challenges for now to wade into the
waters where lawyers hesitate to be
cited
a quick note uh this is a picture of a
lawyer who accidentally used a chat
product filter to show themselves as a
cat during a trial online in 2020. now I
am not a lawyer and I am not a cat but
any good lawyer cat gets super fidgety
when Engineers talk about label matters
so I will not get into exact specifics
today but I will generally refer to
things about regulations and laws and
other good ideas which I think maintain
good social structure and I approve
wholeheartedly of when there is fairly
and not used against people to be able
to keep them in oppressive States please
consult your own lawyer cats as needed
for further clarification advice and the
Dark Matters to avoid
again I am not a lawyer and I'm not your
lawyer but when I talk with lawyer cats
about data and what is data and where
the law sees what is data we get into
some very twisty Charney conversations
there is no Global legal standard on
what is information what is data and
what is the differentiation between data
and a data set
so even more complex is that there's no
Universal legal standard of what data is
who information belongs two what is
public what is open and how we can apply
that across all the different use cases
so when you see open data you have to
keep in mind that open to use may not be
open to us depending on who that us is
so yes Amanda you may say but what about
this concept I hear about in copyright
law which refers to fair use not like
anyone's talking about that these days
again this is not standardized across
every country institution organization
and Industry the most recent advances in
text generation of summarization have
Revisited and accelerated the challenges
to what exactly constitutes fair use so
if you want to give your favorite
intellectual property lawyer cat a
headache ask them what's currently
changing or up for change in legislation
regulation and case law surrounding fair
use and large language models I've
included in the references one recent
Viewpoint from Van Lindberg who also
wrote a book on intellectual property
and open source
uh in the meantime
there are licensing standards that exist
and are used by open source communities
to explain how you can use the
information they post online where we
get into legal challenges here is what
license and terms and condition applies
where and for what use so open source
software by definition falls under
licenses as outlined by the open source
definition and held by the open source
Institute however metadata about the
software which is the data exhaust
created by the platforms they use
proprietary or otherwise and the license
which apply to the data they use can all
be different so in addition some
information depends on use commercial
versus non-commercial some depends on
explicit licensing or what it means when
no license is attached at all but the
data was created by the citizen of a
specific country eliminating the step of
untangling these license and use cases
is not something that can be skipped
when designing a research problem
concerning open source
we've talked about ethical and legal
challenges which are necessary building
blocks to begin to address where policy
challenges present themselves when
researching open source communities
that was necessary this is a lot
okay so one widely used governance model
within open source is to have large
foundations capital L capital F Shepherd
projects and portfolios for open source
maintainers enabling them to offload
some of the work to maintain a globally
used technology product
however this brings us back to the
problem of aggregated consent and
individual consent when working with
data about people
when communities choose to work with
larger foundations sometimes a large
Foundation capital L capital F when does
that Foundation begin to represent the
individuals and Collective groups within
what do communities give up when they
join a larger organization and what do
they retain who ultimately gets to make
this decision do they know they're
making that decision do they know who
they're making that decision for do they
know who they continue to make that
decision for and when do they get to
change that should they decide
can foundations sign up projects and
maintainers for larger initiatives
research studies can they change
mandatory communication and operational
platforms terms and conditions without
the consent of the individual maintainer
communities
I file this under policy because we may
not always be aware of what visibility
and privacy we are signing commitments
to when we are joining a large
organization regardless of the good
intent of either the organization or the
project maintainers themselves
so what does
what do we require of marginalized
people in a community when a larger
organization ethically drifts away from
their best interests
do they have to bring these themselves
or do their identities forward to
confront those changes how are we
thinking about members of our
communities when things start to drift
and we have ethical challenges that no
longer align and how do we disentangle
then from commitments that we may have
signed
this all culminates into how we create
Community consent models and care of the
commons which allows for the people who
create the data to have the most control
over how it is used who has access to it
and for what purposes the platforms both
public and private both commercial and
non-commercial that a community chooses
to use ultimately places additional
policy and conditions on their members
in a way which not every person may
fully agree to
as researchers this returns us to the
fundamental concept of building with and
not for
every open source project is a part of a
larger socio-technical system we cannot
ignore the people for the bits
speaking of not ignoring the people for
the bits you cannot ask me to talk about
policy without talking about open source
I absolutely cannot leave any public
space these days without taking the time
to reinforce these three simple words
that open source matters as someone who
works for a big Corporation I am highly
and always aware that not only does my
precise language build trust but the
washing and generalizing of language
allows all sorts of Mega evil tyrannical
acts to use their power and privilege to
avoid accountability and responsibility
thankfully for me two of my included
references including why open source
matters and just released today open for
business big Tech concentrated power and
the political economy of open AI speak
to this important issue even more
thoroughly than we have time today I
would encourage every person here and
everyone who is not to read these
thoroughly and consider what is in your
power to take action on the ideas and
challenges presented by them
so while these challenges haven't been
solved in the last 20-ish minutes I do
want to offer some tools to help move
open source research and understanding
forward
uh absolutely Shameless self-promotion
together with Julia farioli and Juniper
Lovato we recently published a set of
best practices for open source
researchers we cover recommendations in
the areas of data ethics research best
practices respect and Equity
considerations and how to approach
ecosystem Integrity while avoiding
reductionist thinking this is not an
exhaustive list but we hope it is a good
starting point
if you have appetite for a book one
paper isn't enough my first
recommendation remains data feminism the
2020 book by Catherine de Ignacio and
Lauren F Klein it is expert nuanced and
highly critical of their own methods
which I always appreciate
and if one book isn't enough VM pursuers
book Forge your future with open source
will give you a deeper understanding of
Open Source Community work how it's
grown over time where practitioners
present and what kind of workflows they
share online and why this isn't a book
to teach you about research best
practices but rather to teach you about
how open source is the way it is now and
where you can meet ecosystems where they
are
and of course the breadth and depth
searched through all of these references
while pile up your paper pile even
quicker than I can including the one
that's taking over my dog's bed
to end I want to encourage you to keep
asking yourself and each other the hard
questions about how we design problems
what data we collect and the conclusions
we draw about these ecosystems
after all we're all in this open source
raft together
thank you
thank you very much Amanda and can I
just get everyone to keep uploading
[Music]
[Applause]
[Music]
Amanda thank you for your persistence uh
presenting remotely it's difficult to
present to a non-existent audience but I
can assure you there were giggles at IP
property lawyer cat so thank you very
much for your talk we do have some time
for a question or two if there's
anything from the audience now I'm going
to squint and see if we can see through
the lights are there any questions here
Amanda there's no immediate questions I
will share with everyone that Amanda
will be more present uh in Discord
throughout the conference and is more
active on Mastodon than elsewhere so if
you'd like to reach out after the
conference or during the conference you
can do so online uh Amanda we will take
a break now thank you very much for
joining us remotely and have a wonderful
rest of your evening
thank you so much
thank you everyone