[Music]
hi i'm dania black i am the chief
innovation officer at the marina bay
foundation it's a pleasure to be at
linux conf australia here uh today and
especially talking about this topic
of mourinho and systemd and the
integration of it
just as a little bit of a back story
this uh funding of the initial work of
uh systemd capabilities into marietv uh
all six years ago seven years ago was
actually funded by the open source
developers club and so i'd like to you
know give a little blatant thanks again
for um funding the work and actually
getting me through a few months but
enough of the background
let's start off with a demo and where
i'll actually go through the aspects of
how
to set up a multi-instance version of
marie db
uh with the per user configuration
don't worry about taking notes they'll
be on the the website beside it and so
enjoy the presentation thank you bye
i'm sure you've all seen systemd
services before but i'll just show you
ready service for which begins with far
too much documentation uh just go
documentation is cool
uh we see that it's a type equals notify
service and has been since
the 10.1 days
we see that it's got a user and
group of my
sql there
um we see it's got a bunch of
capabilities
we see in the middle there the protect
home equals true which we're going to
need to address because we're doing
multiple instances
um of in the user's home directory and
we'll also scroll exec um start reading
if it's true that we also need to
address
um when we
do our service
so for the more a student of you will
notice actually on your system there's
already a bunch of templated services
are running on by default
and these are denoted by the at symbol
in the service file
the
element after the at symbol is the
instance name and if we look at say the
service file for the use of runtime
service we see that it's pretty much the
same
except it's got this bunch of
percentiles which is the templated
aspect and there's numerous
other variables but for the most part
we'll just be using percent i today
now marine db has had a multi-instance
service for a while that was
significantly improved
in
10.4 and above
and that
looks like this
so also begins with much
documentation
and there's much more useful
documentation here because there's a
number of ways a user may want to run
their multi-instance service
and we've included a number of ways to
configure all these easily
we
sort of set the the base line rather
consistent with this mysql the
multi-instance environment variable
and that
is used for not only the install db but
starting it also
once we've
got those two consistent we
can pretty much got a reliable system in
that way
there's a number of mdb
system variables or command line options
that
uh by default will actually conflict if
you try to run more than uh one instance
of radio in your system and they are the
socket the port and the data directory
our goal here was to run a instance per
user
and that means that the socket and the
data directory are quite easy we can
just run those in a user's home
directory
the port number is a little bit trickier
one way of doing it is to map
the
user id of the user and use that as the
port number um
on my system here like the default user
id is like a thousand which is less than
1024 so we're going to need to do a
little bit a bit of a trick here to get
that
to make the deployment of this a little
bit easier what i've got here is a
little template file
of the configuration that we're going to
plop into every user's home directory
that contains uh the user port and the
socket and the data directory the the
conflicting items
um well the user isn't completing but
let's um
put each on their own user for
just manageability
now integrate this
into system d by
uh in the service file and we're going
to do the system control edit media dot
service
and actually add an override file
this is a bunch of
options that override the
default configurations so it should be
fairly minimal
we've got our user group both set to
percent i
uh such that it's a user
of the system we've removed the project
home privileges
uh we've also removed the start
only permissions not only so our all our
exit priests
scripts here don't actually run as root
as before i mentioned the the mysql d
multi-instance
environment variable is used
by the
the marine to be installed to be in
starting of the service and we
to make it consistent
our exit pre-line here is
if the
configuration file doesn't exist we
crudely use
set as a sudo templating uh tool to
create the my.cmf file if it's not
already there
uh we see the trick that we did with the
port we grant the user's id added 3000
to it in a little bit of uh bash
arithmetic and um
and that's used for the port number
in our mood to be installed db uh we
notice we're using a
root authentication mechanism of socket
and a root authentic
socket user equivalent to the username
starting the service and we do that for
um the security to ensure that
the the unix user
has to be
um
[Music]
connecting to the socket for it to be
authenticated properly and i'll show an
example of that later
we skipped the testdb here uh
it's not only the testdb it's also the
anonymous user that we typically don't
want
because a user on the command line could
just go shut down and it will
effectively shut down the service we put
a restart always here so the service
starts up if they happen to do it just
to save some support calls
if we
name and reload
let's use
why this does it at this stage
we look to see what
files are there there's no sockets there
and
just to show that
there's no my.cnf
there by default
so if we start our service
udb at dan dot service
you can see within a second or so it's
it started
and if you look at the status
you see it's up and running we've
executed all our prescript we've got a
main pid
we've got a status of taking your sql
request now
so we may as well
if we look at our files there we see
it's created
those socket files
and in our configuration file uh that
one's also been populated
so let's connect
more dippy is the client thing uh
utility as a monitor
uh see it connects there
we do backslash s to show our current
connection
we see the current user is then at
localhost not surprisingly
we also see down the bottom
that the
unix socket is home
dan
marie dibby dot sock
if we show grants
we can see that we've got all privileges
on star.star with the grant option so
we've got all the privileges of
uh what's normally a user
we've authenticated using the unix
socket
however the
mysql native authenticate password
authentication is still there
um however it's got no password hash
currently so that bit's disabled until
we actually set a password
however for the most part it's more
secure this way because once we set a
password
another user on the system could
actually access it
this account using the password
so it's
securing its default installation
without a password set
so what about another user well
let's add tom
easy enough as a user and we start their
service
it takes
after the same time
and if we
look at the status we see it's up and
running with home
slash thomas the configuration file
um yeah and
for the most part it's identical if we
look at
the red processors on the system we
magically see there's two
remember
when we
looked at our home directory we see that
there's a moody socket
oops we've created that
with
global
uh re write permissions
so this means our user
tom
can actually connect to it
so if we do this
uh we see oh access desired for tom
but what if
running as tom we try to connect with
dan we get an access denied message
which is
what we wanted
so all that um functionality of the
multi-instance has been there
uh since marie dv 10.4 or in a slightly
uglier version um all the way back to
temp one
so let's move on to the next section
uh
for those who haven't seen there's also
a systemd unit type of sockets
and so what
these are if we
do system control this sockets is we see
a bunch of services like uh cups
uh
g bass device mesh in your sockets
that
don't need to be running all the time
like you may have on assist v things
however it's got this
uh aspect of a socket that it's listing
on
um so that the service can
be instigated when it happens
and there's a bunch of those
so if we look at what the mariju
socket instances are
this is available since like 10.6 just
just for reference
we see we can do a socket user a socket
group because we want the socket created
as the instance name
we want to be a bit more protective
about our permissions
we remove the protection at home
we create an abstract socket here for
mainly demonstration purposes that
abstract sockets and file system sockets
are supported and you can have a number
of those
what is also here is this marie db extra
socket service
and what marie dibby and mysql have had
by default is this concept of an extra
uh port
notionally
where you can connect to that port and
if your max connections is is
maxed out you'll still be able to
connect on this port without getting
refused and gives the admin the ability
to
try to resolve that lack of resources
uh status
and this is a feature that's only in
really be under
the
uh extra
uh
under under systemd
because it hasn't quite been plugged
into the wall
you may have seen those actually roughly
look the
same
uh but there's a minor difference
actually if we
look at the contents of that cat service
and that's his uh file
descriptor name equals extra
and
inside the moody big code it actually
recognizes that and and treats it as the
special kind of service that it is
so you'll notice at this stage we've
actually dropped
the
reference to the tcp port and we've done
that
um largely out of latentness that there
wasn't an easy way of
grabbing
a
uh user id and in the the language of
the systemd configuration uh converting
it to an id but
um
all your connectors actually
do
unix socket authentications
so what does it look like well let's
first
stop the
current
i'm ready to be at dan.service
and we do that and if we list that that
so the socket's gone the data directory
is still there
excellent
so what we can do now is we can
uh look at
these
yeah look at uh start that service and
start the extra
well
let's say service but really i just mean
socket and if we look at system control
status on
dan marietta socket
we see that the socket is active and
listening
when it's connected to it will trigger
the marine to be at dan.service
and it's listing on that abstract socket
and that file system socket
and that's it
so if we look at what the service looks
like now under systemd
we see it's currently inactive
however it can be triggered by these two
soccer services
because they're they're started
and that's fine so let's do that so
let's do marie db
uh and if we do that we do a connection
and magically it's up and running um
quite quickly item ad
imagine you noticed
so what if we do a shutdown now
click out and look at the status
of the service
we see it's still there
in fact
yep it hasn't shut down in fact it
restarted in that time
uh remember before we had uh restart
always in the service
now that we've actually got socket
activation
we don't actually need that restarted
service anymore so let's comment that
out
shut down
service
and look at status
it's inactive and stopped but it's still
triggered
and we'll do that again for fun
that you know we can just connect and
automatically starts up
so great
at that point we're actually relying on
our users to do the right thing and shut
down the service
when they're actually finished on it
which isn't really the uh
the the best way to do it so let's
encourage this
um
environmentalism along a little bit
so remember our template file here if
in
if i paste the right thing
we see that in 10.8 on the break that
i'm currently developing there's a
option
for max idle execution time
and this means after 10 seconds of there
being no connections and
no sql actually being executed it will
shut down and it's up to you to decide
if 10 seconds is
appropriate or whether i'm just using
that in my
example so to get into effect we're just
going to remove it there
and we're going to stop
the ready at dan.service
um so this means the next time it'll
just populate in
but we've still got actually our
socket activation as we can see from
this warning below
so what we can do is we can
connect in
and we're going to use the abstract
socket
for this time so
we can connect in
and we see that it starts in the usual
time
and then actually regenerate our
configuration file
if we look at the
system variables the max idle execution
the weight timeout and the interactive
timeout the max idle execution time that
we set
gets populated through to the wait
timeout in the interactive timeout
now the interactive timeout is for
terminals like this
that if i blabber on enough it'll
actually disconnect
and the wait timeout is for what's used
for your normal programming connections
that if you've got connection pooling or
something
um after 10 seconds it's gonna drop
out
and you're gonna have to reconnect
so i hope your applications handle that
now that i've waffled on for a note if i
do a backslase s
uh we can see
that it went away as expected
because it's october activated it
instantly came back and we've got an
uptime of zero
so if we do a select one
exit out and show what happens in the
status
as i
it's running it's still running still
running still running
just call this pretend live demo mode
and at a point you know it'll be
inactive and dead again
um all successfully deactivated down the
bottom and it's ready to be connected
um again when ready
so that's uh in development thing that
for 10 8 but
it probably won't make 10 it'll probably
be 10 9.
so how do we make these kind of services
permanent well
uh
that
the instinctive response is to use
something
like
a
system control enable
um mariedy be at dan.service uh however
this not quite right and systemd
friendly they tell you
that no it's not right right uh
the essence of it is that socket
services socket
configurations are there to trigger real
services
so we'd enable it like a real service
and and that's what we do
and if we look at
what the status of say liberty
services it's exactly the same
it's got a number of different sockets
that trigger the service
and that's what you expect
okay we're back live um so a couple of
questions for daniel from the um
thing uh the first question is why not
docker containers or something
resembling docker
uh
well i thought this would just be kind
of an interesting way and the the point
of actually doing socket activation is
that in in a default thing you've got
actually
nothing running apart from consistency
listing on it on a few sockets
um so that sort of
reduces your your memory footprint
significantly if you're uh constrained
that way um and while there's like
kubernetes and other ways to instagram
uh docker there
um at least to my understanding a little
bit more complicated so i thought it'd
be fun to do it this way
i want to mention that i'm actually
doing
uh
the docker library maintenance of
marie debbie as well so i i'm not
totally in the dark about it
and yeah this
socket act sorry the um auto deactivate
that i'm working on getting through
uh any sort of actually plays a role in
making containers into that serverless
aspect so they can auto deactivate and
and in cloud environments start to
benefit
um the the low memory use there
so that's the main reasons
okay
and
next question is what is the use case
for installing mariadb like this
um
additional flexibility if you've got
other requirements or a lot of
your workloads already in um system d
and
if there's low skills i guess in the
organization around containers maybe
this is a a way to you know start off
the segregation um to get
either the
the organizational company actually used
to
the aspect that we can actually provide
a strong separation on
uh users databases
quite well and actually hand out
privileges and
things a little bit more um
maybe you don't quite trust all users to
to run up as many containers as they
want
or don't know how to control that
resource allocation
uh well you know the aspect of having it
as a service gives
a rather limited
but flexible capability to
run
ready to be in its own config in a
number of ways
okay uh next question is
users that might run might want to run
their own server still need sudo to make
it happen any way to move it all into a
user instance
of system data
yeah i've thought about user instance as
a system d i just haven't
um
there probably wouldn't actually be that
many changes
on that it's the same sort of thing
about
uh removing the start proofs only and
a few of the protect homes and
protect systems obviously don't apply in
the user instance but
for the most part the the template on
how to run a user instance system z
ray db is
rather
comparable to the the one provided
okay that's all the questions uh there
any any other questions you think people
should have asked
[Music]
trying to cover it most of um
i guess i could do a big talk on um you
know uh foster men oh sorry on uh
containers and how marie jimmy is
developing on those but i've got to
foster talk on that and people can wait
till next week
go and see that
but okay yeah so thank you everyone for
attending and listening along um and
asking questions
much appreciation okay
thank you very much daniel and thank you
good luck next week at foster
much appreciated bye