\documentclass[11pt,a4]{article}
\topmargin=-1cm
\oddsidemargin=-0.04cm
\evensidemargin=-1.4cm
\textheight=24.0cm
\textwidth=17.0cm
\columnsep=0.9cm

\title{Inside Microsoft Networking\\
\small{all the dirt on the SMB protocol}}
\author{Andrew Tridgell\\
E-mail: tridge@samba.org}
\begin{document}
\maketitle

\begin{abstract}

The SMB protocol has a long and tortuous history which is
unfortunately reflected in the structure of the protocol. In this talk
I'll describe the internals of this protocol, which is now used as the
central pillar of millions of networks worldwide.

\end{abstract}

\section{Introduction}

The SMB protocol is now one of the central pillars of local area
networks worldwide. As the default file sharing and printing protocol
for all Microsoft OSes it is hard to turn on a sniffer without being
hit by barrages of SMB packets. Despite this, many system
administrators know very little about how SMB works. This makes their
job of fixing problems much harder.

\section{The anatomy of SMB}

The SMB protocol is rich protocol with many strange extensions and
conventions. It is transport independent, with implementations
existing for a wide range of common transports including NETBeui, IPX,
DECNet and TCP/IP. I'll concentrate on the TCP/IP implementation
because that is what I know best and it certainly dominates new
installations of SMB.

SMB over TCP/IP works on 3 different ports. Strictly speaking much of
the traffic on the first two ports isn't called SMB, it is more
properly called NBT (Netbios over TCP/IP) but I hope you'll forgive me
if I refer to the lot as SMB occasionally. They are all so intertwined
that it really doesn't matter very much.

\section{NBT on UDP/137}

The first port is UDP/137. You can find a fairly good description of
what happens on that port in RFC1001 but basically it is the name
resolution port. Computers send name queries on this port and get
replies from the owner of the name or from a system setup as a
centralized name server. Name ``claiming'' is also done on this
port. That fact that NBT has name claiming at all immediately
differentiates it from traditional Unix networking protocols. By
default no computer owns any names on the network and a computer, when
switched on, must go through the laborious process of staking its
claim in the netbios world by sending out name claim messages and
seeing if any other computer disputes the name. 

The security conscious among you will immediately see all sorts of
nasty possibilities with a name resolution system of this sort, but
really it isn't too bad. On a LAN it is no worse than the address
resolution protocol (ARP) that ethernet LANs use and on a WAN you can
block any nasty name resolution datagrams from coming in by blocking
UDP/137. There are some consequences to this due to some amazingly
silly hacks Microsoft have put into the protocol over the years but on
the whole blocking UDP/137 from external networks is a very good idea.
Those of you coming to the tutorial might like to ask about some of
the silly things that are affected by blocking UDP/137, I don't really
have room to describe them here.

Anyway, back to UDP/137. The two big things you need to know about
name resolution in NBT is that it is a claim/defend system (and thus
has a dynamic naming system) and that it can either operate as a
broadcast based system or as a point-to-point ``big central server''
system. The former should be fairly obvious and is like ARP. The latter
is called WINS and is rather similar to DNS except that names are
dynamic. 

\section{WINS}

The similarity with DNS isn't just cosmetic. It may surprise you to
know that DNS and WINS (and broadcast based NBT for that matter)
actually have the same packet format! The idea (as described in
RFC1001/1002) was that NBT would eventually be folded in with DNS in
servers so that a single daemon could serve both. That didn't happen
(and is unlikely to ever happen) but I suppose it was a nice aim.
Unfortunately it also led to some rather strange bit twiddling and
name encoding in NBT. That is often the price you pay for
compatibility. 

The knowledge that DNS and NBT share the same packet format probably
doesn't help you much really. In so many other ways they are very
different. Unicast NBT (known as WINS) does use a central server but
doesn't take advantage of the hierarchy system that DNS has. As
originally deployed WINS was really used on small WANS with maybe a
few hundred to a few thousand computers. These small scales are
crucial to understanding a rather nasty design flaw in WINS, it has a
flat name space! With so few systems it is easy to choose a different
name for each computer on the network so the designers/implementors
decided to use the domain field (the bit after the dot in the
computers name) to divide administratively separate sets of computers
in such a way that two computers with a different domain can't talk to
each other at all. They just don't see each other on the network. They
also called the field a ``scope'' instead of a domain to differentiate
its meaning a bit. This allowed people to have two computers called
FRED on the same network without their names colliding and without
requiring users to navigate their way through a tree like
structure. The downside is that WINS is totally hopeless for really
large WANS (like the Internet). Imagine the fight over who gets the
name WWW!

There is one other really big difference between WINS and DNS. WINS
(and of course NBT) has the notion of ``name types''. This is really
just the 16th (and last) character in the name but it plays a crucial
role in how the whole system hangs together. The name type basically
says what purpose the name has. For example type 0x1D is the ``master
browser'' type and is used by a computer that collates names on a LAN
so that everyone can quickly work out who is out there at any one
time. Type 0x03 is the ``user'' type which is registered so that
logged in users can be contacted by utilities such as WinPopUp. There
are lots more but you probably get the idea already.

The fact that name types play such an important role in NBT is what
makes it so hard to map DNS onto WINS. Lots of people suggest doing a
simple DNS->WINS mapping (and Samba can in fact do this) but don't
realize that this brings with it the problem of what you do with name
types. Often there are two names which differ only by their name type,
so how do you work out which is where using DNS? 

Ok, so why do people talk so much about WINS? The big thing that WINS
gives you that broadcast NBT doesn't have is the ability to operate
somewhat reasonably across subnet boundaries. The fact that WINS uses
point-to-point (I also call it unicast) communications means that IP
routers can handle it and you don't have to bridge your entire
network. That is a big win. Is there anything much else to WINS? Not
really. In fact the first implementation of WINS I did for Samba was
totally accidental! The NBT daemon in Samba (confusingly called nmbd)
just didn't bother checking the bits in the packet that say whether it
is a unicast or a broadcast. Then when WINS started becoming used on
Samba networks people wrote to me to say that it worked great as a
WINS server. At the time I hadn't even heard of the name WINS so I was
a bit confused but after I saw some sniffs I realized what was
happening. nmbd was just sending back reply packets to the originating
computer and they were being accepted (perfectly correctly) as though
the whole thing was happening with broadcasts.

Before leaving UDP/137 I should mention one final peculiarity. The
computer names are ``compressed'', or at least that is what it is
called in RFC1001/1002. The compression system isn't the most
efficient that has ever been invented, in fact it doubles the length
of all names! Maybe compression refers to the fact that it reduces
the alphabet size from a 8 bit range to a 4 bit range. Each character
is split into two 4 bit nibbles and is added to 'A' then the whole lot
is concatenated. This certainly makes the names look strange in a
sniff but it does mean that the ``compressed'' names are legal DNS
names. I've never actually found a case where this has been important
so maybe the designers took the notion of DNS compatibility a bit too
far.

\section{NBT on UDP/138}

Enough about port 137, I'll now talk a bit about UDP/138, the browsing
port. Port 138 is one of the weirdest beasts in the TCP/IP world. It
actually has a mixed byte order! This comes from the fact that the
headers are NBT, which uses the traditional network byte order in
order to be compatible with DNS, but the body consists of SMB
transaction packets which are in Intel byte order reflecting their
heritage. Oh, what fun it is dealing with that in a bit of portable
source code!

Port 138 is used mostly for browsing. As browsing is a over used word
I should explain that in this case I am referring to the thing you do
when you click on network neighborhood in a Windows95 box and it
brings up a list of workgroups and computers in those workgroups. Or
at least you hope it does. This ``friendly face'' of Windows networking
is a source of endless frustration for system administrators fighting
for some machine or other to magically appear in that list. I
sometimes think there is a market for a ``browsing problems 'R us''
company or even a whole industry.

The heart of the problem with browsing is its distributed election
based nature. It works by having a computer on each network (the local
master browser or LMB) whose job it is to collate the list of
computers on that network and make that list available to anyone that
asks. This sounds simple enough until you realize that exactly which
computer performs this task is decided by an election system where the
computer with the biggest number wins. That computer might be your
marvelous server with great uptimes or it might be Fred's computer
with the dodgy ethernet cable, the one that loses half the packets
when Fred leans back in his chair.

There are some heuristics that are used to try to make the election
somewhat sane, such as preferring computers with later versions of the
browsing software, and computers that have been up for longer or that
have been marked as ``preferred master browsers'' but it still is a
very hit and miss affair. You even find situations where two computers
both think they won the election which leads to consequences not
unlike those for the same situation in human politics.

Apart from elections there are other nasties lurking. After winning an
election a computer starts listening for ``announce'' messages from
other browse capable systems so it can collate them into a list. If
that list gets a bit bigger than a pocket calculator could handle the
browse master starts appointing deputies to try and spread the
load. These deputies also collate browse lists and one of their
addresses is given to any computer that comes along requesting a
current list of computers. The idea is that the load will be spread so
that the browse master doesn't get overloaded (hardly likely with
current CPU/network ratios!) but the effect is that you now have not
one point of failure but many. Even worse, the failure is intermittent
because you don't know which of the deputies (called a backup master
browser) you are going to be referred to. There are some good points
to the system, such as smoother transitions when the master dies, but
the deputy system certainly does make it a bit harder to debug.

Even if you cope with these problems you will someday be faced with
the nastiest of all browsing setups - cross subnet browsing. This is
where you have a single logical workgroup (group of computers) that are
spread over multiple broadcast networks so they can communicate with
unicast packets but not broadcasts. In that case each subnet elects
its own master browser which in turn elects deputies and you need some
way of getting all these separate lists together so users can ``see''
computers in other subnets in their browse lists. 

The problem is that the browsing protocols were never really designed
for this sort of cross-subnet setup. The broadcast election system
won't work for multiple subnets, so how do you do it? This is where
WINS and a thing called a domain master browser gets in on the
act. WINS allows you to have a central name server for netbios names,
so what happens is that a specially configured computer (the domain
master browser, or DMB) registers the name WORKGROUP\#1B with the WINS
server and local master browsers contact the WINS server to find out
who the DMB is. Then the LMBs can contact the DMB and arrange to
exchange browse lists so that (eventually) all subnets know about all
other subnets computers.

This sounds OK in theory but can be a real mess in practice. The first
stumbling block is placed at your feet courtesy of Microsoft marketing
decisions. Microsoft likes selling lots of copies of NT server so they
made sure that it is needed. A Win95 box doesn't know how to contact
a DMB (even as a client) so if it ends up as the LMB then that subnet
is effectively cut off. Oops. It would have been real easy for
Microsoft to include DMB client support in Win95 but then of course
they would sell fewer copies of NT server. Chalk one up for marketing
madness. The result is that you need either a NT server or a Samba
server on each subnet to do cross subnet browsing (at least if you do
it the way Microsoft intended, Samba has a few extra tricks that allow
it to work without a server on each subnet).

Even if you do have a DMB-client capable machine on each subnet and
make sure it becomes the LMB you are not out of the woods yet. You
have to contend with the problem that the protocol only allows the LMB
to contact the DMB for its own workgroup, not any other workgroups
in your organizations. So if you have different workgroups for
ACCOUNTING and ADMIN then the DMBs need some way to talk to each other
or you will never see a workgroup list containing both
workgroups. Unfortunately that bit of the protocol (which would have
been very easy to do) has been left out completely. Oops again! The
only way it can work is if there happens to be a subnet somewhere in
your organization where both workgroups coexist. Then the two LMBs
will notice each other the two workgroups will magically start to see
each other. 

For Samba we have added a simple extension to make this a slightly
less hit and miss affair but as NT doesn't support our extension you
would have to use Samba as the DMB for each workgroup. 

By now you should be thoroughly confused, so I'll move onto the main
course in the NBT suite.

\section{SMB on TCP/139}

The real action with the SMB protocol happens on TCP port 139. That is
where file and print sharing happens and other large transfers (such as
browse list synchronizations) are done.

Unlike that other major file sharing protocol for Unix systems, NFS,
SMB on TCP/139 is a complex protocol. Whereas NFS has a dozen or so
commands that can be sent over the network SMB has hundreds. In fact
there are so many possible SMB commands and sub-commands that no one
really knows the full extent of the protocol. This happens because the
protocol includes a couple of ``extension pack'' subprotocols (called
transactions) which can contain huge numbers of subcommands, most of
which are undocumented and which grow every time you stare at them
carefully. Unraveling the most important of these has been one of the
more time consuming tasks the Samba Team has had to deal with.

The other notable thing about the file sharing part of SMB is all the
PCisms that it has and the way it has been added onto with each new
development in the Windows world. The protocol shows very little in
the way of foresight, instead having things like 8.3 names mixed up
with special exception cases to the semantics for any files ending in
.exe or .sym. It is not a pretty protocol.

It does have some good features though. The idea of packet chaining,
where you can paste together two requests (such as open and read) into
one packet and get the second part to use the result of the first is
really very good as it has the potential to save on lots of
latency. It is just a pity that common implementations (ie. Windows
clients) don't take advantage of this sort of thing much and instead
use ten packets where one would suffice.

\section{SMB authentication}

Before I go off on too much of a SMB bashing expedition I should take
a detour and actually tell you something that might come in handy. The
thing that comes to mind is the difference between user and share level
security in SMB and why neither is what you probably want, despite the
fact that they are the only choices available.

When a client connects to a SMB server with the intention of accessing
some files or perhaps a printer it first goes through a negotiation
stage where the client and server agree on what SMB dialect they will
talk and what security model they will use. The security model is the
most interesting thing. There are basically two choices, ``user
level'' security and ``share level'' security. The ``modern'' choice
is user level security.

The choice (which is made by the server, not the client) determines
whether the protocol has the notion of a username for authentication
and when the client sends the password. If the server chooses share
level security then it is telling the client that access to resources
is controlled by password only and that the client need not send a
username at all if it doesn't want to. 

If the server chooses user level security then the server is saying
that the client must first login to the server with a valid
username/password before the server will allow access to any resources
at all. Once the login (called a session setup) is done then the
client can assume that it will have access to all resources without
needing to send any further authentication information for individual
resources.

The problems start when you look at when information is made available
to the server with the two security models. With share level security
the client never sends a username at all (unless it decides that it
wants to) so interfacing with standard username/password based
security systems is a bit tricky to say the least. With user level
security the login is done {\em before} the client tells the server
what resource (or resources) it wants to access so the server needs to
send a yes/no answer without taking info account the desired
resource. 

So if you want users to authenticate themselves for their home
directories but also want some users without server accounts to be
able to get to guest resources then you are out of luck. The protocol
will fight you all the way until you finally admit defeat and setup
two virtual servers, one in share level security for guest access and
the other in user level security for home directories.

Which brings us to another bit of ranting and raving. Early on the
Samba Team realized that this guest/user type of access was an
important thing to have working so we made it easy to setup virtual
servers with different security parameters and with multiplexing based
on the name of the server. Microsoft didn't have this facility so when
they started a revamp of the protocol (which is still underway) they
decided to leave out an essential piece of the protocol (the session
request where the client sends the server name to the server). Our
protests were largely ignored and it looks like this facility will be
lost. So much for the CIFS effort to collaboratively develop this
protocol.

\section{The future}

By now you will have probably guessed that I am not actually a great
fan of this protocol despite (or because of!) having spent several
years working with it as part of the Samba Team. The protocol is still
developing, however, and some of the developments aren't too bad. You
should really look at the archive of the CIFS list for details but
I'll give a rough outline here.

\begin{itemize}
  
\item no-netbios operation. netbios, the bit of the protocol that
  gives us those silly compressed names and the flat name space, is
  being scrapped. This is definitely a good thing as long as we don't
  lose any important facilities (such as virtual servers) in the
  bargain. It is being replaced with LDAP which by all accounts seems
  to be a much better solution.
  
\item kerberos authentication. The challenge response authentication
  in SMB is really not very good. Apart from gaping holes resulting
  from poor design choices and bad implementations it also is a
  one-off system that fits in only in the Microsoft world. It needs
  replacing. The CIFS solution is a kerberos based system and another
  negotiation section in SMB to handle it. Although this sounds good
  on the surface there are some downsides. First off, it sounds like
  there will be little bits of Microsoft proprietary data in the
  kerberos stream so it might be tricky to be completely compliant.
  We'll see. The real problem, however, is that it once again relies
  on a specialist back end security database. I pushed quite a bit for
  a public key based system which could interface to any existing
  security backend but without any success. Pity.

\item signed and encrypted traffic. It looks like a few years will see
  SMB being packet signed and encrypted. That could be good if it is
  done well (that would be a first for a Microsoft designed security
  system!) but could also make things tricky for non-Microsoft
  servers.

\item new port number. To support the netbios-less operation the
  protocol will move to a new port number. Microsoft pushed really
  hard for it to be a port number above 1024 which would have been a
  security nightmare but luckily sense finally prevailed and a low
  port number was chosen. Let's hope the new port number also heralds
  a new cleaner attitude to the development of this protocol.
\end{itemize}

\section{Conclusion}

This paper turned into a bit more of a rant and rave than I first
intended, but maybe that is a good thing. I hope it gives a few system
administrators some insight into how SMB works underneath and where
some of the weaknesses are.

If you want more info then go to http://samba.org/ and follow your
nose. You should be able to find lots of reading material from there.

\end{document}