Macintosh Networking Notes v0.9.1 - 2002/07/10 Rich Lafferty 1. Introduction and scope This document discusses networking on the Macintosh as it applies to SME Server. It assumes familiarity with IP and with the user interface of "classic" Mac OS -- finder, chooser, etc. Much of this is from my recollection and needs to be fact-checked against Apple's developer documentation. 2. Protocols a. Network Layer At the network layer, Mac OS speaks two protocols, IP and AppleTalk. AppleTalk is proprietary to Apple, but the specifications are accessible, unlike Microsoft's which often require reverse-engineering. i. AppleTalk AppleTalk is the Macintosh-native protocol, which dates from the early days of the Macintosh. It was originally implemented as LocalTalk, which was AppleTalk over serial connectors (and later "PhoneNet", which was serial over daisy-chained 4-wire telephone cables). The AppleTalk implemented by netatalk is EtherTalk -- AppleTalk on Ethernet. (Others include TokenTalk, DECTalk, and NovellTalk.) Unless one is discussing something specific to one data layer, one usually just calls it AppleTalk. AppleTalk routing is covered in the section below entitled "ZIP". ii. IP IP has seen a few generations on the Mac. In System 7, it was implemented in the inflexible MacTCP extension (CIDR? What's that?), and in later 7.x and in Mac OS 8 and 9 as part of OpenTransport, Apple's generic network API. Systems capable of running OpenTransport should almost always do so. IP in Mac OS routes normally. b. Session and Application Layer i. ZIP ZIP is the Zone Information Protocol, by which Mac OS obtains information on available AppleTalk zones. By its original specification, AppleTalk is not routable. Specifically, it is its own protocol on the wire, *not* encapsulated in IP. It was originally designed for tiny networks (2-6 hosts), and therefore relies heavily on broadcast traffic; and it was originally designed for noisy links, and therefore relies heavily on *repeated* broadcast traffic. With the introduction of non-serial AppleTalk, it was quickly noticed that a reasonably large number of AppleTalk nodes can saturate a network with their broadcast traffic -- and can quickly make the flat namespace unusable by mere humans. In order to make AppleTalk manageable for network administrators and users, zones were introduced. Zones roughly correspond to subnets (and are often implemented parallel to them). All of the nodes within a zone must be able to reach each other and their router via broadcast. The router maintains tables of all of the nodes in their zone and communicates them to other routers on the network. Node-to-router traffic is still broadcast, but router-to-router traffic is unicast. Zones implement one level of hierarchy, turning the flat namespace of a zoneless ("non-extended") network into a two-level hierarchy ("extended"). The canonical host ZIP message is zip_getnetinfo, which is a broadcast message send during boot asking any AppleTalk routers on the local segment to report their presence. Since the protocol is connectionless and broadcast, and since there is no guarantee that a router exists, the OS must wait for a response (usually 10 seconds after the third of three broadcasts), and will hang during boot while waiting. Since it continues booting upon receiving a response, Macintoshes will seem to boot much slower on non-extended networks than on extended ones. ZIP is also used to establish network and host addresses. On a routerless network, a host claims a name and a randomly-selected address on boot by broadcasting a request to use it; if no other systems reply to its (repeated) broadcasts, it considers the name and address available and uses them. At the same time, it will broadcast a request asking for the local network number; if another node is running, it will reply and the booting node will use that network number. If it gets no reply, it will randomly choose a network number of 65280 or above. (Address can be statically allocated, but this is rare.) Conceptually, ZIP can be thought of as resembling RIP and DHCP. ii. NBP NBP is the Name Binding Protocol, which is used to register hosts and services with the local AppleTalk router and to request and respond to name information from other hosts. Nodes send broadcast messages registering their presence and the services they offer when they start up. A typical set of registered services for a system running netatalk might be (in format "name:service network.host:port"): smeserver:netatalk 65280.100:4 smeserver:Workstation 65280.100:4 smeserver:AFPServer 65280.100:254 smeserver:LaserWriter 65280.100:253 These broadcast messages are not cached by other nodes; on an extended network, they would be cached by the router. NBP is also used to find out what services are being made available by other nodes. A typical NBP request and response might go as follows: Mac1: lookup *@*:AFPServer Mac2: tangerine:AFPServer 65280.140:540 Here, Mac1 has requested all nodes ("*") in the local zone ("@*") that provide AFPServer (":AFPServer") to report, and Mac2 has informed Mac1 that its name is "tangerine", its address is 140 on network 65280, and it has an AFPServer on AppleTalk port 540. If another node registers a service, its initial registration message would function identically to a response to Mac1's lookup. (The Chooser on a busy AppleTalk network often shows a changing list of available nodes as nodes register and unregister services.) AppleTalk ports are arbitrary; there is no equivalent to /etc/services (although patterns do tend to emerge). iii. AFP AFP is the Apple Filing Protocol, better known as "AppleShare" (although technically AFP is the protocol that the AppleShare feature uses). AFP was originally designed for peer-to-peer file sharing between desktop systems, although it is now often encountered with a central server. There are two parallel implementations of AFP -- one over AppleTalk and the other over TCP in IP. Apple is probably in the process of phasing out AFP over AppleTalk; the transport supported by various versions of Mac OS is: Version Client Server ----------- ------------- ------------ System 7 AppleTalk Appletalk Mac OS 8 Appletalk, IP Appletalk Mac OS 8 with ASIP* Appletalk, IP Appletalk, IP Mac OS 9 Appletalk, IP Appletalk, IP Mac OS X IP, Appletalk** IP, AppleTalk** * ASIP (AppleShare IP) is an expensive "server" package for Mac OS. ** AppleTalk is disabled by default in OS X, which prefers users to use AFP/TCP or SMB. AFP is unicast but connectionless in its AppleTalk implementation and unicast and connectionful (TCP) in its IP implementation. An AFP connection has three stages from the user's point of view: authentication, resource selection, and use. In the authentication stage, the client Mac prompts its user for a username and password (or, in OS 9, obtains a password from the Keychain), and conveys its credentials to the server using one of a number of User Authentication Methods (UAMs): UAM OS Version Description --------- ---------- -------------------------- Plaintext All* Password is transmitted from client to server in the clear. (Like telnet.) Randnum, 7 8 9 X Password is stored in the clear on Rand2num the server; server issues a mathematical challenge to the client, which uses the user-supplied password as a key. (Roughly like CHAP.) DHX 7** 8 9 X Diffie-Hellman exchange; password is transmitted from client to server in cipher. (Roughly like SSH.) * The Plaintext UAM can be disabled on the client in recent versions of Mac OS; all versions warn the user in the login widget that their password will be passed in plaintext. ** A DHX UAM is available from Apple for System 7.6 and up, but is not distributed with the OS. AppleShare and AppleTalk are not interchangeable terms. iv. PAP PAP is the Printer Access Protocol, Apple's network printing protocol. PAP is implemented only over Appletalk. Printing over PAP is almost always PostScript (to a LaserWriter resource, even if the printer is of another brand); if printing to another type of printer or to a PostScript printer with nonstandard options, a printer description file (.ppd) must be obtained and installed on the client Macintosh. 3. Linux i. netatalk Of a handful of Unix AppleTalk implementations, netatalk has emerged as the single standard solution. The only other open-source competitor was CAP, which like early versions of netatalk only supported AFP over AppleTalk. Only with Adrian Sun's "+asun" branch of UMich's netatalk did AFP/TCP become possible. Recently, asun has decided to stop maintaining netatalk, and development has been taken over by a of developers. The protocols discussed above are implemented in netatalk as follows: AppleTalk - kernel, NIC driver, atalkd IP - kernel, NIC driver ZIP - atalkd NBP - nbprgstr, nbpunrgstr, nbplkup, atalkd (for AppleTalk) AFP - afpd, atalkd (for AppleTalk) PAP - papd, atalkd (for AppleTalk) Note that the NBP functions are handled by executables; while some applications call library calls (e.g. afpd and papd), atalk itself relies on calls to nbprgstr/nbpunrgstr in its init script to register the netatalk and Workstation services. iv. AppleDouble Apple files are composed of two forks, a data fork and a resource fork. The data fork contains what we are accustomed to a file containing: raw data, a compiled executable, etc. The resource fork contains Macintosh-specific resources such as icons, sounds, widgets, fonts, and parameters. A file without its resource fork will often be unusable and unrecoverable, although conventional inter-platform data formats like TIFF will often have a disposable data fork. In Mac OS, the resource fork and data fork are both first-class data structures in its HFS and HFS+ filesystems. When copying Macintosh files to non-HFS systems (via FTP, for instance, or as an email attachment) it is necessary to flatten the resource and data forks together into just a data fork, using the AppleDouble or BinHex algorithms. When storing Macintosh files in the ext2 filesystem, netatalk has to handle both forks. It does so by storing the data fork as the file itself (which lets non-Macintosh software read it) while storing the resource fork in the .AppleDouble directory below the same directory as the file, with the same name; the resource fork for ~richl/home/test.psd will be in ~richl/home/.AppleDouble/test.psd. Aliases (like symbolic links, or better like Windows shortcuts) are implemented with empty data forks and meaningful resource forks, so they cannot be navigated without a Mac. v. Other special files ~/.AppleDesktop is the Desktop Folder for the volume; the Macintosh desktop is comprised of desktop folders for each mounted volume. ~/.NetworkTrashFolder is the Network Trash Folder, which allows files to be put into the Trash (like the Windows Recycling Bin) and kept there when the shared volume is disconnected. (Some versions of Mac OS will force the user to empty the trash when disconnecting a shared volume.) ~/^MIcon (yes, control-M!) is present if a user chose to replace the icon used for the shared volume on their desktop. These can safely be removed if corrupt. (Other file icons are stored in the resource fork of the file itself.) 4. Example scenarios The following are rough descriptions of what happens in the Macintosh-networking world in certain scenarios. They involve handwaving where necessary. a. SME Server boot - /etc/init.d/atalk runs - starts atalkd, the appletalk superserver - broadcasts asking for routers, waits for responses - If it finds one, records its AppleTalk network number and chooses an address in that network - If it doesn't, either uses the AppleTalk network number already established for the local network, or randomly chooses one from the non-extended network range - Checks that no other hosts are using that address; if one is, chooses another and repeats - nbprgstrs: hostname:netatalk net.addr:4 hostname:Workstation net.addr:4 - listens for NBP and ZIP requests - starts afpd if required - registers hostname:AFPServer net.addr:port - listens via atalkd for AppleTalk AFP connections - listens via its own socket for TCP AFP connections - starts papd if required - registers printername:LaserWriter net.addr:port for each defined printer - listens via atalkd for AppleTalk PAP connections b. AFP connection via Chooser - (Client Macintosh learned that it was on a non-extended network at boot; this would be slightly different on an extended network.) - User on Macintosh opens Chooser, selects AppleShare - Chooser broadcasts nbplkup for *@*:AFPServer (any host in the local zone offering AFPServer) - All hosts with that service broadcast response, and client Mac adds those hosts to list in chooser - While waiting for user to select a host, chooser continues listening for nbprgstrs and nbpunrgstrs of AFPServer services, updating list - User selects a host, clicks OK - Client Mac connects to advertised port on address corresponding to selected host (cached from earlier nbplkup) - Asks selected host to use IP - If server replies with IP address to connect to (on 548/tcp), establishes connection and uses that channel for rest of communication -- otherwise continues using AppleTalk - Server chooses UAM and presents authentication challenge - Client presents authentication dialog to user, with warning if plaintext - Client sends auth response to server - If allowed, server sends list of available volumes to client - User selects a volume, clicks OK - Volume mounts on desktop -- is now accessible. If the user tries to connect to that server again, the auth information will be cached, and he will be presented with a dialog noting which user he has already authenticated as instead of the authentication dialog. c. AFP connection via URL - (OS 8, 9): User opens Chooser, clicks "Server IP address", types address or hostname of server - (OS X): User opens "Connect to Server" in Finder, types AFP url (eg. afp://www.example.com/) - Client makes direct connection, no AppleTalk involved, then per description in (ii). 5. Troubleshooting Notes a. Pinging OpenTransport Macs: One anomaly of OpenTransport IP is that Mac OS boots with no interfaces up, and will only bring an interface up when outgoing traffic needs it. This means that a freshly-booted Mac might not be pingable. This "feature" can be disabled in the TCP/IP control panel, but requires enabling Advanced mode: in the Edit menu of the control panel, choose "User Mode...", and select Advanced. Then press the newly- revealed Options button and uncheck "Load only when needed". b. Interfaces: Both OpenTransport and Appletalk must be configured to use Ethernet; one doesn't inherit the setting from the other. Do so in the TCP/IP and AppleTalk control panels. c. In chooser, but no service: Remember that the netatalk init scripts register resources separate from the daemon which provides the resources. For instance, if nbprgstr succeeds but afpd startup fails, then the host will claim to have an AFPServer resource but will have no AFP daemon running. d. Many Macintosh applications -- particularly in the publishing industry, which tends to work with large files -- assume that they will only talk to Macintosh AFP servers, and make assumptions that break when speaking with netatalk (or NT, or anything else that isn't Mac OS). Quark Xpress and FrameMaker are particularly well known for their misbehavior. Application vendors tend to ignore bug reports about the failed optimizations; the usual user strategy is to copy files locally before opening them. e. Not all network cards will speak AppleTalk correctly! Unpredictable, irregular errors that quietly go away when doing TCP-only AppleShare can indicate a network card which does not play nice with AppleTalk. Specifically, TLAN-based cards are known to encounter difficulties. 6. Further Reading http://www.cs.mu.oz.au/appletalk/aboutatalk.html ObAustralianLink: AppleTalk 101, a quick overview of the AppleTalk protocol suite, with a good paragraph on routing http://developer.apple.com/techpubs/mac/NetworkingOT/NetworkingWOT-58.html "Appletalk" from "Networking with Open Transport", part of Apple's "Inside Macintosh" series. http://www.protocols.com/pbook/appletalk.html Far more than you ever wanted to know about AppleTalk, including all of the intermediate protocols omitted from this document. http://developer.apple.com/macos/opentransport/docs/dev/Inside_Appletalk.pdf Apple's official developer's documentation for AppleTalk (600pp).