With Word and Lotus sitting on Moby Fred, our reliable Netware 3.1 server, and our centrally controlled Saber Menu making it impossible for anyone to change anything, it was a sweet world. Before long Netscape replaced Mosaic and WinWeb, but nothing else much changed.

At the LAN level, Ethernet has pretty well taken over the world. There is a smattering of other topologies, but percentage-wise they add up to single digits. Down deep in the bits and bytes, Ethernet is kind of a strange animal. In fact, when you first study how it works you come away with puzzlement. How can it possibly work? It is utter chaos down there—absolute utter chaos.

The acronyms most often associated with Ethernet are "CSMA/CD." The first two-thirds of this stand for "Carrier Sense Multiple Access." What does this mean in English? Well, "carrier sense" is like listening for the dial tone on a telephone. When you pick up your phone and hear the dial tone sound, you are "sensing the carrier." Your ear listens for a dial tone. That little network interface card in your computer, the "NIC," senses the carrier. It's the same thing.

"Multiple Access" is like a party line. In a basic Ethernet system, every computer that is turned on and hooked up is listening for a dial tone on the party line all the time. That's what "Carrier Sense Multiple Access" does. Each computer is always listening for words from another computer that are destined for itself. Just like a letter in the mail, those words are wrapped in an envelope with an address written on the outside. The post office calls this a letter; Ethernet calls this a "packet."

While a letter has information printed on paper and folded up inside the envelope, a packet is more like a freight train traveling on a single track (a monorail), in this case a single wire, where the contents are strung out on this monorail one after the other, bit by bit. The address information is in the locomotive part of the train along with some other stuff identifying what kind of packet it is, how long it is, etc. The locomotive is the "header." The caboose is the "footer" whose basic job is to say, "OK, I'm done. Anything after this does not belong to this train."

Now, you have a single monorail track on your local area network. Just one. How many trains can be on that monorail at one time?

One, of course. This is real life. It's very physical. Only one train (packet) at one time. The interesting thing about these packets (let's leave the train alone for awhile) is that they are seen by every single computer on the LAN. It's as if the postman were delivering the same letter to every household on the block and depending on the homeowner to take only the letters belonging to him while rejecting the letters belonging to others. The only packets accepted by your computer are the ones with your computer's address on them; all others are seen, but ignored.
 

Like a Speeding Train
What happens if you want to send a packet out onto the Ethernet wire? Your computer just goes ahead and does it whenever it wants. It shoves a packet onto the wire without even waiting for the green signal light. Most of the time that is successful. Ethernet speeds begin at 10 megabits, or 10 million bits per second. A modest-sized packet of a few hundred bytes is on and off the wire in nanoseconds, literally. There is room for thousands of packets on the Ethernet wire in a single second. On a slow network, your packets basically have the wire to themselves at any given time. In fact, they must have the wire to themselves. If they don't, a collision occurs.

That's what the "CD" part of CSMA/CD stands for. "CD" is "Collision Detection." Ethernet has to know when this happens. If you happen to be looking at most hubs these days when it does, you see a little yellow or orange light blink. Most of the time the light blinks green, but if it blinks orange, a collision has occurred. Ethernet is smart enough to know when this happens, and when it does happen the network's response is to wait a random length of time, measured in hundredths of a second, and then retransmit the same packet again. The second time through it usually goes fine, and you never know the difference. At these speeds it's transparent to you.

At the physical level this is all very straightforward and, once you take the jargon out of it, very easy to understand. There's no magic here at all, though it's very fast; that's true. You certainly don't want to delve into the contents of the headers and footers if you can possibly avoid it. CSMA/CD is just a quirky kind of party line transmitting data instead of voices. No big deal.
 

Hunka Hunka Burnin' Hub
The Ethernet architecture works very well when there is a small amount of traffic on the network. What is considered small? Less than 30-percent utilization. Whoa! What happened to our 10 megabits? Well, they went away. You get to use a maximum of three. That's still fast. We had 100 computers on a single LAN like this for years with no ill effect. We ran about 10-percent utilization and peaked to 30 percent occasionally, usually when I downloaded some monster file from the Internet.

Once you reach 30-percent utilization, collisions happen more and more often. A collision means retransmitting the packet. When you do this a lot, performance suffers and the network begins to slow down. You could solve the issue by changing the type of system you have. For example, Token Ring uses a much different approach. It comes in 4-megabit and 16-megabit flavors, but the key difference here is that you can't send a packet unless you have the token. It's like the stories of Native American tribal councils where you couldn't talk unless you had the "talking stick." When you stop and think about it, it's an orderly way to organize a meeting. Likewise, with Token Ring you can utilize your bandwidth much more effectively.

The problem with that approach is that you have to change out all your network interface cards in all your computers, throw away your Ethernet hubs and get Token Ring hubs, and bring your entire system down when you do this. As we all know, a single malfunctioning computer is not good; a single malfunctioning network is not allowed.
 

Hubba Hubba Hubba
We saw this issue heating up a couple of years ago by virtue of using a cute little product called "Lanalyzer," by Novell. I'm not going to give you addresses and all that because it is no longer sold, but it's relevant to the story. You'll see why below. Anyway, Lanalyzer ran on a Windows 3.11 computer that I had at my desk. It had three yellow dials that looked like speedometers where utilization percentages and a couple of other factors were shown as they happened (in real time). You could see the number of packets sent and received by each computer on the network, capture those packets, and look at them (because each computer sees 100 percent of all the packets), and generate alarms if traffic exceeded a certain threshold, say 30 percent.

With Lanalyzer we saw our LAN traffic get busier and busier. There were more and more 30-percent peaks, probably because the Big-I Internet was weaseling its way more and more into our lives. Graphical stuff was taking over from text-based stuff. We had to do something. The cheapest quick fix we could muster was to install a single "switched hub" in place of the "Mother of All Hubs," which was at the center of our star pattern. Mother Hub fed all other hubs from a central location.

A switched hub divides your network into segments so that traffic from the other segments is not allowed through to your segment. Each port of the switched hub is effectively its own Ethernet segment, with nothing getting through unless it is destined for a computer on your segment. Each segment's percentage of utilization went down instantly to the 1- to 2-percent range. Collisions were nonexistent. Our network was saved, for about $2,000.

Unfortunately, Lanalyzer, the little guy that had warned us of the impending bandwidth catastrophe, wouldn't work anymore. You can guess why. It was measuring traffic only on my segment, no one else's, so now I could see traffic to technical services, but not to administration or, most importantly, to the computer room where all our outside lines converged. Besides, it wouldn't work on Windows 9X either, so Lanalyzer effectively worked its way out of a job.

The other nice thing about these switched hubs was that they had a couple of 100-megabit ports on them as well, so we stuck another hub just like it in the computer room, thereby exponentially increasing our bandwidth to most of the servers and the Internet gateway. We managed to very cheaply extricate ourselves from what amounted to a huge capacity issue, but in so doing we went blind because we couldn't see our network anymore. Recently we found some new tools that have rectified the situation, which is what I want to talk about next.
 

Regulating Traffic Bursts
One suite of programs that can replace Lanalyzer is from SolarWinds (http://www.solarwinds.net; $995). The new Engineer's Edition (v. 3.2) includes a Network Performance Monitor, and it circumvents the segmentation problem by allowing you to measure traffic at a router port. Since the router is usually pretty close to all your servers, if you can see the router you can see the majority of traffic on your LAN. In our case the main router has interface cards for every single branch, therefore not only can we measure traffic on the LAN, but we can also get to traffic at every branch. Figure 1 shows traffic on the Qwest port, which is our Internet provider, over several days. You can see that traffic jumps up to about 1.25 Mbps (million bits per second) during midday. Since this port is hooked to a T-1 line that has a maximum speed of 1.544 Mbps, you can see that we're pretty close to capacity on that line.

Figure 1: Our Network Performance Monitor shows daily  traffic bursts.

This is a frame relay T-1. Without getting into too many specifics, frame has a "Committed Information Rate" (CIR) which is guaranteed by the phone company, and a maximum line speed, which is the capacity of the T-1 (1.544 mbps). Our CIR is 768 Kbps (kilobits—not megabits; so thousands instead of millions), about half the maximum line speed. Anything above 768 (about .75 on the graph) is "burst traffic." That traffic is not guaranteed and it can be lost on the line. If this happens, the packets are re-sent.

Now you have all the information to come to a conclusion. That conclusion is: We're probably in trouble. Every day we regularly burst far above our guaranteed rate. When everyone starts pounding on those PCs in the afternoon to get out onto the Big-I Internet, we go to near-maximum capacity. We're not only well above our CIR, we're almost to the maximum capable line speed. Increasing our CIR isn't the real solution here, since bursting is allowed anyway. The real solution is getting another T-1 line.

We've decided to go with AT&T for our second T-1 line, albeit this time a straight T-1 where we enjoy all the capacity of the line without the need to "burst." This will allow some redundancy to our Internet service. If both Qwest and AT&T ever go down, it will very likely be due to a problem that will affect the entire city, not just the library.

This Network Performance Monitor is but one of many different SolarWinds tools that you get for the thousand-dollar price that I mentioned. There are several dozen tools included (see Figure 2) that measure bandwidth, discover just what is on your network (with 300 devices it can get a little complicated), measure the CPU load of your routers, and do all sorts of network management tasks. This Engineer's Edition is the most expensive of four editions, but it is the only one that offers this Performance Monitor as part of the suite. At this stage I couldn't imagine being without a monitor of this sort. You can pay a lot more for more sophisticated monitors. We've seen them available in hardware form for many tens of thousands of dollars. From that perspective, $1,000 is cheap insurance.
 

MIB Browser Network Sonar MAC Address Discovery IP Network Browser Router CPU Load Proxy Ping Subnet List MIB Walk Adv. Subnet Calculator Ping Sweep IP Address Management Bandwidth Monitor

Network Performance Monitor DNS Audit SNMP Sweep DHCP Scope Monitor TCP Reset Watch It! DNS/Who Is Resolver Ping Enhanced Ping TraceRoute Network Monitor Syslog Server

Adv. Bandwidth Monitor SNMP Brute Force Attack SNMP Dictionary Attack Password Decryption Dictionary Editor Config Downloader Config Uploader Config Viewer Config Compare Update System MIBs TFTP Server WAN Killer

Michael Schuyler is the systems librarian for the Kitsap Regional Library System in Bremerton, Washington. His e-mail address is michael@krl.org.