Been a while…Simple network problems…

It has been some time since I last posted on this blog…

I decided to bring to live and to post on regular basis, I’ll try to cover a few topics from my day to day work…

Let’s start here and now…

Across my daily work, most of the problems come from the most overlooked things, like ARP, weired problem, weired behaviour, that you can’t explain…

For example, you have an online machine, and all of a sudden you loose connectivity and you can no longer login to it again even though you’re sure about the username/password you’re using, this can be a show stopper for some people, and cause them to look around and try to troubleshoot for hours and even for days…

For me, I check the simplest things 1st…

  1. Network connectivity (mii-tool, ethtool)
  2. IP configuration (simply do an ifconfig to double check the configuration)
  3. IP conflicts

Network Connectivity

if you can still access the machine, let’s say through a local IP address with a seconday NIC, you might want to check the cable status, running mii-tool can give you a quick look, you should be seeing something like this…

[root@server3 ~]# mii-tool
eth0: negotiated 100baseTx-FD flow-control, link ok
eth1: no link

As you can see here, eth1 has no connection at all…

Also sometimes you may be experiencing some super slow transfers even between adjacent serevrs in the same subnet, you might be having a link in half duplex mode or even connected 10Mbit/sec speed, mii-tool will show you this too…

You can also force the link to a certain link speed, check this out…

[root@server3 ~]# mii-tool –help
usage: mii-tool [-VvRrwl] [-A media,… | -F media] [interface …]
       -V, –version               display version information
       -v, –verbose               more verbose output
       -R, –reset                 reset MII to poweron state
       -r, –restart               restart autonegotiation
       -w, –watch                 monitor for link status changes
       -l, –log                   with -w, write events to syslog
       -A, –advertise=media,…   advertise only specified media
       -F, –force=media           force specified media technology
media: 100baseT4, 100baseTx-FD, 100baseTx-HD, 10baseT-FD, 10baseT-HD,
       (to advertise both HD and FD) 100baseTx, 10baseT

–help
usage: mii-tool [-VvRrwl] [-A media,… | -F media] [interface …]
       -V, –version               display version information
       -v, –verbose               more verbose output
       -R, –reset                 reset MII to poweron state
       -r, –restart               restart autonegotiation
       -w, –watch                 monitor for link status changes
       -l, –log                   with -w, write events to syslog
       -A, –advertise=media,…   advertise only specified media
       -F, –force=media           force specified media technology
media: 100baseT4, 100baseTx-FD, 100baseTx-HD, 10baseT-FD, 10baseT-HD,
       (to advertise both HD and FD) 100baseTx, 10baseT

You can run something like this to set a connection to 10Mbit/sec half duplex (why on earth would any one want to do this?)

[root@server3 ~]# mii-tool eth0 –force=10baseT-HD

trying this on a live server can bring it down easily due to link saturation, and also not all NIC drivers support this tool, in this can ethtool to the rescue 🙂

IP Conflicts

Talking about ARP problems, this can totally lock you off your server/machine…

let’s have these scenarios

  1. You can still access the machine via a secondary/internal IP/KVM
  2. You have another machine available in the same subnet

Let’s take scenario 1, you can still access the machine by any mean, as long as you get a shell…

Assuming that your machine have the IP 192.168.52.129 (just for demonistration)

[root@localhost ~]# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:0C:29:6A:52:22 
          inet addr:192.168.52.129  Bcast:192.168.52.255  Mask:255.255.255.0

Running this command will show if there’s any other machines on the network having the same IP assigned to it…

arping 192.168.52.129 -I eth0 –c 10

If none you should see something similar to this

ARPING 192.168.52.129 from 192.168.52.129 eth0
Sent 10 probes (10 broadcast(s))
Received 0 response(s)

If there’s at least one machine having this IP assigned to it, you’d see something similar to this:

[root@localhost ~]# arping 192.168.52.129 -I eth0 -c 5
ARPING 192.168.52.129 from 192.168.52.129 eth0
Broadcast reply from 192.168.52.129 [00:0C:29:5B:85:83]  1.225ms
Unicast reply from 192.168.52.129 [00:0C:29:5B:85:83]  1.325ms
Unicast reply from 192.168.52.129 [00:0C:29:5B:85:83]  1.118ms
Unicast reply from 192.168.52.129 [00:0C:29:5B:85:83]  2.187ms
Unicast reply from 192.168.52.129 [00:0C:29:5B:85:83]  2.158ms
Sent 5 probes (1 broadcast(s))
Received 5 response(s) (1 broadcast(s))

This loudly indicates that there’s an IP conflict and shows the MAC address of the conflicting machine…

Let’s look as scenario 2:

You have another machine on the same subnet,

Run the a command similar to this from its console/shell

[root@localhost ~]# arping 192.168.52.128 -I eth0 -c 5
ARPING 192.168.52.128 from 192.168.52.129 eth0
Broadcast reply from 192.168.52.128 [00:0C:29:5B:85:83]  1.225ms
Unicast reply from 192.168.52.128 [00:0C:29:5B:85:83]  1.325ms
Unicast reply from 192.168.52.128 [00:0C:29:5B:85:83]  1.118ms
Unicast reply from 192.168.52.128 [00:0C:29:5B:85:83]  2.187ms
Unicast reply from 192.168.52.128 [00:0C:29:5B:85:83]  2.158ms
Sent 5 probes (1 broadcast(s))
Received 5 response(s) (1 broadcast(s))

The example here shows no IP conflicts since there’s only one MAC address replying to the ARPING request…

If there’s a conflict you’d see something similar to this…

ARPING 192.168.52.128 from 192.168.52.1 eth0
Unicast reply from 192.168.52.128 [00:16:3E:0F:61:3A]  0.798ms
Unicast reply from 192.168.52.128 [00:16:3E:21:3E:20]  0.865ms
Sent 1 probes (1 broadcast(s))
Received 2 response(s)

Here you can see that two machines replied with two MACs for a single ARP request, this can show an IP conflict…

However, the last may not be a problem if the machine has two interfaces connected to the same VLAN, the machine (esp. Linux OS) would reply to the ARP request from both interfaces resulting in duplicate ARP replies from two different MACs…

However, most servers that come with two NICs onboard will have sequential MAC address, take this example (I’ve masked the IP because this is a real server):

ARPING xx.yy.zz.10 from xx.yy.zz.53 eth0
Unicast reply from xx.yy.zz.10 [00:1E:4F:1F:9D:B1]  0.792ms
Unicast reply from xx.yy.zz.10 [00:1E:4F:1F:9D:B3]  0.934ms
Sent 1 probes (1 broadcast(s))
Received 2 response(s)

This indicates no problem at all, note the MAC address in the replies are almost identical except for the last HEX digit, this shouldn’t be a problem, UNLESS, you have some IPTABLES rules that are limiting based on incoming/outgoing interface…

That’s all for now…

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: