Re: Severe packet loss on fxp interfaces with the new bridging code...

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:On an SMP machine without any of the new mpsafe sysctls active, bridging
:over fxp interfaces barely works.  Maximum bandwidth is somewhere around
:100K/s, and I am seeing up to 80% packet loss.  (as measured with a
:simultaneous scp and ping)  After a bit of detective work, I have found
:the location of these mysteriously vanishing packets, but I do not
:understand the true nature of the problem.
:
:At if_fxp.c, line 1335, fxp_add_rfabuf() often fails to allocate a
:replacement mbuf for use in the receive ring.  It then silently proceeds
:to reuse the packet.   Any ideas on why the
:
:	m_getcl(MB_DONTWAIT, MT_DATA, M_PKTHDR)
:
:would fail here?  In any case, it might be nice to add an
:"else ifp->if_ierrors++;" clause to the relevant if block...

    It can only fail here if the objcache gets exhausted, which shouldn't
    happen for normal bridging so either there aren't enough mbufclusters
    specified via kern.ipc.nmbclusters or there is a bug in the objcache.

    The easiest way for me to see what is going on is if you intentionally
    panic your machine and get kernel core via DDB while it is messing up
    during your test.  e.g. by doing ctl-alt-esc to pop into the debugger
    and then 'panic' + <return> + <return>.

:While tracking this down, I also discovered that one of the BPF_MTAP()
:calls is misplaced.  If you tcpdump on a bridge interface, you will miss
:any traffic destined for the local machine itself.  The below diff  
:corrects
:this, as well as acounts for packets sent out via the bridge interface.
:
:Chris

    I'll get your patch committed.

					-Matt
					Matthew Dillon 
					<dillon@xxxxxxxxxxxxx>