Re: ath(4) failing with ath_reset for AR9285

[Adrian Chadd <adrian@xxxxxxxxxxx>]

Hi!

Thanks for replying Matt.

On 5 March 2012 15:30, Matthew Dillon <dillon@apollo.backplane.com> wrote:

>    I'd say we're definitely interested, but I don't know how easy it would
>    be to maintain a common infrastructure.  I have to say honestly that it
>    might not be worth maintaining a common infrastructure for the driver.
>    DragonFly's network subsystem differs hugely from FreeBSD's and the
>    wireless infrastructure also has some major differences on top of that.

>    We're working with Rui's original port of the 802.11 infrastructure
>    to DragonFly, plus various hacks we added afterwords to stabilize it.

I'd like to try and sort out what those differences are/were and push
them upstream into FreeBSD.

>    The major differences relative to his port are related to the fact
>    that the WLAN pseudo-interfaces and the ATH (and other) physical
>    interfaces used different locks, but were able to call each other
>    recursively via the 802.11 framework.  This led to the drivers doing
>    crazy unlock/relock sequences and STILL hitting deadlocks in some cases.
>    But even with the unlock/relock sequences that left the driver open
>    to reentrancy issues which we were constantly hitting.

Right. That stuff is a bit on the terrible side. I'm trying to bring
some sanity to it but it's not going to happen soon.
I've worked around it for now in ath(4) by enforcing exclusive states
(ie, can't reset and TX/RX in taskqueues at the same time) but I'd
rather have net80211 sensibly support this behaviour.

>    So in our network infrastructure I basically gave up on the 802_11
>    psuedo-vs-phy locking issue and I have a single global serializer
>    that covers the *entire* wireless infrastructure on the system...
>    all pseudo-interfaces and all physical interfaces use the same
>    serializer.

I've thought about re-global locking things. The trouble I have with
that is it will interfere with the 802.11ac work. Well, it will
interfere with 802.11n work on multi-CPU, low-speed devices (eg the
ARM/MIPS stuff that's out there now) where we may need to run TX/RX in
parallel.

In terms of the configuration path - I'd like to see that serialised.
Ie, all the reset, channel change, protocol handling stuff - that
should all be serialised in a sensible fashion. But I'd like to have
TX/RX occur on the side. This does add locking complexity as protocol
updates do happen in the TX/RX path.

>    --
>
>    In our normal network subsystem there are also major differences.
>    We have an ifnet serializer that is installed on a per-driver basis
>    (but as per above, for the wireless physical interfaces we just specify
>    a single global serializer), but the system infrastructure is
>    responsible for acquiring the serializer (our version of the driver
>    locks) rather than the drivers having to code them up for each entry
>    point, which greatly simplifies driver design).  Even the fan-in/fan-out
>    locking for multiple rx/tx queues is handled by the higher level
>    system for the most part, for things like IF_EMX.

Right. Again, that should be stuff we can abstract out, no?

>    We also have to always change all the M_NOWAIT allocations in FreeBSD
>    drivers that appear to be used for everything, including small one-time
>    critical driver structures, to M_INTWAIT in DragonFly.  I guess M_NOWAIT
>    in FreeBSD doesn't actually ever return NULL, but in DragonFly it can
>    return NULL in virtually any situation (such as if a lock cannot be
>    acquired non-blocking).
>
>    Plus some (mostly) minor changes to numerous other things... differences
>    in how BUSDMA works, for example.  Other issues such as DragonFly not
>    overloading the stdio namespace so kernel printfs in dragonfly are
>    'kprintf()' not 'printf()', and allocations are 'kmalloc()' rather
>    than 'malloc', and so forth.

I'm very tempted to just put back the malloc/printf/etc portability
stuff that Sam removed from FreeBSD's net80211. I'd like to see this
portable to multiple operating systems rather than directly fondling
some FreeBSD stuff.

We can talk more about this at bsdcan if you/I are going.

FWIW - 802.11ac is > 1GBit/sec PHY rate wireless and people are
already thinking of embedded devices with multiple 11ac NICs in them.
Scary stuff. :)

Adrian