Hard-coded M_* flags

[YONETANI Tomokazu <qhwt+dfly@xxxxxxxxxx>]

Hi, all.
I've been trying to update my PPPoE router/mail server in my room from
1.2.6-RELEASE to 1.4.4-RELEASE for a week now and always resulted
in a panic saying "bad mbuf flags" in m_free().  At first I thought
it was netgraph code was at fault because I'm using mpd, but the same
thing happened with ppp.  After poking here and there in the source tree,
I found that ipfw2 is abusing mbuf mh_flags by defining M_SKIP_FIREWALL
as 0x4000 in ip_fw2.c file.  Since 0x4000 in mh_flags also mean
M_EXT_CLUSTER, this produced an mbuf entry with M_EXT_CLUSTER set
without M_EXT, which led to a panic.  Argh.

Instead of dumping ipfw and switching over to pf, I decided to investigate
other instances of M_* flags in DragonFly_RELEASE_1_4 source code by
grep'ping through with the following pattern:
  $ egrep -r '(\.|->)mh?_flags *[|&]=? *[(~]?M_' /sys/

So far the problematic(IMO) flags are M_SKIP_FIREWALL, M_FEC_*,
and M_NOTIFICATION.  I tried to think about how to solve the problem, but
as m_hdr.mh_flags is 16bit-wide, there's no extra room left to fit them.
(and changing the size of struct mbuf within a release branch is not
desirable, right?)  I thought M_PROTO4 and M_PROTO5 can be used since
they are ipv6-specific and ipfw doesn't handle ipv6, but I don't know.
On FreeBSD 5 or later, M_SKIP_FIREWALL is already moved to <sys/mbuf.h>,
but M_FEC_* are left hardcoded in ng_fec.c.


[hardcoded M_ flags]
  M_SKIP_FIREWALL: defined as 0x4000, which conflicts with M_EXT_CLUSTER
    /sys/net/ipfw/ip_fw2.c

  M_FEC_MAC: defined as 0x2000, which may conflict with M_CLCACHE
  M_FEC_INET: defined as 0x4000, which may conflict with M_EXT_CLUSTER
  M_FEC_INET6: defined as 0x8000, which may conflict with M_PHCACHE
    /sys/netgraph/fec/ng_fec.c
    (I've never used ng_fec, BTW)

  M_NOTIFICATION: defined as 0x10000 in mbuf.h(doesn't fit in a short int!)
    /sys/kern/uipc_socket.c
    /sys/netinet/sctputil.c

  M_CANFASTFWD: probably harmless, unless someone defines M_CANFASTFWD
    /sys/contrib/ipfilter/netinet/fil.c

  M_HASFCS: probably harmless, unless someone defines M_HASFCS
    /sys/netproto/802_11/ieee80211_input.c

  M_LINK0: probably harmless, unless someone defines M_LINK0
    /sys/net/if_fddisubr.c

[uses of M_PROTOn flags]
  M_PROTO1: defined in mbuf.h
    /sys/dev/netif/bge/if_bge.c
    /sys/dev/netif/em/if_em.c
    /sys/dev/netif/gx/if_gx.c
    /sys/dev/netif/nge/if_nge.c
    /sys/dev/netif/re/if_re.c
    /sys/dev/netif/ti/if_ti.c
    /sys/dev/netif/txp/if_txp.c
    /sys/net/bridge/if_bridge.c
    /sys/net/if_ethersubr.c
    /sys/net/vlan/if_vlan.c

  M_AUTHIPHDR: defined as M_PROTO2 in in6.h
    /sys/netinet6/ah_input.c
    /sys/netinet6/in6.h
    /sys/netinet6/ip6_input.c
    /sys/netinet6/ipsec.c
    /sys/netinet6/nd6_rtr.c
    /sys/netproto/ipsec/ipsec.c
    /sys/netproto/ipsec/xform_ah.c
    /sys/netproto/ipsec/xform_ipip.c

  M_HIGHPRI: defined as M_PROTO2 in if_ppp.c
    /sys/net/ppp/if_ppp.c

  M_DECRYPTED: defined as M_PROTO3 in in6.h
    /sys/netinet/ip_icmp.c
    /sys/netinet6/esp_input.c
    /sys/netinet6/icmp6.c
    /sys/netinet6/in6.h
    /sys/netinet6/ipcomp_input.c
    /sys/netinet6/ipsec.c
    /sys/netproto/ipsec/xform_esp.c
    /sys/netproto/ipsec/ipsec.c
    /sys/netproto/ipsec/xform_ipip.c

  M_ERRMARK: defined as M_PROTO3 in if_ppp.c
    /sys/net/ppp/if_ppp.c

  M_LOOP: defined as M_PROTO4 in ip6.h
    /sys/net/if_loop.c		(#ifdef INET6)
    /sys/inetinet6/ip6.h
    /sys/inetinet6/icmp6.c
    /sys/inetinet6/ip6_input.c
    /sys/inetinet6/ip6_mroute.c
    /sys/inetinet6/ip6_output.c
    /sys/inetinet6/mld6.c

  M_AUTHIPDGM: defined as M_PROTO5 in in6.h
    /sys/netinet6/ah_input.c
    /sys/netinet6/esp_input.c
    /sys/netinet6/in6.h
    /sys/netinet6/ip6_input.c
    /sys/netinet6/ipsec.c
    /sys/netinet6/nd6_rtr.c
    /sys/netproto/ipsec/ipsec.c
    /sys/netproto/ipsec/xform_ah.c
    /sys/netproto/ipsec/xform_ipip.c