[Bug 714] New: Kernel panics in same_src()

Thu Apr 7 14:44:03 CEST 2011

http://bugzilla.netfilter.org/show_bug.cgi?id=714

           Summary: Kernel panics in same_src()
           Product: netfilter/iptables
           Version: linux-2.6.x
          Platform: x86_64
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: NAT
        AssignedTo: netfilter-buglog at lists.netfilter.org
        ReportedBy: d.petuhov at electro-com.ru
   Estimated Hours: 0.0

Created an attachment (id=353)
 --> (http://bugzilla.netfilter.org/attachment.cgi?id=353)
Kernel config

Overview:
NATing router server. 2x Intel Xeon X7550, 2x i82599 network adapters.
Linux nat9 2.6.32.32 #1 SMP Fri Mar 25 12:32:45 MSK 2011 x86_64 GNU/Linux

.config file attached
Modules Loaded         binfmt_misc ipt_NETFLOW ts_kmp xt_length ipt_REJECT
xt_comment xt_state xt_string iptable_filter iptable_nat ip_tables x_tables
ext2 nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_proto_gre
nf_nat_sip nf_conntrack_sip nf_nat_ftp nf_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_conntrack_ftp nf_conntrack ipmi_devintf ipmi_si ipmi_msghandler loop tpm_tis
pcspkr i2c_i801 evdev joydev tpm psmouse processor button i2c_core serio_raw
tpm_bios ext3 jbd mbcache dm_mod sd_mod crc_t10dif usbhid hid ahci libata igb
scsi_mod uhci_hcd ehci_hcd ixgbe usbcore nls_base dca thermal thermal_sys

Single -j SNAT --persistent rule.

Related non-default sysctls:
net.netfilter.nf_conntrack_max = 6000000
net.netfilter.nf_conntrack_count = 809342
net.netfilter.nf_conntrack_buckets = 1048576

Under some load (currently we have ~1.5+1.0 Gig/200+200kpps -- in+out -- on
this server) kernel just panics. 

Here's some debug I've done on crash dump:
crash> bt
PID: 0      TASK: ffff88027f1e1560  CPU: 21  COMMAND: "swapper"
 #0 [ffff880010ba3520] machine_kexec at ffffffff8102a078
 #1 [ffff880010ba3580] crash_kexec at ffffffff810952d8
 #2 [ffff880010ba3650] oops_end at ffffffff813af0e0
 #3 [ffff880010ba3680] no_context at ffffffff81034132
 #4 [ffff880010ba36d0] __bad_area_nosemaphore at ffffffff810343b5
 #5 [ffff880010ba3720] bad_area_nosemaphore at ffffffff81034483
 #6 [ffff880010ba3730] do_page_fault at ffffffff813b0b24
 #7 [ffff880010ba3780] page_fault at ffffffff813ae4ef
    [exception RIP: nf_nat_setup_info+1497]
    RIP: ffffffffa023bf19  RSP: ffff880010ba3830  RFLAGS: 00010286
    RAX: 0000000000000000  RBX: ffff88027a4325c8  RCX: ffff880277fc05f8
    RDX: 0000000000000011  RSI: ffffffff816f27c0  RDI: 0000000000000011
    RBP: ffff880010ba3910   R8: 0000000000004000   R9: 0000000036551144
    R10: 0000000064c1f078  R11: ffff88027e3b8d38  R12: ffff88026b6ba950
    R13: 0000000000000000  R14: ffffc9001363b744  R15: ffff880010ba3880
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #8 [ffff880010ba3878] do_IRQ at ffffffff813b2dc5
 #9 [ffff880010ba3918] ipt_snat_target at ffffffffa02a611f
#10 [ffff880010ba3928] ipt_do_table at ffffffffa029e16a
#11 [ffff880010ba3ab8] nf_nat_rule_find at ffffffffa02a61d4
#12 [ffff880010ba3ad8] nf_nat_fn at ffffffffa02a6469
#13 [ffff880010ba3b28] nf_nat_out at ffffffffa02a6638
#14 [ffff880010ba3b48] nf_iterate at ffffffff812fd45c
#15 [ffff880010ba3b98] nf_hook_slow at ffffffff812fd514
#16 [ffff880010ba3c08] ip_output at ffffffff8130b6f3
#17 [ffff880010ba3c38] ip_forward_finish at ffffffff81306cde
#18 [ffff880010ba3c58] ip_forward at ffffffff81306ede
#19 [ffff880010ba3c98] ip_rcv_finish at ffffffff81305125
#20 [ffff880010ba3cd8] ip_rcv at ffffffff813056ad
#21 [ffff880010ba3d18] netif_receive_skb at ffffffff812d787a
#22 [ffff880010ba3d78] napi_skb_finish at ffffffff812d7c68
#23 [ffff880010ba3d98] napi_gro_receive at ffffffff812d8179
#24 [ffff880010ba3db8] ixgbe_poll at ffffffffa00be1cc
#25 [ffff880010ba3e68] net_rx_action at ffffffff812d830e
#26 [ffff880010ba3ec8] __do_softirq at ffffffff8105c726
#27 [ffff880010ba3f38] call_softirq at ffffffff8100d15c
#28 [ffff880010ba3f50] do_softirq at ffffffff8100e995
#29 [ffff880010ba3f70] irq_exit at ffffffff8105c205
#30 [ffff880010ba3f80] do_IRQ at ffffffff813b2dc5
--- <IRQ stack> ---
#31 [ffff88047f0bfe38] ret_from_intr at ffffffff8100c9d3
    [exception RIP: mwait_idle+113]
    RIP: ffffffff810140a1  RSP: ffff88047f0bfee8  RFLAGS: 00000246
    RAX: 0000000000000000  RBX: ffff88047f0bfef8  RCX: 0000000000000000
    RDX: 0000000000000000  RSI: ffff88047f0bffd8  RDI: ffffffff8161f308
    RBP: ffffffff8100c9ce   R8: 0000000000000000   R9: 0000000000000000
    R10: 0000000000000001  R11: 0000000000000000  R12: ffff88047f0bfef8
    R13: ffffffff8100c9ce  R14: ffffffff813abbf8  R15: ffff88047f0bfef8
    ORIG_RAX: ffffffffffffff41  CS: 0010  SS: 0018
#32 [ffff88047f0bff00] cpu_idle at ffffffff8100ae7d
gdb nf_nat.ko
(gdb) l* nf_nat_setup_info+1497
0xf49 is in nf_nat_setup_info (net/ipv4/netfilter/nf_nat_core.c:135).
130              const struct nf_conntrack_tuple *tuple)
131     {
132             const struct nf_conntrack_tuple *t;
133
134             t = &ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple;
135             return (t->dst.protonum == tuple->dst.protonum &&
136                     t->src.u3.ip == tuple->src.u3.ip &&
137                     t->src.u.all == tuple->src.u.all);
138     }
139

Steps to Reproduce: 
I don't really know. It happends in unpredictable periods of time (sometimes
few days, sometimes few hours).

If I can do or tell something else to help, just ask.

-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
You are watching all bug changes.