[Bug 1402] New: Race errors with nft
bugzilla-daemon at netfilter.org
bugzilla-daemon at netfilter.org
Thu Jan 30 15:42:20 CET 2020
https://bugzilla.netfilter.org/show_bug.cgi?id=1402
Bug ID: 1402
Summary: Race errors with nft
Product: nftables
Version: unspecified
Hardware: All
OS: Debian GNU/Linux
Status: NEW
Severity: normal
Priority: P5
Component: nft
Assignee: pablo at netfilter.org
Reporter: netfilter at morp.org
Summary
-------
To evaluate the atomicity/stability of nftables, I set up the following
experiment:
- Created two distinct sets of rules, R1 and R2 oaded via `nft -f`
- In two different screens, started two processes to constantly re-load the
rulesets with no pause between reloads
- Created a small script that would dump the currently active ruleset and save
it to a file, named after the hashed ruleset contents
The expectation is that the `nft list ruleset` output would always display
either R1 or R2 - no empty ruleset, or a cross between R1 or R2, or any other
output. Those expectations were true the vast majority of the time, but in some
rare cases I get an unexpeced output - hence the report.
Bug Summary
-------
When competing nft ruleset imports are running, and the ruleset is dumped (with
`nft list ruleset`), a couple of rare errors can be seen:
1. An error message returned by `nft list ruleset`: "free(): double free
detected in tcache 2"
2. Output is composed of duplicate (or even trippled) dumps of the same
tables/rules
Test files to reproduce
-----------------------
R1.tf
```
#!/usr/sbin/nft -f
flush ruleset
table inet filter {
set DROP-NETS-V4 {
type ipv4_addr
flags interval
elements = { 1.2.3.0/27 }
}
chain predefrag {
type filter hook prerouting priority -450; policy accept;
ip frag-off & 49151 != 0 counter packets 0 bytes 0 drop
ip6 nexthdr ipv6-frag counter packets 0 bytes 0 drop
}
chain INPUT {
type filter hook input priority 0; policy drop;
udp sport domain udp dport 1024-65535 accept
tcp sport domain tcp dport 1024-65535 accept
udp dport domain ip daddr 127.0.0.1 accept
udp sport 1024-65535 udp dport 1024-65535 ip ttl 1 counter reject
udp sport 1024-65535 udp dport 1024-65535 ip6 hoplimit 1 counter reject
ip saddr @DROP-NETS-V4 counter drop
ct state invalid drop
ct state established,related accept
iifname "lo" counter accept
meta l4proto icmp counter accept
meta l4proto ipv6-icmp counter accept
tcp dport 22 accept
}
}
```
R2.tf
```
#!/usr/sbin/nft -f
flush ruleset
add table inet my-filter
add set inet my-filter ALLOW_SSH_NET {
type ipv4_addr;
flags interval;
elements={
0.0.0.0/0
}
}
add chain inet my-filter FORWARD {type filter hook forward priority 0; policy
drop;}
add chain inet my-filter INPUT {type filter hook input priority 0; policy
drop;}
add rule inet my-filter INPUT ip protocol icmp accept
add rule inet my-filter INPUT ip saddr @ALLOW_SSH_NET accept
add rule inet my-filter INPUT ct state related,established accept
```
nftables_hash.sh:
```
#!/usr/bin/env bash
output=$(nft list ruleset 2>&1 | grep -v counter)
md5=$(md5sum <<< $output | cut -f 1 -d ' ')
$(cat <<< "$output" > "hash_${md5}.out")
echo $(date) "$md5"
```
Test commands
-------------
screen -S R1 bash -c 'while true; do ./R1.nft && echo $(date) "Reloaded R1
ruleset"; done'
screen -S R2 bash -c 'while true; do ./R2.nft && echo $(date) "Reloaded R2
ruleset"; done'
screen -S ruleset_hash bash -c 'while true; do ./nftables_hash.sh | tee
hashes_nft.txt; done'
Results
-------
After running for a few minutes, you would see more than 2 modes of 'nftables
list ruleset' outputs:
# ls -la hash_*
-rw-r--r-- 1 root root 41 Jan 30 14:30
hash_45b12278a9565f2b5d7107d1be71c93a.out
-rw-r--r-- 1 root root 966 Jan 30 14:30
hash_911e7f4ec5194058a36774026e22001b.out
-rw-r--r-- 1 root root 1035 Jan 30 14:30
hash_c4d068fa75ff27bccc495a3e3a550018.out
-rw-r--r-- 1 root root 363 Jan 30 14:30
hash_ce58eea5fcc69506faccd30dc0b68868.out
-rw-r--r-- 1 root root 483 Jan 30 14:30
hash_d33d213b4cfcc7bc056cab22b8d23196.out
The most frequently found error mode is where nft ruleset dumping:
```
# cat hash_45b12278a9565f2b5d7107d1be71c93a.out
free(): double free detected in tcache 2
```
Another error mode is where the dump contains duplicated version of the
ruleset, e.g.
```
# cat hash_911e7f4ec5194058a36774026e22001b.out
table inet filter {
set DROP-NETS-V4 {
type ipv4_addr
flags interval
elements = { 1.2.3.0/27 }
}
chain predefrag {
type filter hook prerouting priority -450; policy accept;
}
chain INPUT {
type filter hook input priority 0; policy drop;
udp sport domain udp dport 1024-65535 accept
tcp sport domain tcp dport 1024-65535 accept
udp dport domain ip daddr 127.0.0.1 accept
ct state invalid drop
ct state established,related accept
tcp dport ssh accept
}
}
table inet filter {
set DROP-NETS-V4 {
type ipv4_addr
flags interval
elements = { 1.2.3.0/27 }
}
chain predefrag {
type filter hook prerouting priority -450; policy accept;
}
chain INPUT {
type filter hook input priority 0; policy drop;
udp sport domain udp dport 1024-65535 accept
tcp sport domain tcp dport 1024-65535 accept
udp dport domain ip daddr 127.0.0.1 accept
ct state invalid drop
ct state established,related accept
tcp dport ssh accept
}
}
```
In (more) rare cases, the same table can even be displayed 3 times, rather than
just doubled.
System details:
---
OS: Debian 10.2
Kernel: 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11) x86_64
GNU/Linux
nftables version: nftables v0.9.0 (Fearless Fosdick)
Related LKMs:
# lsmod | grep -i tables
nf_tables_set 32768 18
nf_tables 143360 248
nft_ct,nft_reject_inet,nft_counter,nf_tables_set,nft_reject
nfnetlink 16384 1 nf_tables
--
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200130/b6f4f0f3/attachment.html>
More information about the netfilter-buglog
mailing list