<html>
<head>
<base href="https://bugzilla.netfilter.org/" />
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_NEW "
title="NEW - Adding nftables interval sets progressively gets slower and makes the nft CLI less responsive with each added set"
href="https://bugzilla.netfilter.org/show_bug.cgi?id=1735">1735</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>Adding nftables interval sets progressively gets slower and makes the nft CLI less responsive with each added set
</td>
</tr>
<tr>
<th>Product</th>
<td>nftables
</td>
</tr>
<tr>
<th>Version</th>
<td>1.0.x
</td>
</tr>
<tr>
<th>Hardware</th>
<td>All
</td>
</tr>
<tr>
<th>OS</th>
<td>All
</td>
</tr>
<tr>
<th>Status</th>
<td>NEW
</td>
</tr>
<tr>
<th>Severity</th>
<td>major
</td>
</tr>
<tr>
<th>Priority</th>
<td>P5
</td>
</tr>
<tr>
<th>Component</th>
<td>nft
</td>
</tr>
<tr>
<th>Assignee</th>
<td>pablo@netfilter.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>anton.khazan@gmail.com
</td>
</tr></table>
<p>
<div>
<pre>Created <span class=""><a href="attachment.cgi?id=734" name="attach_734" title="shell script demonstrating the bug">attachment 734</a> <a href="attachment.cgi?id=734&action=edit" title="shell script demonstrating the bug">[details]</a></span>
shell script demonstrating the bug
The title basically describes the issue.
Steps to reproduce the issue:
Take an ip list for any country (in most of my tests I've been using the ip
list for GB fetched from ipdeny, which currently has 8589 ipv4 ip blocks),
measure time taking to load it into a new set, measure time adding the same ip
list again in a new set under a new name (without removing the 1st one), repeat
n times. Each new iteration takes a little longer. On the OpenWRT installation
in VM, 1st iteration takes 0.12s. 10th iteration takes 0.47s. 40th iteration
takes 1.59s.
Getting a response from the command 'nft list tables' also takes progressively
longer with each added set, as well as any other nft CLI operation involving
the table under test. Starting with 0.04s after 1st iteration and up to 1.49s
after the 40th iteration.
Monitoring memory use while running this test also reveals that the transient
memory use spikes (similar to those reported in <a class="bz_bug_link
bz_status_ASSIGNED "
title="ASSIGNED - nft large sets load high memory requirements"
href="show_bug.cgi?id=1584">Bug 1584</a>) get progressively
larger as you go, reaching about 180MB around the 40th iteration (i've been
just monitoring memory use via the 'top' utility which samples once a second at
most, so I might have missed the peak of the spike).
Using the same ip list is not an essential part of the bug, as well as the
specific list for GB. I picked GB because it's a mid-sized list, and re-using
it because this way it is possible to get comparable results. The issue also
extends to ipv6 ip lists.
nft CLI operations involving other tables seem unaffected.
This is a pretty big problem for applications like geoip, especially so on
embedded devices like routers with limited memory.
I'm not sure whether this belongs to the kernel component or nft userspace.
Filing this under nft userspace component in the meanwhile.
Hardware used for testing: Intel i7-4770
Same behavior on Linux Mint (kernel 6.2.0, nftables 1.0.2) and OpenWRT x86-64
installed in a VM (kernel 5.15.137, nftables 1.0.8).
A shell script reproducing the issue is attached.</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are watching all bug changes.</li>
</ul>
</body>
</html>