[ANNOUNCE] nftables 0.8 release

Pablo Neira Ayuso pablo at netfilter.org
Fri Oct 13 01:49:41 CEST 2017


The Netfilter project proudly presents:

        nftables 0.8

This release contains new features available up to the (upcoming)
Linux 4.14 kernel release:

* Support for stateful objects, these objects are uniquely identified by
  a user-defined name, you can refer to them from rules, and there is a
  well established interface to operate with them, eg.

     # nft add counter filter test

  This creates a quota object whose name is 'test'.

     # nft list counters
     table ip filter {
            counter test {
                     packets 0 bytes 0

  You can then refer to these objects from maps:

     # nft add table filter
     # nft add chain filter input { type filter hook input priority 0\; }
     # nft add map filter badguys { type ipv4_addr : counter \; }
     # nft add rule filter input counter name ip saddr map @badguys
     # nft add counter filter badguy1
     # nft add counter filter badguy2
     # nft add element filter badguys { : "badguy1" }
     # nft add element filter badguys { : "badguy2" }

  Implicit map definitions are supported too:

     table ip filter {
            counter http-traffic {
                    packets 8 bytes 672

            chain input {
                    type filter hook input priority 0; policy accept;
                    counter name tcp dport map { 80 : "http-traffic", 443 : "http-traffic"}

  You can atomically dump and reset these objects:

     # nft reset counter ip filter badguy1
     counter test {
            packets 1024 bytes 100000
     # nft reset counter ip filter badguy1
     counter test {
            packets 0 bytes 0

  Currently: counters, quota and limit are supported. Note: limit is
  available starting 4.14-rc.

* Sort set elements when listing them, from lower to largest, eg.

     # nft add table x
     # nft add set x y { type ipv4_addr\; }
     # nft add element x y {,,, }
     # nft list ruleset
     table ip x {
            set y {
                    type ipv4_addr
                    elements = {,,
                       , }

  When listing very large sets, nft takes almost the same time as
  before, so impact of this new feature is negligible.

* TCP option matching and mangling support. This includes TCP maximum
  segment size mangling, eg.

    # nft add rule mangle forward tcp flags syn tcp option maxseg size set rt mtu

  People that own routers with ppp interfaces, you have no excuses to
  migrate to nftables, this is your replacement for the TCPMSS target ;-)

* Add new `-s' option for listings without stateful information:

     # nft -s list ruleset
     table ip filter {
            chain output {
                    type filter hook output priority 0; policy accept;
                    tcp dport https counter
                    tcp dport https quota 25 mbytes

* Add new -c/--check option for nft, to tests if your ruleset loads fine,
  into the kernel, this is a dry run mode, eg.

     # nft -c ruleset.nft

  You can also use it in incremental rule updates scenarios:

     # nft -c add rule x y counter

* Connection tracking helper support, eg.

     table ip filter {
             ct helper ftp-standard {
                    type "ftp" protocol tcp

             chain y {
                    tcp dport ftp ct helper set "ftp-standard"

  Note for iptables users: In nftables, you have to specify what helper
  you want to enable specifically, then set it from rules, given the
  former automatic helper assignment approach is deprecated, see for
  more info: https://home.regit.org/netfilter-en/secure-use-of-helpers/

* Add --echo option, to print the handle that the kernel allocates to
  uniquely identify rules, eg.

     # nft --echo --handle add rule ip t c tcp dport {22, 80} accept
     add rule ip t c tcp dport { ssh, http } accept # handle 2

* Conntrack zone support, eg.

     table raw {
            chain pre {
                   type filter hook prerouting priority -300;
                   iif eth3 ct zone set 23
            chain out {
                   type filter hook output priority -300;
                   oif eth3 ct zone set 23

* Symmetric hash support, eg.

     # nft add rule ip nat prerouting ct mark set symhash mod 2

* Add support to include directories from nft natives scripts, files are
  loaded in alphanumerical order, eg.

     include "/foo/*.nft"

  Assuming the following content on such folder:


  "01_rules.nft" is loaded before "02_rules.nft".

* Allow to check if IPv6 extension header or TCP option exists or is
  missing, eg.

     # nft add rule ip6 x y exthdr frag exists drop
     # nft add rule inet x y tcp option window missing drop

  Same for fib lookups, eg.

     # nft add rule x y fib daddr oif exists accept

* Extend quota support to display wasted bytes, eg.

    table ip x {
            chain y {
                    type filter hook forward priority 0; policy accept;
                    quota over 200 mbytes used 1143 kbytes drop

  This is useful to restore quota between reboots.

* Add ct average matching, to match average bytes per packet a connection has
  transferred so far, to map the existing feature available in the
  iptables connbytes match.

   eg. match average pkt in both directions:
     # nft add rule x y ct avgpkt gt 100

   eg. match avgpkt in original direction:
     # nft add rule x y ct original avgpkt gt 200

* Allow to flush maps and flow tables, eg.

     # nft flush map filter map1
     # nft flush flow table filter ft-https

* Allow to embed set definition into an existing set, eg.

     # nft -f ruleset.nft
     define myset = {
     add rule ip saddr { $myset, }

  And scripting like now works too:

     define dnat_ports      = { 1234-1567 }
     define port_allow      = {
            53,             # dns
            $dnat_ports,    # dnat

* Slightly better error reporting when ruleset loaded via nft -f, eg.

     # nft -f /home/test/x
     /home/test/x:4:17-70: Error: Could not process rule: No such file or directory
                    ip saddr {,,, } jump x

  In the example above, the chain 'x' that this jump refers to does not exist.
  We can still do better and make it even more fine grain...

* Don't wraparound set elements when listing, eg.

    % nft list ruleset -nnn
    table ip t {
            set s {
                    type inet_service
                    elements = { 1, 2, 3, 4, 10,
                                 432, 433, 434, 435, 436,
                                 437, 438, 439, 440, 441,
                                 442, 443, 444, 445, 446,
                                 447, 448, 449, 450, 12345 }

  Current approach is rather simple, improvable since the number of
  elements to fit in is not calculated based of the number of terminal
  columns, but we can make it at some point.

* Allow to list sets in a table, eg.

     # nft list sets table inet filter

* Conntrack event filtering support via rule, eg.

     # nft add rule x y ct event new counter

... lots of bugfixes too:

* Fix wrong bytecode generation for IPv6 that may result in mismatching
  packets due to extensions placed before the transport header.
* Always print range expressions numerically. This has been a problem
  with network services, where character '-' is possible as part of the
* Replace getaddrinfo() by inet_service internal lookup table, we don't
  rely on /etc/services anymore, so we have a predictable input from the
  parser side.
* Do not round consumed quota bytes when listing then.
* Fix print of ip dnat address.
* Fix icmp matching in inet tables.
* Fix display of set keys with host byte endian byteorder.
* Fix nested set merge size adjustment, eg.

     # nft add rule ip saddr { {, }, }

* Fix missing seed option in hash expression in listings.
* Generate a random seed in hash expression if seed option is empty, eg.

     # nft add rule x y ct mark set jhash ip saddr mod 2

  generates a random seed internally, that is not displayed. If you want
  an explicit seed, you have to specify it.
* Allow to match ICMPv6 packets from IPv4, this is a firewall software so
  we should allow to match even things that don't make sense.
* Fix printing of range elements in named sets in monitor mode.
* Fix printing of optional attributes 'flags', 'gc-interval' and
  'timeout' in set declarations.
* Fix parsing of IPv4-Mapped IPv6 addresses.
* Fix lots of memory leaks.

... and lots of documentation updates! and regression tests!


The nftables code can be obtained from:

* http://netfilter.org/projects/nftables/downloads.html
* ftp://ftp.netfilter.org/pub/nftables
* git://git.netfilter.org/nftables

To build the code, libnftnl 1.0.8 and libmnl >= 1.0.2 are required:

* http://netfilter.org/projects/libnftnl/index.html
* http://netfilter.org/projects/libmnl/index.html

Visit our wikipage for user documentation at:

* http://wiki.nftables.org

For the manpage reference, check man(8) nft.

In case of bugs and feature request, file them via:

* https://bugzilla.netfilter.org

Make sure you create no duplicates already, thanks!

Happy firewalling!
-------------- next part --------------
Anatole Denis (2):
      scanner: fix search_in_include_path test
      erec: Fix input descriptors for included files

Anders K. Pedersen (1):
      netlink_linearize: skip set element expression in set statement key

Arturo Borrero Gonzalez (5):
      payload: explicit network ctx assignment for icmp/icmp6 in special families
      expression: print sets and maps in pretty format
      evaluate: avoid reference to multiple src data in statements which set values
      monitor: add debug messages
      monitor: Fix printing of range elements in named sets

Bertrand Bonnefoy-Claudet (1):
      doc: Fix typo in manpage

Daniel Kahn Gillmor (1):
      man: use https for wiki link

Duncan Roe (3):
      datatype: printf format warnings on 32-bit build
      doc: nft.8 add chain synopsis
      doc: nft.8 Document rule replace

Elise Lennion (27):
      src: sort set elements in netlink_get_setelems()
      tests: shell: add maps tests
      src: Allow to list ruleset without stateful information
      doc: Include stateless option
      tests: py: Use stateless option on tests
      tests: shell: Add test for set element leak bug
      src: Honor obligatory stateless printing of flow tables
      tests: shell: Add test for flush bug on sets with size
      src: Allow reset single stateful object
      src: Allow list single stateful object
      evaluate: Evaluate table name before reset stateful objects in a table
      src: Allow list stateful objects in a table
      tests: py: Add suport for stateful objects in python tests
      tests: py: Add tests for stateful objects
      main: Validate the number of numeric options
      src: Always print range expressions numerically
      configure: Require newer version of libxtables
      datatype: Replace getaddrinfo() by internal lookup table
      doc: Document stateful objects
      doc: Document sets
      statement: Avoid rounding bytes in get_rate()
      doc: Document maps
      parser_bison: Allow flushing flow tables
      parser_bison: Allow flushing maps
      doc: Document add / delete element operations of sets and maps
      src: Make flush command selective of the set structure type
      tests: Add test for flush command selectiveness on sets

Eric Leblond (12):
      src: fix memory leak when listing rules
      parser: fix memory leak in set creation
      parser: fix bison warnings
      parser: error if needed at EOF
      evaluate: fix build with clang
      scanner: free filename when destroying scanner
      cli: fix heap buffer overflow
      mnl: fix error handling in mnl_batch_talk
      src: add nft_ctx_new() and nft_ctx_free()
      src: add nft_run_cmd_*() functions
      src: release caches from nft_ctx_free() path
      src: add flags fo nft_ctx_new

Florian Westphal (75):
      parser_bison: remove obsolete tokens
      evaluate: fix export length and data corruption
      statement: fix print of ip dnat address
      payload: insert implicit meta tcp dependency when matching tcp options
      src: add conntrack zone support
      ct: refactor print function so it can be re-used for ct statement
      src: support zone set statement with optional direction
      tests: add test entries for conntrack zones
      netlink: BUG when object type is unknown
      doc: ct zone set support
      files: provide 'raw' table equivalent
      doc: fix ct mark set example
      src: add initial ct helper support
      evaluate: refactor CMD_OBJ_QUOTA/COUNTER handling
      src: allow listing all ct helpers
      src: implement add/create/delete for ct helper objects
      tests: py: add ct helper tests
      tests: add insert-failure test
      doc: ct helper objects and helper set support
      exthdr: avoid crash with older kernels
      src: fix build warning on i686
      ct: add conntrack event mask support
      parser: allow listing sets in one table
      netlink_delinearize: don't kill dependencies accross statements
      tests: don't kill dependencies accross statements
      netlink_delink_delinearize: don't store dependency unless relop checks is eq check
      src: add a comment wrt. reject dependency insertion
      netlink_delinearize: reject: remove dependency for tcp-resets
      tests: add ip reject with tcp and check for mark too
      payload: split ll proto dependency into helper
      src: allow update of net base w. meta l4proto icmpv6
      src: ipv6: switch implicit dependencies to meta l4proto
      payload: enforce ip/ip6 protocol depending on icmp or icmpv6
      tests: fix up meta l4proto change for ip6 family
      src: ip: switch implicit dependencies to meta l4proto too
      tests: fix up meta l4proto change for ip family
      Merge branch 'meta_l4_dependency'
      ct: fix inet/bridge/netdev family handling for saddr/daddr
      meta: permit meta nfproto ip in ip family
      parser: allow ct eventmask set new,related
      netlink_delinearize: prefer ct event set foo,bar over 'set foo|bar'
      src: rename ct eventmask to event
      tests: restrict ct saddr test to inet family
      tests: remove two non-sensical rules
      tests: restrict meta nfproto test cases to inet family
      evaluate: reject meta nfproto outside of inet family
      tcpopt: make tcptopt structs static
      exhtdr: pacify compiler warning
      parser: compact list of rhs keyword expressions
      rename struct ct to ct_helper
      tests: fix arp chain test
      evaluate: shift immediate value when adjusting size for csum fixup
      tests: add test case for ttl/protocol set
      src: add tcp options set support
      rt: add path mtu support
      tests: add tcp option set support / tcp mss mangling test cases
      files: add arp filter and add in/output to nat skeleton
      evaluate: convert expr_rt byteorder when evaluating statment arg
      evaluate: prepare to store expr key rather than datatype
      src: store expression as set key instead of data type
      src: ct: remove unused functions
      src: prepare for future ct timeout policy support
      src: parser_bison: rename ct_block to ct_helper_block
      bison: permit keywords in list_stmt_expressions
      src: add alternate syntax for ct saddr
      src: ct: store proto base of ct key, if any
      src: ct: add eval part to inject dependencies for ct saddr/daddr
      src: unifiy meta and ct postprocessing
      tests: update inet/bridge icmp test case
      src: ct: print nfproto name for some header fields
      tests: ct: adjust test case commands
      src: rt: add keyword distinction for nexthop vs nexthop6
      tests: rt: fix test cases
      doc: update man page
      Merge branch 'ct_rt_syntax_06'

Harsha Sharma (5):
      INSTALL: Update dependency list and configure with libxtables support
      test: shell: execute shell/run-tests.sh from any directory
      tests: shell: add testcases for named objects
      test: shell: update shell/run-tests.sh to refer to relative path of testcase
      nftables: make pointers in string arrays constant

Ismo Puustinen (8):
      scanner: add support for include directories
      tests: test include directories
      scanner: add files in include dirs in alphabetical order.
      man: add include directory documentation.
      tests: added tests for ordering files in include dirs.
      scanner: support for wildcards in include statements.
      tests: update include directory tests to support wildcard syntax.
      man: add include statement documentation.

JingPiao Chen (1):
      include: remove outdated comment

Laura Garcia Liebana (2):
      src: hash: support of symmetric hash
      src: hash: fix seed attribute not listed

Liping Zhang (4):
      ct: add average bytes per packet counter support
      src: fix crash when inputting an incomplete set add command
      hash: generate a random seed if seed option is empty
      src: delete the old cache when dumping is interrupted

Manuel Messner (7):
      evaluate: fix typo
      include: linux: netfilter: nf_tables: copy file from nf-next
      exthdr: prepare for tcp support
      exthdr: prepare exthdr_gen_dependency for tcp support
      src: add TCP option matching
      payload: automatically kill dependencies for exthdr and tcpopt
      tests: py: Add basic tests for ip, ip6 and inet

Oleksandr Natalenko (2):
      utils: provide array allocation wrapper
      segtree: allocate memory for arrays on heap

Pablo M. Bermudo Garay (5):
      src: add new generic context structure nft_ctx
      src: add --check option flag
      parser: fix typo
      src: add stateful object support for limit
      tests: py: add tests for limit stateful object

Pablo Neira Ayuso (65):
      include: fetch nf_tables.h updates
      src: remove SET_F_* flag definitions
      src: add used quota support
      src: listing of stateful objects
      src: add/create/delete stateful objects
      src: reset internal stateful objects
      parser_bison: allow RESET token from rhs
      src: add stateful object reference expression
      src: add support for stateful object maps
      src: support for stateful object monitoring
      mnl: use nftnl_nlmsg_build_hdr()
      tests: shell: cover cache flush on flush ruleset
      netlink: don't bail out on dump errors
      rule: check for errors from cache_init_objects() for stateful objects
      tests: shell: validate set size
      include: refresh linux/netfilter/nf_tables.h
      tests: shell: remove nft_set_bitmap on each run
      include: add tcpopt.h to Makefile.am
      evaluate: store byteorder for set keys
      netlink: store set byteorder in NFTA_SET_USERDATA
      netlink_delinearize: remove integer_type_postprocess()
      tests: py: fix incorrect bytecode in numgen and hash mappings
      datatype: add DTYPE_F_CLONE flag
      evaluate: set byteorder as lhs expression context in stmt_evaluate_arg()
      src: rename set_keytype_alloc() to set_datatype_alloc()
      netlink: rework NFTNL_SET_USERDATA to accomodate new attributes
      src: store byteorder for set data
      src: revisit tcp options support
      netlink: use nftnl_udata_put_u32()/nftnl_udata_get_u32()
      tests: shell: test comments with small sets
      doc: revisit fib examples
      tests: shell: delete element and chain in transaction
      rule: adjust set expression size accordingly with intervals
      segtree: reset element size counter before adding intervals to set
      netlink: add size description for constant sets
      tests: py: flush stdout on each test run
      expression: don't trim off unary expression on delinearization
      src: remove global nftnl_batch structure in mnl layer
      doc: nft: describe -I override behaviour when path starts by ./ and /
      src: display default directory for file inclusion in -h/--help
      src: error reporting for nested ruleset representation
      evaluate: merge nested set flags
      evaluate: Better error reporting for bad set references
      include: Pass nf_sock where needed as parameter
      src: handle rule tracing as an monitor object
      src: make netlink sequence number non-static
      rule: remove sequence number from struct eval_ctx
      src: add include_paths to struct nft_ctx
      src: add maximum number of parser errors to struct nft_ctx
      src: remove ifdef DEBUG pollution
      src: add struct mnl_ctx
      mnl: pass struct netlink_ctx to mnl_nft_socket_sendmsg()
      src: add debugging mask to context structure
      src: call nft_init() and nft_exit() from context routines
      src: move nf_sock into nft_ctx structure
      netlink: remove nfsock_open()
      src: add nft_ctx_netlink_init()
      mnl: do not set NLM_F_CREATE in deletion requests
      parser_bison: consolidate stmt_expr rule
      parser_bison: use keywords in ct expression
      parser_bison: allow helper keyword in ct object kind
      mnl: fix broken sequence number allocation
      parser_bison: allow to used named limit from dictionaries too
      scanner: IPv4-Mapped IPv6 addresses support
      configure: Bump version to v0.8

Phil Sutter (56):
      Introduce boolean datatype and boolean expression
      exthdr: Add support for exthdr specific flags
      exthdr: Implement existence check
      fib: Support existence check
      tests: Adjust for changed exthdr debug output
      doc: Document boolean type and applications
      proto: Add some exotic ICMPv6 types
      doc: Describe ICMP(v6) expression and types
      evaluate: set: Allow for set elems to be sets
      evaluate: set: Fix nested set merge size adjustment
      tests: Add test cases for nested anonymous sets
      tests: shell: netns/0003many_0: Fix cleanup after error
      sets: Fix for missing space after last element
      nft.8: Enhance NAT documentation
      segtree: Fix expr_value_cmp()
      src: Allow passing the parent set to set_expr_alloc()
      segtree: Introduce flag for half-open range elements
      tests: Add basic monitor testing framework
      monitor: Print NEWGEN events
      tests/monitor: Ignore newgen messages in output
      tests/monitor: Add a small README
      monitor: Fix printing of set declarations
      tests/monitor: Simplify testcases
      tests/monitor: Clear ruleset after testing
      mnl: Consolidate mnl_batch_talk() parameters
      netlink: Pass nlmsg flags from rule.c
      Implement --echo option
      tests: Add a simple test suite for --echo option
      nft.8: Document operations on ruleset
      nft.8: Describe conntrack types
      nft.8: Review reject statement description
      nft.8: Add note about supported hooks for bridge family
      nft.8: Describe base chain details
      nft.8: Complete short description of arp family
      mnl: Drop --echo support for non-batch calls
      netlink: Fix segfault when using --echo flag
      tests: Merge monitor and echo test suites
      echo: Fix for added delays in rule updates
      src: Keep cache in struct nft_ctx
      src: Fix debug_proto_ctx()
      scanner: Fix for memleak due to unclosed file pointer
      scanner: Fix for wrong parameter type of scanner_destroy()
      scanner: Make use of yylex_init_extra()
      parser: Fix for memleak when commands fail
      parser: Fix memleaks for STRING token (and derived ones)
      doc: nft: Fix and enhance synopsis section
      main: Fix for return of uninitialized variable in nft_run_cmd_from_filename()
      monitor: Fix for incorrect debug_mask
      rule: Use C99-style initializer in cache_init()
      exthdr: Simplify tcp option printing a bit
      erec_print: Pass output FILE pointer to netlink_dump_expr()
      rule: Refactor chain_print_declaration()
      src: get rid of printf
      main: Flush output from nft_gmp_print()
      main: Drop stdout hack to expose nft_print() implementation issues
      evaluate: Fix debug output

Shyam Saini (4):
      tests: py: Add test for ambiguity while setting the value
      tests: shell: Add test for incomplete set add set command
      tests: shell: Test input descriptors for included files
      tests: py: Fail test forcefully when bug is not fixed

Timothy Redaelli (1):
      tests: py: fix some typos in README

Tobias Klauser (1):
      build: add missing backslash to list of CFLAGS

Varsha Rao (18):
      src: Remove expire information from list stateless ruleset.
      src: Pass stateless, numeric, ip2name and handle variables as structure members.
      include: Remove datatype_register().
      src: Remove __init and __exit macro definitions.
      src: netlink: Remove variable nf_mon_sock.
      src: nft monitor ruleset
      man: Document nft monitor ruleset.
      src: netlink: Subscribe nft monitor and nft monitor trace to respective groups.
      src: introduce struct nft_cache
      man: Fix typo.
      src: Remove xt_stmt_() functions.
      src: gmputil: Remove mpz_get_be64() function.
      src: cli: Remove cli_display() function.
      src: netlink: Remove unused functions.
      src: mnl: Remove unused functions.
      src: Initialize struct stmt in _match and _target functions.
      tests: shell: Add tests for chain rename.
      tests: files: Remove old tests for chain rename.

More information about the netfilter-announce mailing list