Eric Dumazet
ebc0ffae5d
fib: RCU conversion of fib_lookup()
fib_lookup() converted to be called in RCU protected context, no
reference taken and released on a contended cache line (fib_clntref)
fib_table_lookup() and fib_semantic_match() get an additional parameter.
struct fib_info gets an rcu_head field, and is freed after an rcu grace
period.
Stress test :
(Sending 160.000.000 UDP frames on same neighbour,
IP route cache disabled, dual E5540 @2.53GHz,
32bit kernel, FIB_HASH) (about same results for FIB_TRIE)
Before patch :
real 1m31.199s
user 0m13.761s
sys 23m24.780s
After patch:
real 1m5.375s
user 0m14.997s
sys 15m50.115s
Before patch Profile :
13044.00 15.4% __ip_route_output_key vmlinux
8438.00 10.0% dst_destroy vmlinux
5983.00 7.1% fib_semantic_match vmlinux
5410.00 6.4% fib_rules_lookup vmlinux
4803.00 5.7% neigh_lookup vmlinux
4420.00 5.2% _raw_spin_lock vmlinux
3883.00 4.6% rt_set_nexthop vmlinux
3261.00 3.9% _raw_read_lock vmlinux
2794.00 3.3% fib_table_lookup vmlinux
2374.00 2.8% neigh_resolve_output vmlinux
2153.00 2.5% dst_alloc vmlinux
1502.00 1.8% _raw_read_lock_bh vmlinux
1484.00 1.8% kmem_cache_alloc vmlinux
1407.00 1.7% eth_header vmlinux
1406.00 1.7% ipv4_dst_destroy vmlinux
1298.00 1.5% __copy_from_user_ll vmlinux
1174.00 1.4% dev_queue_xmit vmlinux
1000.00 1.2% ip_output vmlinux
After patch Profile :
13712.00 15.8% dst_destroy vmlinux
8548.00 9.9% __ip_route_output_key vmlinux
7017.00 8.1% neigh_lookup vmlinux
4554.00 5.3% fib_semantic_match vmlinux
4067.00 4.7% _raw_read_lock vmlinux
3491.00 4.0% dst_alloc vmlinux
3186.00 3.7% neigh_resolve_output vmlinux
3103.00 3.6% fib_table_lookup vmlinux
2098.00 2.4% _raw_read_lock_bh vmlinux
2081.00 2.4% kmem_cache_alloc vmlinux
2013.00 2.3% _raw_spin_lock vmlinux
1763.00 2.0% __copy_from_user_ll vmlinux
1763.00 2.0% ip_output vmlinux
1761.00 2.0% ipv4_dst_destroy vmlinux
1631.00 1.9% eth_header vmlinux
1440.00 1.7% _raw_read_unlock_bh vmlinux
Reference results, if IP route cache is enabled :
real 0m29.718s
user 0m10.845s
sys 7m37.341s
25213.00 29.5% __ip_route_output_key vmlinux
9011.00 10.5% dst_release vmlinux
4817.00 5.6% ip_push_pending_frames vmlinux
4232.00 5.0% ip_finish_output vmlinux
3940.00 4.6% udp_sendmsg vmlinux
3730.00 4.4% __copy_from_user_ll vmlinux
3716.00 4.4% ip_route_output_flow vmlinux
2451.00 2.9% __xfrm_lookup vmlinux
2221.00 2.6% ip_append_data vmlinux
1718.00 2.0% _raw_spin_lock_bh vmlinux
1655.00 1.9% __alloc_skb vmlinux
1572.00 1.8% sock_wfree vmlinux
1345.00 1.6% kfree vmlinux
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2010-10-05 20:39:38 -07:00
..
2010-08-02 14:28:34 -05:00
2010-09-23 14:33:39 -07:00
2010-06-29 00:08:21 -07:00
2010-09-23 14:33:39 -07:00
2010-03-30 22:02:32 +09:00
2010-08-02 17:20:54 +02:00
2010-05-11 14:40:55 +02:00
2010-09-27 21:30:41 -07:00
2010-09-23 14:33:39 -07:00
2010-08-20 01:42:59 -07:00
2010-09-23 14:33:39 -07:00
2010-06-11 18:37:08 -07:00
2010-10-05 00:47:39 -07:00
2009-08-12 21:54:50 -07:00
2010-06-16 14:58:16 -07:00
2009-11-04 05:06:25 -08:00
2010-09-29 19:45:35 -07:00
2010-03-30 22:02:32 +09:00
2010-09-16 15:19:44 -04:00
2010-09-03 09:55:24 -07:00
2009-12-11 15:07:56 -08:00
2009-09-01 01:24:30 -07:00
2010-06-03 03:21:52 -07:00
2010-03-27 19:23:46 -07:00
2010-06-03 03:21:52 -07:00
2010-06-10 23:31:35 -07:00
2009-09-01 17:40:31 -07:00
2010-10-03 22:17:54 -07:00
2010-10-05 20:39:38 -07:00
2010-09-28 23:38:15 -07:00
2009-11-04 05:06:25 -08:00
2010-06-02 05:16:23 -07:00
2010-08-21 23:05:39 -07:00
2010-04-13 14:49:31 -07:00
2010-02-08 16:50:53 -05:00
2009-11-06 14:32:18 +03:00
2010-05-18 15:36:06 -07:00
2010-04-15 23:36:37 -07:00
2009-12-08 20:17:51 -08:00
2010-07-12 20:21:46 -07:00
2010-08-30 13:23:33 -07:00
2010-09-23 14:33:39 -07:00
2010-06-30 13:44:29 -07:00
2009-12-08 20:17:51 -08:00
2010-06-23 13:16:38 -07:00
2010-04-27 12:39:01 -07:00
2010-06-16 11:55:39 -07:00
2010-06-10 23:31:35 -07:00
2010-04-13 01:41:33 -07:00
2010-06-03 03:21:52 -07:00
2010-10-05 20:39:38 -07:00
2010-09-08 10:39:57 -07:00
2010-09-23 14:33:39 -07:00
2010-02-16 23:05:38 -08:00
2010-09-27 21:30:42 -07:00
2010-09-23 14:33:39 -07:00
2010-06-03 03:21:52 -07:00
2010-03-31 14:49:12 -04:00
2010-07-26 15:09:04 -04:00
2009-12-26 20:43:57 -08:00
2009-12-26 20:46:28 -08:00
2010-09-27 15:57:54 -04:00
2010-06-03 03:21:52 -07:00
2010-04-23 13:35:55 +09:00
2010-06-03 03:21:52 -07:00
2010-10-05 14:54:36 -07:00
2010-03-12 15:52:40 -08:00
2010-03-30 22:02:32 +09:00
2010-06-23 12:58:40 -07:00
2010-03-30 22:02:32 +09:00
2009-08-19 23:08:22 +04:00
2009-11-20 15:35:04 -08:00
2010-06-02 03:23:51 -07:00
2009-11-05 21:40:17 -08:00
2010-09-08 13:42:22 -07:00
2009-11-04 05:06:25 -08:00
2010-07-20 16:44:35 -04:00
2010-01-17 19:09:39 -08:00
2009-07-26 19:11:14 -07:00
2010-09-27 15:03:33 -07:00
2010-02-27 02:43:40 -08:00
2010-09-23 14:33:39 -07:00
2010-06-16 14:55:56 -07:00
2010-06-30 13:31:19 -07:00
2010-09-26 18:53:07 -07:00
2010-09-29 19:45:36 -07:00
2010-04-23 23:35:28 -07:00
2010-09-08 21:45:01 -07:00
2009-10-07 16:39:43 -04:00
2009-12-09 19:43:33 -08:00
2009-11-06 14:32:24 +03:00
2010-05-17 17:39:27 -07:00
2010-04-22 16:12:36 -07:00
2010-09-20 11:11:38 -07:00