Project

General

Profile

Actions

Bug #7405

closed

segmentation fault: tab at beginning of header line in http request

Added by Ray . about 4 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
01/18/2020
Due date:
% Done:

0%

Estimated time:

Description

This is a hypothesis of what causes a segmentation fault. Do I need to collect additional info?

This is a packet capture. Notice at the end of the line indicated with offset 0x0110, there is 09 (which is a tab character). It is effectively at the beginning of line.

20:46:42.052919 IP 209.0.146.77.45247 > 63.85.43.206.http: Flags [P.], seq 1:669, ack 1, win 115, options [nop,nop,TS val 2215263374 ecr 736241370], length 668: HTTP: GET /blog/Trading/SmartQuant/index.blog HTTP/1.1
        0x0000:  4500 02d0 4050 4000 3806 3167 d100 924d  E...@P@.8.1g...M
        0x0010:  3f55 2bce b0bf 0050 7875 bb27 ae93 6803  ?U+....Pxu.'..h.
        0x0020:  8018 0073 ee66 0000 0101 080a 840a 3c8e  ...s.f........<.
        0x0030:  2be2 26da 4745 5420 2f62 6c6f 672f 5472  +.&.GET./blog/Tr
        0x0040:  6164 696e 672f 536d 6172 7451 7561 6e74  ading/SmartQuant
        0x0050:  2f69 6e64 6578 2e62 6c6f 6720 4854 5450  /index.blog.HTTP
        0x0060:  2f31 2e31 0d0a 4578 7069 7265 733a 2053  /1.1..Expires:.S
        0x0070:  6174 204a 616e 2031 3820 3230 3a35 353a  at.Jan.18.20:55:
        0x0080:  3335 2055 5443 2032 3032 300d 0a43 6f6e  35.UTC.2020..Con
        0x0090:  7465 6e74 2d54 7970 653a 2074 6578 742f  tent-Type:.text/
        0x00a0:  6874 6d6c 0d0a 4c6f 6361 7469 6f6e 3a20  html..Location:.
        0x00b0:  6874 7470 3a2f 2f77 7777 2e6f 6e65 756e  http://www.oneun
        0x00c0:  6966 6965 642e 6e65 742f 626c 6f67 2f54  ified.net/blog/T
        0x00d0:  7261 6469 6e67 2f53 6d61 7274 5175 616e  rading/SmartQuan
        0x00e0:  742f 696e 6465 782e 626c 6f67 0d0a 4461  t/index.blog..Da
        0x00f0:  7465 3a20 5361 742c 2031 3820 4a61 6e20  te:.Sat,.18.Jan.
        0x0100:  3230 3230 2032 303a 3435 3a33 3520 474d  2020.20:45:35.GM
        0x0110:  540d 0a56 6172 793a 2048 6f73 740d 0a09  T..Vary:.Host...
        0x0120:  4163 6365 7074 2d45 6e63 6f64 696e 670d  Accept-Encoding.
        0x0130:  0a58 2d46 726f 6d3a 2073 6670 2d77 6562  .X-From:.sfp-web
        0x0140:  2d32 0d0a 4163 6365 7074 3a20 7465 7874  -2..Accept:.text
        0x0150:  2f68 746d 6c2c 6170 706c 6963 6174 696f  /html,applicatio
        0x0160:  6e2f 7868 746d 6c2b 786d 6c2c 6170 706c  n/xhtml+xml,appl
        0x0170:  6963 6174 696f 6e2f 786d 6c3b 713d 302e  ication/xml;q=0.
        0x0180:  392c 2a2f 2a3b 713d 302e 380d 0a41 6363  9,*/*;q=0.8..Acc
        0x0190:  6570 742d 456e 636f 6469 6e67 3a20 677a  ept-Encoding:.gz
        0x01a0:  6970 2c64 6566 6c61 7465 0d0a 486f 7374  ip,deflate..Host
        0x01b0:  3a20 7777 772e 6f6e 6575 6e69 6669 6564  :.www.oneunified
        0x01c0:  2e6e 6574 0d0a 5573 6572 2d41 6765 6e74  .net..User-Agent
        0x01d0:  3a20 4d6f 7a69 6c6c 612f 352e 3020 2857  :.Mozilla/5.0.(W
        0x01e0:  696e 646f 7773 204e 5420 362e 313b 2057  indows.NT.6.1;.W
        0x01f0:  4f57 3634 2920 4170 706c 6557 6562 4b69  OW64).AppleWebKi
        0x0200:  742f 3533 372e 3336 2028 4b48 544d 4c2c  t/537.36.(KHTML,
        0x0210:  206c 696b 6520 4765 636b 6f29 2043 6872  .like.Gecko).Chr
        0x0220:  6f6d 652f 3530 2e30 2e32 3636 312e 3934  ome/50.0.2661.94
        0x0230:  2053 6166 6172 692f 3533 372e 3336 0d0a  .Safari/537.36..
        0x0240:  5669 613a 2031 2e31 2070 6c70 726f 7879  Via:.1.1.plproxy
        0x0250:  636c 7573 7434 2e70 726f 642e 6379 7665  clust4.prod.cyve
        0x0260:  696c 6c61 6e63 652e 636f 6d20 2873 7175  illance.com.(squ
        0x0270:  6964 2f33 2e32 2e38 290d 0a58 2d46 6f72  id/3.2.8)..X-For
        0x0280:  7761 7264 6564 2d46 6f72 3a20 3130 2e32  warded-For:.10.2
        0x0290:  302e 3132 2e31 3334 0d0a 4361 6368 652d  0.12.134..Cache-
        0x02a0:  436f 6e74 726f 6c3a 206d 6178 2d61 6765  Control:.max-age
        0x02b0:  3d36 3030 0d0a 436f 6e6e 6563 7469 6f6e  =600..Connection
        0x02c0:  3a20 6b65 6570 2d61 6c69 7665 0d0a 0d0a  :.keep-alive....

This is the segmentation fault stack (pardon for the control characters, not sure how to fix that):

Thread 6 "AppOneUnifiedNe" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffebfff700 (LWP 9742)]
[33mhttp::server::RequestParser::consumeChar[m ([36mthis[m=0x7fffe0002f18, [36md[m=0x7fffd400b24c "Accept-Encoding\r\nX-From: sfp-web-2\r\nAccept: text/html,application/xht
ml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows N"...) at [32m/home/rpb/data/project
s/libs-build/wt/src/http/RequestParser.C[m:120
120     /home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C: No such file or directory.
(gdb) p d
$1 = [34m0x7fffd400b24c[m "Accept-Encoding\r\nX-From: sfp-web-2\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\
r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows N"...
(gdb)

Here is a top few stack entries:

#0  [34m0x00007ffff6ce0085[m in [33mhttp::server::RequestParser::consumeChar(char*)[m (this=0x7fffe0002f18, d=0x7fffd400b24c "Accept-Encoding\r\nX-From: sfp-web-2\r\nAccept
: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows N"...) at
 [32m/home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C[m:120
#1  [34m0x00007ffff6ce5861[m in [33mhttp::server::RequestParser::consume(http::server::Request&, char*)[m (this=0x7fffe0002f18, req=..., it=0x7fffd400b24c "Accept-Encoding\
r\nX-From: sfp-web-2\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent:
Mozilla/5.0 (Windows N"...) at [32m/home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C[m:1089
#2  [34m0x00007ffff6ce01cd[m in [33mhttp::server::RequestParser::parse(http::server::Request&, char*, char*)[m (this=0x7fffe0002f18, req=..., begin=0x7fffd400b24d "ccept-En
coding\r\nX-From: sfp-web-2\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-
Agent: Mozilla/5.0 (Windows NT"..., end=0x7fffd400b3fc "") at [32m/home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C[m:161
#3  [34m0x00007ffff6c95181[m in [33mhttp::server::Connection::handleReadRequest0()[m (this=0x7fffe0002d00) at [32m/home/rpb/data/projects/libs-build/wt/src/http/Connection.
C[m:192
(More stack frames follow...)

In looking at the source code in src/http/RequestParser.C:

boost::tribool& RequestParser::consume(Request& req, char *it)

may need a switch statement for handling that character, although it does have a default false return, which may not be processed properly

Actions #1

Updated by Ray . about 4 years ago

a better stack trace:

(gdb) info stack 10
#0  0x00007ffff6ce0085 in http::server::RequestParser::consumeChar(char*)
    (this=0x7fffd0004598, d=0x7fffc800b0fc "Accept-Encoding\r\nX-From: sfp-web-3\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows N"...) at /home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C:120
#1  0x00007ffff6ce5861 in http::server::RequestParser::consume(http::server::Request&, char*)
    (this=0x7fffd0004598, req=..., it=0x7fffc800b0fc "Accept-Encoding\r\nX-From: sfp-web-3\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows N"...) at /home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C:1089
#2  0x00007ffff6ce01cd in http::server::RequestParser::parse(http::server::Request&, char*, char*)
    (this=0x7fffd0004598, req=..., begin=0x7fffc800b0fd "ccept-Encoding\r\nX-From: sfp-web-3\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows NT"..., end=0x7fffc800b2ac "") at /home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C:161
#3  0x00007ffff6c95181 in http::server::Connection::handleReadRequest0() (this=0x7fffd0004380) at /home/rpb/data/projects/libs-build/wt/src/http/Connection.C:192
#4  0x00007ffff6c95ad7 in http::server::Connection::handleReadRequest(boost::system::error_code const&, unsigned long) (this=0x7fffd0004380, e=..., bytes_transferred=668) at /home/rpb/data/projects/libs-build/wt/src/http/Connection.C:264
#5  0x00007ffff6d4ea65 in std::__invoke_impl<void, void (http::server::Connection::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<http::server::TcpConnection>&, boost::system::error_code const&, unsigned long const&>(std::__invoke_memfun_deref, void (http::server::Connection::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<http::server::TcpConnection>&, boost::system::error_code const&, unsigned long const&) (__f=
    @0x7ffff0fb1040: (void (http::server::Connection::*)(http::server::Connection * const, const boost::system::error_code &, unsigned long)) 0x7ffff6c95a64 <http::server::Connection::handleReadRequest(boost::system::error_code const&, unsigned long)>, __t=std::shared_ptr<http::server::TcpConnection> (use count 4, weak count 1) = {...}) at /usr/include/c++/9/bits/invoke.h:73
#6  0x00007ffff6d4e99f in std::__invoke<void (http::server::Connection::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<http::server::TcpConnection>&, boost::system::error_code const&, unsigned long const&>(void (http::server::Connection::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<http::server::TcpConnection>&, boost::system::error_code const&, unsigned long const&) (__fn=
    @0x7ffff0fb1040: (void (http::server::Connection::*)(http::server::Connection * const, const boost::system::error_code &, unsigned long)) 0x7ffff6c95a64 <http::server::Connection::handleReadRequest(boost::system::error_code const&, unsigned long)>) at /usr/include/c++/9/bits/invoke.h:95
#7  0x00007ffff6d4e8e0 in std::_Bind<void (http::server::Connection::*(std::shared_ptr<http::server::TcpConnection>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>::__call<void, boost::system::error_code const&, unsigned long const&, 0ul, 1ul, 2ul>(std::tuple<boost::system::error_code const&, unsigned long const&>&&, std::_Index_tuple<0ul, 1ul, 2ul>) (this=0x7ffff0fb1040, __args=...) at /usr/include/c++/9/functional:400
#8  0x00007ffff6d4e838 in std::_Bind<void (http::server::Connection::*(std::shared_ptr<http::server::TcpConnection>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>::operator()<boost::system::error_code const&, unsigned long const&, void>(boost::system::error_code const&, unsigned long const&) (this=0x7ffff0fb1040) at /usr/include/c++/9/functional:484
#9  0x00007ffff6d4e7a1 in boost::asio::detail::binder2<std::_Bind<void (http::server::Connection::*(std::shared_ptr<http::server::TcpConnection>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>, boost::system::error_code, unsigned long>::operator()() (this=0x7ffff0fb1040) at /usr/local/include/boost-1_69/boost/asio/detail/bind_handler.hpp:164
Actions #2

Updated by Ray . about 4 years ago

Source is from git master around Jan 11.

Actions #3

Updated by Ray . about 4 years ago

Seems currentString_ hasn't been initialized:

(gdb) p d
$1 = 0x7fffc800b0fc "Accept-Encoding\r\nX-From: sfp-web-3\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows N"...
(gdb) p currentString_
$2 = (http::server::buffer_string *) 0x0
(gdb) p currentString_->data
Cannot access memory at address 0x0
(gdb) p currentString_->len
Cannot access memory at address 0x8
(gdb) p &currentString_
$3 = (http::server::buffer_string **) 0x7fffd0004650

in this bit of code in src/http/RequestParser.C:

   2 bool RequestParser::consumeChar(char *d)
   1 {
120    if (currentString_->data == 0)
   1     currentString_->data = &(*d);
   2
   3   if (++currentString_->len > maxSize_)
   4     return false;
   5
   6   return true;
   7 }

which is called from

  42   case header_line_start:
  41     if (input == '\r')
  40     {
  39       httpState_ = expecting_newline_3;
  38       return Indeterminate;
  37     }
  36     else if ((input == ' ' || input == '\t') && haveHeader_)
  35     {
  34       // continuation of previous header
  33       httpState_ = header_lws;
  32       return Indeterminate;
  31     }
  30     else if (!is_char(input) || is_ctl(input) || is_tspecial(input))
  29     {
  28       return False;
  27     }
  26     else
  25     {
  24       req.headers.push_back(Request::Header());
  23       haveHeader_ = true;
  22       consumeToString(req.headers.back().name, MAX_FIELD_NAME_SIZE);
  21       consumeChar(it);
  20       httpState_ = header_name;
  19       return Indeterminate;
  18     }
  17   case header_lws:
  16     if (input == '\r')
  15     {
  14       httpState_ = expecting_newline_2;
  13       return Indeterminate;
  12     }
  11     else if (input == ' ' || input == '\t')
  10     {
   9       return Indeterminate;
   8     }
   7     else if (is_ctl(input))
   6     {
   5       return False;
   4     }
   3     else
   2     {
   1       httpState_ = header_value;
1089       consumeChar(it);
   1       return Indeterminate;
   2     }
   3   case header_name:

and the code at line --36 was probably invoked prior to arrival at 1089 which doesn't initialize with a call to consumeToString of some sort.

Looking at the previous frame:

(gdb) frame 1
#1  0x00007ffff6ce5861 in http::server::RequestParser::consume (this=0x7fffd0004598, req=...,
    it=0x7fffc800b0fc "Accept-Encoding\r\nX-From: sfp-web-3\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows N"...)
    at /home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C:1089
1089    in /home/rpb/data/projects/libs-build/wt/src/http/RequestParser.C
(gdb) p it
$8 = 0x7fffc800b0fc "Accept-Encoding\r\nX-From: sfp-web-3\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate\r\nHost: www.oneunified.net\r\nUser-Agent: Mozilla/5.0 (Windows N"...
(gdb) p input
$9 = 65 'A'
(gdb) p httpState_
$10 = http::server::RequestParser::header_value
(gdb) p req.headers
$11 = std::__cxx11::list = {[0] = {name = {data = 0x7fffc800b042 "Expires", len = 7, next = 0x0}, value = {data = 0x7fffc800b04b "Sat Jan 18 22:46:10 UTC 2020", len = 28, next = 0x0}}, [1] = {name = {data = 0x7fffc800b069 "Content-Type", len = 12,
      next = 0x0}, value = {data = 0x7fffc800b077 "text/html", len = 9, next = 0x0}}, [2] = {name = {data = 0x7fffc800b082 "Location", len = 8, next = 0x0}, value = {
      data = 0x7fffc800b08c "http://www.oneunified.net/blog/Trading/SmartQuant/index.blog", len = 60, next = 0x0}}, [3] = {name = {data = 0x7fffc800b0ca "Date", len = 4, next = 0x0}, value = {data = 0x7fffc800b0d0 "Sat, 18 Jan 2020 22:36:10 GMT",
      len = 29, next = 0x0}}, [4] = {name = {data = 0x7fffc800b0ef "Vary", len = 4, next = 0x0}, value = {data = 0x7fffc800b0f5 "Host", len = 4, next = 0x0}}}
(gdb) p haveHeader_
$12 = true
Actions #4

Updated by Ray . about 4 years ago

I am trying this code as a work around:

$ git diff src/http/RequestParser.C
diff --git a/src/http/RequestParser.C b/src/http/RequestParser.C
index 654092ec..dcbb8fc2 100644
--- a/src/http/RequestParser.C
+++ b/src/http/RequestParser.C
@@ -1050,6 +1050,10 @@ boost::tribool& RequestParser::consume(Request& req, char *it)
       httpState_ = expecting_newline_3;
       return Indeterminate;
     }
+    else if (( input == '\t') && (nullptr == currentString_ ))
+    {
+      return Indeterminate;
+    }
     else if ((input == ' ' || input == '\t') && haveHeader_)
     {
       // continuation of previous header
Actions #5

Updated by Ray . about 4 years ago

or

$ git diff ../src/http/RequestParser.C
diff --git a/src/http/RequestParser.C b/src/http/RequestParser.C
index 654092ec..5dc0973d 100644
--- a/src/http/RequestParser.C
+++ b/src/http/RequestParser.C
@@ -1050,6 +1050,10 @@ boost::tribool& RequestParser::consume(Request& req, char *it)
       httpState_ = expecting_newline_3;
       return Indeterminate;
     }
+    else if (((input == ' ') || (input == '\t')) && (nullptr == currentString_ ))
+    {
+      return Indeterminate;
+    }
     else if ((input == ' ' || input == '\t') && haveHeader_)
     {
       // continuation of previous header

but that haveHeader_ check may cause another hole in the logic somewhere I think.

Actions #6

Updated by Roel Standaert about 4 years ago

Good catch. I'm tempted to drop support for line folding rather than fixing this issue.

Page 26 of RFC 7230 says:

   A server that receives an obs-fold in a request message that is not
   within a message/http container MUST either reject the message by
   sending a 400 (Bad Request), preferably with a representation
   explaining that obsolete line folding is unacceptable, or replace
   each received obs-fold with one or more SP octets prior to
   interpreting the field value or forwarding the message downstream.
Actions #7

Updated by Roel Standaert about 4 years ago

I mean, that would actually fix this issue. The fact that you're the first person to report this indicates to me that clients generally don't use line folding, and indeed the latest RFC says they "MUST NOT" do that.

Actions #8

Updated by Ray . about 4 years ago

Someone was doing something weird, and could maybe have been a misbehavior probe anyway. If dropping support fixes the problem, with an appropriate 400 return, I'm all for it.

I can't fathom what the sender was doing with this request anyway.

Actions #9

Updated by Roel Standaert about 4 years ago

Yeah, if folding worked correctly the request still wouldn't be right. The value of the header would become Vary: Host Accept-Encoding, while this should actually be a comma-separated list: Vary: Host, Accept-Encoding.

Actions #10

Updated by Roel Standaert about 4 years ago

  • Status changed from New to Resolved

I pushed a commit removing support line folding in headers, so requests like this will get a 400 error instead of a crash.

Actions #11

Updated by Roel Standaert about 4 years ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF