Project

General

Profile

Possible Wt::Http::Request::uploadedFiles() bug or am I missing something?

Added by George McFie 12 months ago

Context:

Wt::Wresource::handleRequest (const Wt::Http::Request &request, Wt::Http::Response &response)
{
. . .
const Wt::Http::UploadedFileMap& files = request.uploadedFiles();
. . .
}

The issue:

When handling inbound HTTP POST messages I have the problem that request.uploadedFiles() will return nullptr for some POSTed files but not for others – even though a Wireshark trace reveals that the HTTP messages are structured in exactly the same way. In point of fact, the sender for all files uses exactly the same CURL-based code.

Below are extracts from the same Wireshark trace .

The first file is not handled correctly (i.e. request.uploadedFiles() returns nullptr), while the next one is handled correctly.

This is NOT handled correctly in Wt

Frame 309: 15100 bytes on wire (120800 bits), 15100 bytes captured (120800 bits)
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 50926, Dst Port: 8180, Seq: 6226266, Ack: 1, Len: 15034
[194 Reassembled TCP Segments (6241299 bytes): #14(345), #16(32741), #18(32741), #20(54), #22(32741), #23(32741), #24(54), #27(32768), #29(32768), #31(65483), #33(53), #34(65483), #36(53), #37(65483), #39(53), #40(65483), #42(53), #43(65483]
Hypertext Transfer Protocol
POST /lfr/put?filename=LE3188_230502_100730_000042.mp4&source=LC4&mime=video%2fmp4&transaction=4fe7b720-4a5f-4f28-9bb5-b2af7c34a042 HTTP/1.1\r\n
Host: 127.0.0.1:8180\r\n
User-Agent: VMS\r\n
Accept: /\r\n
Satellite: LC4\r\n
Content-Length: 6240954\r\n
Content-Type: multipart/form-data; boundary=------------------------6fea749b4e74880d\r\n
Expect: 100-continue\r\n
\r\n
[Full request URI: http://127.0.0.1:8180/lfr/put?filename=LE3188_230502_100730_000042.mp4&source=LC4&mime=video%2fmp4&transaction=4fe7b720-4a5f-4f28-9bb5-b2af7c34a042]
[HTTP request 1/1]
[Response in frame: 311]
File Data: 6240954 bytes
MIME Multipart Media Encapsulation, Type: multipart/form-data, Boundary: "------------------------6fea749b4e74880d"
[Type: multipart/form-data]
First boundary: --------------------------6fea749b4e74880d\r\n
Encapsulated multipart part: (application/octet-stream)
Content-Disposition: form-data; name="LE3188_230502_100730_000042.mp4"; filename="LE3188_230502_100730_000042.mp4"\r\n
Content-Type: application/octet-stream\r\n\r\n
Data (6240704 bytes)
Last boundary: \r\n--------------------------6fea749b4e74880d--\r\n

This is handled correctly

Frame 321: 1044 bytes on wire (8352 bits), 1044 bytes captured (8352 bits)
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 50940, Dst Port: 8180, Seq: 313, Ack: 1, Len: 978
[2 Reassembled TCP Segments (1290 bytes): #319(312), #321(978)]
Hypertext Transfer Protocol
POST /lfr/put?filename=VideoDetails.json&source=LC4&mime=application%2fjson&transaction=4fe7b720-4a5f-4f28-9bb5-b2af7c34a042 HTTP/1.1\r\n
Host: 127.0.0.1:8180\r\n
User-Agent: VMS\r\n
Accept: /\r\n
Satellite: LC4\r\n
Content-Length: 978\r\n
Content-Type: multipart/form-data; boundary=------------------------f9f9890042ec6cca\r\n
\r\n
[Full request URI: http://127.0.0.1:8180/lfr/put?filename=VideoDetails.json&source=LC4&mime=application%2fjson&transaction=4fe7b720-4a5f-4f28-9bb5-b2af7c34a042]
[HTTP request 1/1]
[Response in frame: 323]
File Data: 978 bytes
MIME Multipart Media Encapsulation, Type: multipart/form-data, Boundary: "------------------------f9f9890042ec6cca"
[Type: multipart/form-data]
First boundary: --------------------------f9f9890042ec6cca\r\n
Encapsulated multipart part: (application/octet-stream)
Content-Disposition: form-data; name="VideoDetails.json"; filename="VideoDetails.json"\r\n
Content-Type: application/octet-stream\r\n\r\n
Data (756 bytes)
Last boundary: \r\n--------------------------f9f9890042ec6cca--\r\n


Replies (3)

RE: Possible Wt::Http::Request::uploadedFiles() bug or am I missing something? - Added by Christian Meyer 12 months ago

Hi

Did you check the wt_config file about the upload limits?

As the working one is small and the failing one is big, this might be the issue =)

Good Luck

RE: Possible Wt::Http::Request::uploadedFiles() bug or am I missing something? - Added by George McFie 12 months ago

Thanks for the feedback.

Thanks. Yes, I had already checked that.

I should point out that I do actually get the messages, but I've had to implement a hack work-around to extract them out of the HTTP body - separating them out from the MIME part headers and bracketing boundary markers.

But, the message size does seem to be a factor.

Strangely, it works fine on my dev machine (Linux Mint 20.3 + Wt4.1.1) but not on other production Ubuntu machines (separate builds but with the same version of Wt).

Even stranger, very similar code in another of my servers works perfectly well for the same large messages sent in the same way (as attachments).

This does increasingly look like a subtle Wt bug.

RE: Possible Wt::Http::Request::uploadedFiles() bug or am I missing something? - Added by Christian Meyer 12 months ago

Hello again

I have nothing new for your specific problem, but wanted to include that I just ran into the size limitation of the upload when handling a JSignal...

It would be amazing to get a warning in debug mode that a request has been tried that exceeded the respective limit

I did put up a Ticket for it: [[[https://redmine.emweb.be/issues/11593]]]

    (1-3/3)