Upload fails silently if upload file size > post_max_size setting in php.ini

(Dw Andrew) #1

Hi,
Got a weird error/issue on 4.6.4 - pretty sure it's in all versions. OS is RHEL 5.8



We first came across this in the EES, but after further testing it also happens in the admin interface so is a little lower level.



It seems that upload_max_filesize is 'ignored' when the file uploaded is larger than than the post_max_size, ie post_max_size is hit first.



Example:

in php.ini:

    upload_max_filesize = 20M
    
    post_max_size = 30M


Upload a file 45 MB

Output:
Nothing on admin interface

apache error_log:
    [Mon Apr 23 12:38:52 2012] [error] [client 0.0.0.0] PHP Warning:  POST Content-Length of 46500343 bytes exceeds the limit of 31457280 bytes in Unknown on line 0,



Ok, lets increase the post_max_size so it's approx upload filesize + php/program size

    upload_max_filesize = 20M

post_max_size = 60M</pre><br />




Upload a file 45 MB





Output:

Good error in admin interface:


    [2012-04-23 12:41:00][186003:andrew][512:mysource warning][R] (/core/include/locale_manager.inc:547) - The uploaded file "test2.tar.gz" exceeded the size specified by the 'upload_max_filesize' directive in php.ini (20M) [SYS0022]
    [2012-04-23 12:41:00][186003:andrew][512:mysource warning][R] (/core/include/locale_manager.inc:547) - Failed updating web path. Path already exists? [CORE0056]







Again, if you were to upload a file larger than post_max_size you wouldn't get an error apart form apache





I think this should mean that there needs to be an exception raised when the post_max_size is exceeded, or somehow the file size should be calculated first (and raise the upload_max_filesize error).  Otherwise, as shown the user gets no feedback when uploading files that are in this grey area - and it flows through to EES users as well.



Is anyone else able to replicate this?  Please let me know if any more details would help
(Edison Wang) #2

HI,


The post_max_size is something enforced by Apache. If user tries to upload a large file exceeding post_max_size, the HTTP request will be stopped by Apache, and error logged in Apache.

The request won't even go through to the PHP code, therefore there is little Matrix can do about it. Matrix won't even know user has submitted the form.



upload_max_filesize is controlled by PHP, so when user finished uploading process. The temporary file is already uploaded to the server, and PHP can tell if the file is over limit, and then Matrix can action on it.



I think it's always sensible to set POST limit to be larger than upload_max_filesize, so this can allow Matrix to report error.

(Dw Andrew) #3

Thanks Edison,




It looks like some people have a way around this, by checking the superglobals after a form submits

http://stackoverflow.com/questions/2133652/how-to-gracefully-handle-files-that-exceed-phps-post-max-size



I only bring this up as it has been reported by our end users

(Edison Wang) #4

Unfortunately, the workaround suggested by stackoverflow can't be easily applied to Matrix.


If we have to place a flag in the $_GET super variable, that means we have to append the flag to the URL of every form(both in front end and backend, including custom form, file assets and many more).

And even worse, we probably have to also append a bunch of other parameters to the URL as well(like assetid, formid, submit action), so Matrix knows which form has just been submitted, so Matrix can display the error properly in form, rather than just throw out a system warning message.



This is not a perfectly good workaround because

  • we simply can't append those parameters in URL, it could break client's existing implementation in frontend, revealing unnecessary information in URL.
  • another workaround is comparing $_SERVER['REQUEST_LENGTH'] with max POST limit, we have to do it for every POST request. That's a lot of overhead. Even worse, the REQUEST_LENGTH includes more data than the uploaded file, so it is not accurate anyway.





    The root cause of this issue is really about the PHP itself. It blanks out POST array whenever the uploaded file exceeds POST limit, without even reporting any error for it.

    It's very difficult for any PHP application to know what has happened, and if it's blank, it may be intentionally left blank. And if it's blanked out, we lost track of what is happening.



    I'm not really convinced with the workaround provided in the stackoverflow. I think we should leave it until PHP provides a proper fix for it.

    All we need here is just a better error reporting. But if we put in those hacks, it changed Matrix's behavior and over killing.



    I also have a feeling that, post_max_size should be always set larger than upload_file_size, because of the fact that post data contains other POST variables + uploaded file. It should be configured correctly in first place. It is not a common scenario that post_max_size is set to be smaller upload_file_size.



    I suggest putting a note in the Matrix's config screen, so that admin knows current post_max_size, and reminded that it should be set larger than max_file_upload limit.