Python Http Upload File and Wait for Return File
A common characteristic in web applications is to let users upload files to the server. The HTTP protocol documents the mechanism for a customer to upload a file in RFC 1867, and our favorite spider web framework Flask fully supports it, merely at that place are many implementation details that fall exterior of the formal specification that are unclear for many developers. Things such every bit where to store uploaded files, how to utilise them afterwards, or how to protect the server against malicious file uploads generate a lot of confusion and uncertainty.
In this commodity I'm going to show yous how to implement a robust file upload characteristic for your Flask server that is uniform with the standard file upload support in your web browser also equally the cool JavaScript-based upload widgets:
A Basic File Upload Form
From a high-level perspective, a client uploading a file is treated the aforementioned as whatever other class data submission. In other words, you lot have to ascertain an HTML course with a file field in information technology.
Here is a uncomplicated HTML page with a form that accepts a file:
<!doctype html> <html> <head> <title>File Upload</championship> </head> <torso> <h1>File Upload</h1> <form method="POST" action="" enctype="multipart/form-data"> <p><input blazon="file" name="file"></p> <p><input type="submit" value="Submit"></p> </course> </torso> </html> 
As y'all probably know, the method attribute of the <form> element tin can be Go or POST. With Go, the data is submitted in the query string of the asking URL, while with POST information technology goes in the request trunk. When files are being included in the grade, you must apply Postal service, as it would be incommunicable to submit file data in the query cord.
The enctype aspect in the <class> chemical element is normally non included with forms that don't take files. This attribute defines how the browser should format the data before it is submitted to the server. The HTML specification defines iii possible values for it:
-
application/10-world wide web-class-urlencoded: This is the default, and the all-time format for whatever forms except those that contain file fields. -
multipart/form-data: This format is required when at least one of the fields in the class is a file field. -
text/plain: This format has no practical utilize, so yous should ignore it.
The actual file field is the standard <input> chemical element that we use for well-nigh other form fields, with the blazon prepare to file. In the case to a higher place I haven't included whatsoever additional attributes, merely the file field supports 2 that are sometimes useful:
-
multipletin be used to allow multiple files to be uploaded in a unmarried file field. Example:
<input type="file" proper noun="file" multiple> -
havecan exist used to filter the allowed file types that can exist selected, either by file extension or past media blazon. Examples:
<input type="file" proper noun="doc_file" accept=".doc,.docx"> <input type="file" name="image_file" accept="image/*"> Accepting File Submissions with Flask
For regular forms, Flask provides access to submitted form fields in the request.class dictionary. File fields, however, are included in the request.files lexicon. The request.grade and asking.files dictionaries are actually "multi-dicts", a specialized lexicon implementation that supports duplicate keys. This is necessary because forms tin include multiple fields with the same name, equally is often the case with groups of check boxes. This also happens with file fields that allow multiple files.
Ignoring important aspects such as validation and security for the moment, the short Flask application shown below accepts a file uploaded with the class shown in the previous department, and writes the submitted file to the electric current directory:
from flask import Flask, render_template, request, redirect, url_for app = Flask(__name__) @app.route('/') def alphabetize(): return render_template('index.html') @app.road('/', methods=['Post']) def upload_file(): uploaded_file = asking.files['file'] if uploaded_file.filename != '': uploaded_file.salvage(uploaded_file.filename) return redirect(url_for('index')) The upload_file() function is decorated with @app.road so that it is invoked when the browser sends a POST asking. Note how the same root URL is dissever betwixt two view functions, with index() set to accept the Go requests and upload_file() the Mail service ones.
The uploaded_file variable holds the submitted file object. This is an instance of class FileStorage, which Flask imports from Werkzeug.
The filename aspect in the FileStorage provides the filename submitted by the client. If the user submits the form without selecting a file in the file field, then the filename is going to be an empty string, so it is important to always check the filename to determine if a file is bachelor or non.
When Flask receives a file submission it does not automatically write it to disk. This is actually a good affair, because it gives the awarding the opportunity to review and validate the file submission, equally you will see later. The actual file data can be accessed from the stream aspect. If the awarding just wants to save the file to disk, then information technology tin can call the salve() method, passing the desired path as an argument. If the file's save() method is not called, then the file is discarded.
Want to examination file uploads with this application? Make a directory for your application and write the code higher up every bit app.py. Then create a templates subdirectory, and write the HTML folio from the previous section equally templates/index.html. Create a virtual surround and install Flask on it, and so run the application with flask run. Every time you submit a file, the server will write a copy of information technology in the current directory.
Before I move on to the topic of security, I'thou going to talk over a few variations on the code shown above that you may find useful. Every bit I mentioned earlier, the file upload field can be configured to take multiple files. If yous apply request.files['file'] as above you will get simply one of the submitted files, but with the getlist() method you can access all of them in a for-loop:
for uploaded_file in request.files.getlist('file'): if uploaded_file.filename != '': uploaded_file.save(uploaded_file.filename) Many people code their form handling routes in Flask using a unmarried view role for both the Get and POST requests. A version of the example application using a single view part could exist coded every bit follows:
@app.route('/', methods=['Go', 'POST']) def index(): if request.method == 'Mail service': uploaded_file = request.files['file'] if uploaded_file.filename != '': uploaded_file.save(uploaded_file.filename) return redirect(url_for('index')) return render_template('alphabetize.html') Finally, if you use the Flask-WTF extension to handle your forms, yous can utilize the FileField object for your file uploads. The class used in the examples you've seen so far can be written using Flask-WTF as follows:
from flask_wtf import FlaskForm from flask_wtf.file import FileField from wtforms import SubmitField grade MyForm(FlaskForm): file = FileField('File') submit = SubmitField('Submit') Note that the FileField object comes from the flask_wtf parcel, unlike most other field classes, which are imported directly from the wtforms package. Flask-WTF provides 2 validators for file fields, FileRequired, which performs a check similar to the empty string check, and FileAllowed, which ensures the file extension is included in an allowed extensions list.
When you use a Flask-WTF form, the data aspect of the file field object points to the FileStorage instance, so saving a file to disk works in the same mode as in the examples above.
Securing file uploads
The file upload example presented in the previous department is an extremely simplistic implementation that is not very robust. One of the about important rules in spider web evolution is that data submitted past clients should never be trusted, and for that reason when working with regular forms, an extension such as Flask-WTF performs strict validation of all fields before the class is accepted and the data incorporated into the application. For forms that include file fields there needs to be validation likewise, because without file validation the server leaves the door open up to attacks. For example:
- An attacker can upload a file that is so large that the deejay space in the server is completely filled, causing the server to malfunction.
- An attacker can craft an upload request that uses a filename such every bit ../../../.bashrc or similar, with the attempt to trick the server into rewriting system configuration files.
- An attacker tin can upload files with viruses or other types of malware in a place where the application, for example, expects images.
Limiting the size of uploaded files
To prevent clients from uploading very big files, you can use a configuration option provided by Flask. The MAX_CONTENT_LENGTH option controls the maximum size a request body tin accept. While this isn't an option that is specific to file uploads, setting a maximum request body size effectively makes Flask discard any incoming requests that are larger than the allowed corporeality with a 413 status code.
Let'due south change the app.py example from the previous section to only accept requests that are upwardly to 1MB in size:
app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 If you attempt to upload a file that is larger than 1MB, the application will at present refuse it.
Validating filenames
We can't really trust that the filenames provided past the client are valid and safe to use, and so filenames coming with uploaded files have to be validated.
A very unproblematic validation to perform is to make sure that the file extension is one that the application is willing to accept, which is similar to what the FileAllowed validator does when using Flask-WTF. Permit's say the application accepts images, then it can configure the list of approved file extensions:
app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] For every uploaded file, the application tin can make sure that the file extension is one of the allowed ones:
filename = uploaded_file.filename if filename != '': file_ext = os.path.splitext(filename)[one] if file_ext not in current_app.config['UPLOAD_EXTENSIONS']: abort(400) With this logic, any filenames that do not have one of the approved file extensions is going to be responded with a 400 mistake.
In addition to the file extension, it is besides important to validate the filename, and whatever path given with it. If your awarding does not care virtually the filename provided by the client, the most secure manner to handle the upload is to ignore the client provided filename and generate your own filename instead, that you pass to the save() method. An case use case where this technique works well is with avatar image uploads. Each user's avatar can exist saved with the user id every bit filename, and so the filename provided by the client can be discarded. If your application uses Flask-Login, yous could implement the post-obit save() call:
uploaded_file.save(os.path.bring together('static/avatars', current_user.get_id())) In other cases it may be improve to preserve the filenames provided by the client, so the filename must be sanitized outset. For those cases Werkzeug provides the secure_filename() function. Permit's see how this function works by running a few tests in a Python session:
>>> from werkzeug.utils import secure_filename >>> secure_filename('foo.jpg') 'foo.jpg' >>> secure_filename('/some/path/foo.jpg') 'some_path_foo.jpg' >>> secure_filename('../../../.bashrc') 'bashrc' As y'all see in the examples, no matter how complicated or malicious the filename is, the secure_filename() function reduces it to a apartment filename.
Allow's incorporate secure_filename() into the case upload server, and likewise add together a configuration variable that defines a defended location for file uploads. Here is the complete app.py source file with secure filenames:
import os from flask import Flask, render_template, request, redirect, url_for, abort from werkzeug.utils import secure_filename app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads' @app.route('/') def alphabetize(): return render_template('index.html') @app.route('/', methods=['POST']) def upload_files(): uploaded_file = asking.files['file'] filename = secure_filename(uploaded_file.filename) if filename != '': file_ext = os.path.splitext(filename)[ane] if file_ext not in app.config['UPLOAD_EXTENSIONS']: abort(400) uploaded_file.save(os.path.join(app.config['UPLOAD_PATH'], filename)) render redirect(url_for('index')) Validating file contents
The third layer of validation that I'm going to discuss is the most complex. If your application accepts uploads of a certain file blazon, it should ideally perform some class of content validation and reject any files that are of a different type.
How you achieve content validation largely depends on the file types your application accepts. For the example application in this article I'yard using images, so I can use the imghdr package from the Python standard library to validate that the header of the file is, in fact, an epitome.
Let's write a validate_image() function that performs content validation on images:
import imghdr def validate_image(stream): header = stream.read(512) stream.seek(0) format = imghdr.what(None, header) if not format: return None return '.' + (format if format != 'jpeg' else 'jpg') This function takes a byte stream every bit an statement. Information technology starts by reading 512 bytes from the stream, and and then resetting the stream pointer back, because later when the save() function is chosen we desire information technology to run into the entire stream. The first 512 bytes of the image data are going to be sufficient to identify the format of the image.
The imghdr.what() function can look at a file stored on disk if the start argument is the filename, or else information technology tin look at data stored in memory if the first argument is None and the information is passed in the second argument. The FileStorage object gives us a stream, so the near convenient option is to read a safe amount of information from it and laissez passer it as a byte sequence in the second statement.
The render value of imghdr.what() is the detected prototype format. The function supports a diverseness of formats, among them the popular jpeg, png and gif. If not known paradigm format is detected, then the render value is None. If a format is detected, the name of the format is returned. The well-nigh convenient is to return the format as a file extension, because the application can and then ensure that the detected extension matches the file extension, so the validate_image() role converts the detected format into a file extension. This is as uncomplicated as adding a dot as prefix for all image formats except jpeg, which normally uses the .jpg extension, so this case is treated equally an exception.
Here is the complete app.py, with all the features from the previous sections plus content validation:
import imghdr import bone from flask import Flask, render_template, request, redirect, url_for, arrest from werkzeug.utils import secure_filename app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads' def validate_image(stream): header = stream.read(512) stream.seek(0) format = imghdr.what(None, header) if non format: render None return '.' + (format if format != 'jpeg' else 'jpg') @app.route('/') def index(): return render_template('alphabetize.html') @app.route('/', methods=['POST']) def upload_files(): uploaded_file = request.files['file'] filename = secure_filename(uploaded_file.filename) if filename != '': file_ext = os.path.splitext(filename)[one] if file_ext non in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): arrest(400) uploaded_file.salve(os.path.bring together(app.config['UPLOAD_PATH'], filename)) return redirect(url_for('index')) The merely alter in the view function to contain this last validation logic is here:
if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): abort(400) This expanded bank check first makes sure that the file extension is in the allowed listing, and then ensures that the detected file extension from looking at the data stream is the same as the file extension.
Earlier you test this version of the awarding create a directory named uploads (or the path that you defined in the UPLOAD_PATH configuration variable, if different) then that files tin be saved there.
Using Uploaded Files
You lot now know how to handle file uploads. For some applications this is all that is needed, equally the files are used for some internal process. But for a large number of applications, in detail those with social features such equally avatars, the files that are uploaded by users have to be integrated with the application. Using the example of avatars, once a user uploads their avatar epitome, any mention of the username requires the uploaded image to announced to the side.
I carve up file uploads into 2 large groups, depending on whether the files uploaded by users are intended for public utilise, or they are private to each user. The avatar images discussed several times in this article are clearly in the get-go grouping, as these avatars are intended to be publicly shared with other users. On the other side, an awarding that performs editing operations on uploaded images would probably exist in the 2d group, considering y'all'd want each user to only have access to their own images.
Consuming public uploads
When images are of a public nature, the easiest way to brand the images available for employ by the awarding is to put the upload directory within the application's static folder. For example, an avatars subdirectory can be created inside static, so avatar images tin can exist saved in that location using the user id equally name.
Referencing these uploads stored in a subdirectory of the static binder is washed in the aforementioned style equally regular static files of the awarding, using the url_for() office. I previously suggested using the user id equally a filename, when saving an uploaded avatar image. This was the way the images were saved:
uploaded_file.salvage(os.path.bring together('static/avatars', current_user.get_id())) With this implementation, given a user_id, the URL for the user's avatar tin be generated as follows:
url_for('static', filename='avatars/' + str(user_id)) Alternatively, the uploads can be saved to a directory outside of the static folder, and then a new route can be added to serve them. In the example app.py application file uploads are saved to the location set in the UPLOAD_PATH configuration variable. To serve these files from that location, we can implement the following route:
from flask import send_from_directory @app.route('/uploads/<filename>') def upload(filename): render send_from_directory(app.config['UPLOAD_PATH'], filename) One advantage that this solution has over storing uploads inside the static folder is that hither you tin implement additional restrictions before these files are returned, either directly with Python logic inside the body of the role, or with decorators. For example, if you lot want to only provide admission to the uploads to logged in users, you can add Flask-Login'due south @login_required decorator to this road, or whatever other authentication or function checking mechanism that you lot use for your normal routes.
Permit'due south use this implementation idea to show uploaded files in our case application. Hither is a new complete version of app.py:
import imghdr import os from flask import Flask, render_template, request, redirect, url_for, abort, \ send_from_directory from werkzeug.utils import secure_filename app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads' def validate_image(stream): header = stream.read(512) # 512 bytes should exist plenty for a header check stream.seek(0) # reset stream pointer format = imghdr.what(None, header) if non format: return None render '.' + (format if format != 'jpeg' else 'jpg') @app.route('/') def index(): files = os.listdir(app.config['UPLOAD_PATH']) render render_template('index.html', files=files) @app.route('/', methods=['Mail']) def upload_files(): uploaded_file = request.files['file'] filename = secure_filename(uploaded_file.filename) if filename != '': file_ext = os.path.splitext(filename)[1] if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): arrest(400) uploaded_file.save(os.path.join(app.config['UPLOAD_PATH'], filename)) return redirect(url_for('index')) @app.road('/uploads/<filename>') def upload(filename): return send_from_directory(app.config['UPLOAD_PATH'], filename) In addition to the new upload() function, the index() view function gets the list of files in the upload location using os.listdir() and sends information technology downwardly to the template for rendering. The index.html template updated to prove uploads is shown beneath:
<!doctype html> <html> <head> <championship>File Upload</championship> </head> <body> <h1>File Upload</h1> <form method="POST" action="" enctype="multipart/course-data"> <p><input blazon="file" proper noun="file"></p> <p><input type="submit" value="Submit"></p> </class> <hr> {% for file in files %} <img src="{{ url_for('upload', filename=file) }}" fashion="width: 64px"> {% endfor %} </body> </html> With these changes, every time you upload an paradigm, a thumbnail is added at the bottom of the page:
Consuming private uploads
When users upload private files to the application, additional checks need to exist in place to prevent sharing files from one user with unauthorized parties. The solution for these cases require variations of the upload() view function shown above, with additional access checks.
A mutual requirement is to only share uploaded files with their owner. A convenient way to shop uploads when this requirement is present is to use a separate directory for each user. For example, uploads for a given user tin can be saved to the uploads/<user_id> directory, and then the uploads() function can be modified to only serve uploads from the user'southward own upload directory, making it impossible for ane user to run into files from another. Below yous can see a possible implementation of this technique, once again assuming Flask-Login is used:
@app.road('/uploads/<filename>') @login_required def upload(filename): render send_from_directory(os.path.bring together( app.config['UPLOAD_PATH'], current_user.get_id()), filename) Showing upload progress
Upward until now nosotros have relied on the native file upload widget provided by the web browser to initiate our file uploads. I'chiliad certain nosotros can all concord that this widget is not very appealing. Non only that, but the lack of an upload progress display makes it unusable for uploads of big files, equally the user receives no feedback during the entire upload procedure. While the scope of this article is to comprehend the server side, I idea it would be useful to requite you a few ideas on how to implement a modern JavaScript-based file upload widget that displays upload progress.
The adept news is that on the server at that place aren't any big changes needed, the upload machinery works in the aforementioned mode regardless of what method you use in the browser to initiate the upload. To evidence you an example implementation I'm going to replace the HTML form in index.html with 1 that is compatible with dropzone.js, a popular file upload client.
Here is a new version of templates/index.html that loads the dropzone CSS and JavaScript files from a CDN, and implements an upload class according to the dropzone documentation:
<html> <head> <title>File Upload</title> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.7.1/min/dropzone.min.css"> </head> <body> <h1>File Upload</h1> <grade action="{{ url_for('upload_files') }}" class="dropzone"> </form> <script src="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.seven.one/min/dropzone.min.js"></script> </body> </html> The one interesting thing that I've plant when implementing dropzone is that it requires the activity attribute in the <form> chemical element to be set, even though normal forms accept an empty activity to indicate that the submission goes to the same URL.
Start the server with this new version of the template, and this is what you'll get:
That's basically it! Yous can now drop files and they'll be uploaded to the server with a progress bar and a final indication of success or failure.
If the file upload fails, either due to the file being too large or invalid, dropzone wants to display an error message. Because our server is currently returning the standard Flask error pages for the 413 and 400 errors, you will see some HTML gibberish in the error popup. To correct this we can update the server to return its error responses as text.
The 413 mistake for the file too big status is generated by Flask when the asking payload is bigger than the size set in the configuration. To override the default error page we have to use the app.errorhandler decorator:
@app.errorhandler(413) def too_large(e): render "File is too large", 413 The second error condition is generated past the application when any of the validation checks fails. In this case the mistake was generated with a abort(400) phone call. Instead of that the response tin be generated directly:
if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): return "Invalid prototype", 400 The terminal change that I'k going to make isn't really necessary, just information technology saves a bit of bandwidth. For a successful upload the server returned a redirect() back to the main route. This caused the upload form to be displayed over again, and also to refresh the list of upload thumbnails at the bottom of the folio. None of that is necessary now because the uploads are done equally background requests past dropzone, and then we can eliminate that redirect and switch to an empty response with a lawmaking 204.
Here is the consummate and updated version of app.py designed to work with dropzone.js:
import imghdr import os from flask import Flask, render_template, request, redirect, url_for, abort, \ send_from_directory from werkzeug.utils import secure_filename app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 2 * 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads' def validate_image(stream): header = stream.read(512) stream.seek(0) format = imghdr.what(None, header) if non format: return None render '.' + (format if format != 'jpeg' else 'jpg') @app.errorhandler(413) def too_large(e): return "File is also big", 413 @app.route('/') def index(): files = bone.listdir(app.config['UPLOAD_PATH']) return render_template('index.html', files=files) @app.route('/', methods=['POST']) def upload_files(): uploaded_file = request.files['file'] filename = secure_filename(uploaded_file.filename) if filename != '': file_ext = os.path.splitext(filename)[1] if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): return "Invalid image", 400 uploaded_file.salvage(os.path.join(app.config['UPLOAD_PATH'], filename)) render '', 204 @app.route('/uploads/<filename>') def upload(filename): return send_from_directory(app.config['UPLOAD_PATH'], filename) Restart the application with this update and now errors will accept a proper message:
The dropzone.js library is very flexible and has many options for customization, so I encourage you to visit their documentation to learn how to adapt it to your needs. You can also look for other JavaScript file upload libraries, as they all follow the HTTP standard, which means that your Flask server is going to work well with all of them.
Decision
This was a long overdue topic for me, I tin't believe I have never written anything on file uploads! I'd love you lot hear what you lot recollect about this topic, and if you think there are aspects of this feature that I haven't covered in this article. Feel free to let me know below in the comments!
winchesterhishoupers.blogspot.com
Source: https://blog.miguelgrinberg.com/post/handling-file-uploads-with-flask
0 Response to "Python Http Upload File and Wait for Return File"
Post a Comment