Can You Upload Files to Api Gateway Amazon

Why practise we need a File Uploader?

When y'all are working equally an engineer for a software development visitor, information technology is credible that you accept to troubleshoot problems reported past the users of your software. In such a instance, by and large, y'all demand to get various types of files from your customer for these troubleshooting purposes. For case, you lot may request them to send the application log files to troubleshoot a certain issue they are facing in their production surroundings. Or they may be facing a UI related effect and need to share some screenshots with you.

In this kind of a scenario, an HTTP based file uploader comes handy. The master functionalities of such a file uploader are as follows.

• Betrayal an HTTP/S endpoint (preferably PUT or POST) then the customers can invoke that endpoint with the file they want to share with you
• Receive the file from the HTTP request and shop it in a secured file storage

In addition to the above chief functionalities, the following can also exist added as useful features of such a file uploader.

• Provide a web interface to make the file upload request to the in a higher place HTTP endpoint
• Send notification to the engineer that a file has been uploaded

What are nosotros going to create?

In this article, we are going to implement a serverless file uploader with the above-mentioned chief functionalities, utilizing AWS provided services such as API Gateway, Lambda, and S3. For the development and deployment of the solution, we are going to use SLAppForge Sigma, which is a purposely-built IDE for serverless applications.

If you don't have a Sigma business relationship, you can sign-up for a gratuitous account. After signing and confirming your e-mail, please follow this getting started guide to gear up the residue of your account.

File Uploader Architecture

Let'due south implement our application

ane. Create a new projection

Once your Sigma account is set upwards, let's commencement create a new Sigma projection for our application. For that, provide a projection name (east.g.: ServerlessFileUploader) and a version as you desired, and optionally you tin can provide a projection clarification likewise. Since we are going to deploy this awarding on AWS, keep the Base of operations Platform as AWS and select whatsoever AWS region for the deployment.

In this article, we are going to implement the application using NodeJS. So allow'due south choose NodeJS as the primary language. So click on the Create Project button to go into the editor.

Projection Creation

Once the editor is opened, yous can see that a Lambda file with your project name has been already added to the project with a basic code skeleton.

2. Add an HTTP API Gateway trigger

Our first task is to define an API Gateway and set it every bit a trigger for this Lambda function. AWS provides 3 flavors of their API Gateway as REST APIs, HTTP APIs, and WebSocket APIsouth. For our purpose, we can utilize either a Balance or an HTTP API. In this article, we are going to use an HTTP API, which is recently introduced by AWS. In our context, using an HTTP API has the following advantages over a REST API.

• Low Cost - HTTP API pricing starts at $1.00 per 1000000 requests, which is nearly 70% cheaper than a REST API.
• Low Latency - The latency associated with an HTTP API is almost 50% of the latency of a similar REST API, due to the feature simplicity.
• Better handling of Binary Payloads - HTTP APIs handle binary payloads (such as image files) without setting upward any circuitous custom configurations

To set an API Gateway as Lambda trigger, click on the API Gateway block on the left side Resource panel and elevate-drop information technology on to the event variable of the Lambda handler. It will open the API Gateway configuration panel on the right side of the editor.

HTTP API Gateway Configuration

On the configuration panel, select API Type every bit HTTP API and provide a suitable proper noun for the API (eastward.g.: FileUploaderAPI). For the deployment stage, nosotros can either utilize the default stage or ascertain a custom stage. To define a custom stage, check the Custom Deployment Stage checkbox so provide a valid name (e.m: examination) for the stage name. (This phase name can only comprise alpha-numeric characters or the underscore character.)

And so we are going to apply PUT equally the HTTP method and /upload every bit the road for accepting file upload requests.

One time all these fields have been configured correctly, click on the Inject push to set the API Gateway as the trigger for the Lambda function. Now you will see that the trigger icon in front of the Lambda handler has been turned greenish. Yous can click information technology to practice any changes to the API trigger if required.

3. Extracting the file content from the asking

When an HTTP API Gateway is set as a trigger for a Lambda function, the Lambda outcome variable contains the HTTP request details such equally headers, path parameters, query parameters, and the payload. A sample such event tin can be found below.

https://gist.github.com/Udith/e0487ff91842f693f5f51a64f11e6142

If the request is a file upload, the file content is sent in the body field of the upshot. If the file content is a text format such as CSV, this raw content is sent in the above field. Else, if the file content is a binary type such equally an paradigm, that content is sent in Base64 encoded format. In the latter case, the isBase64Encoded field of the upshot is ready to true as well.

Therefore our next stride is to excerpt this file content from the issue object. Since nosotros demand to back up both binary and text-formatted files, we have to identify the Base64 encoded payloads and decode them appropriately.

Nosotros can implement this functionality as follows.

let fileContent = upshot.isBase64Encoded ? Buffer.from(event.body, 'base64') : event.torso;

4. Generating the file proper name

Our next job is to generate a name for the file to be stored in S3. For the file name part, we are going to use the current epoch timestamp in milliseconds, which makes information technology easier to identify a file uploaded at a given time. We can implement this functionality as follows.

permit fileName = `${Engagement.now()}`;

Then the complex task is to decide the extension of the file. For that, i pick is to enquire the file uploader to include an HTTP header with the original file name so that nosotros tin extract the extension from that name. The other option is to decide the file extension based on the content-blazon header of the HTTP request. For that approach, we can use an NPM library such as mime-types without implementing it from scratch.

1 of the useful features of Sigma IDE is the capability to search and add NPM dependencies to the project from IDE itself. For that click on the Add Dependencies button on the toolbar and search for the in a higher place library. Once you constitute the library on the search results listing, click on the Add button to add information technology equally a project dependency.

Calculation NPM dependency

And so nosotros can determine the file extension as follows.

let contentType = event.headers['content-type'] || event.headers['Content-Blazon']; let extension = contentType ? mime.extension(contentType) : '';

The full file proper noun can be generated past combining the to a higher place ii parts every bit follows.

permit fullFileName = extension ? `${fileName}.${extension}` : fileName;        

5. Uploading the file content to S3

The last task of our implementation is to define an S3 saucepan and implement the code to store these files into that bucket. For that drag-n-drop the S3 block from the resource console onto the lawmaking editor line where we demand to put our upload lawmaking snippet. Similar to API Gateway, this will too open up the S3 configuration panel.

S3 Saucepan Configuration

Since we are going to create a new bucket to store the uploaded files, go to the New Saucepan tab and provide a name for the bucket and select the performance equally Put Object.

Then for the content of the object field, nosotros need to provide the fileContent variable we defined earlier. Since Sigma supports providing variables in the configuration fields with the syntax @{varname} , we can set the value for the field equally @{fileContent} . Similarly, we can set the value for the Object Key field as @{fullFileName} .

Once these values are set, click on the Inject button to generate the code snippet and inject information technology into the editor.
In the successful file upload scenario, we are going to print a logline and so return the bulletin "Successfully uploaded". In that instance, the user who uploaded the file will receive an HTTP 200 response with this message every bit the response. In the failure scenario, nosotros are going to log the mistake message and and then throw the error, and then that the user volition receive an HTTP 500 response indicating an mistake.

attempt {     let data = look s3.putObject({         Saucepan: "file-upload-bucket",         Fundamental: fullFileName,         Torso: fileContent,         Metadata: {}     }).hope();      panel.log("Successfully uploaded file", fullFileName);     return "Successfully uploaded";  } catch (err) {     console.log("Failed to upload file", fullFileName, err);     throw err; };

The full implementation of the Lambda is as follows.

half dozen. Saving and Deploying the project

Since at present our projection implementation is complete, nosotros can save information technology into a Git repository and deploy it into our AWS account.

To save the projection, click on the Salve button on the toolbar. Then if you haven't already integrated Sigma with a VCS provider, you can choose either GitHub, BitBucket, or GitLab to integrate with. Once successfully integrated, you can commit your changes to a repository with the desired proper name. In example yous are a premium user, you lot can commit to a private repository too.

Sigma provides the capability to deploy your project even without saving it into a VCS repository. Simply information technology is recommended to salvage so deploy as a best practice.

After committing the changes, click on the Deploy push button to deploy your application to the AWS business relationship. Sigma will automatically build your project and then display a list of actions that are to exist performed on your AWS account during the deployment. The changeset volition look similar to below.

Deployment Changes

In summary, the following are the entities that will be added into your AWS business relationship equally per this changeset.

• ServerlessFileUploader Lambda function
• IAM role to be used equally the execution role of the in a higher place Lambda function. (Sigma should have automatically added the necessary permissions to for the S3 upload)
• An HTTP API
• A custom deployment phase for the above API
• A road for the to a higher place API for PUT method on /upload path
• An integration between the to a higher place route and the Lambda function
• Permissions for the higher up API to invoke Lambda part
• An S3 bucket

Since these changes look fine, we can click on the Execute button to start the deployment. Once the deployment is completed successfully, you will become an output similar to the following, containing the URL of the API endpoint and the Lambda ARN.

Deployment Output

You can use an HTTP client such equally Postman to brand a PUT request to this endpoint with a file fastened as the payload. Both textual files such as CSV and binary files such as image files should piece of work without any difference.

File Upload Asking with Postman

One time the file upload is successful, you can log into your AWS S3 console and cheque the newly created bucket for the uploaded files.

Uploaded files in S3 Saucepan

Now nosotros take a fully functioning serverless file uploader, that we can share with our clients to share their files.

parkeranianded.blogspot.com

Source: https://www.slappforge.com/blog/serverless-file-uploader-with-aws-http-api-gateway-and-s3

Share this post