Lua

AWS S3 File Streaming Upload

See more Amazon S3 (new) Examples

Demonstrates how to do a streaming upload from a file to the AWS S3 storage service. The AWS authorization presents some difficulties when the REST request body is to be streamed from a file (or from some other source). The issue is that the SHA-256 hash of the file data must be calculated. There are only two possible ways to do this: (1) stream the file into memory in its entirety and calculate the SHA-256 hash prior to uploading, or (2) pre-calculate the SHA-256 in a streaming fashion, and then provide it to the AWS authentication object.

If the application does NOT pre-compute the SHA-256, then Chilkat (internally) is forced to stream into memory, calculate the SHA-256, and then upload from the in-memory copy of the file.

Lua

local chilkat = require("chilkat")

-- This example requires the Chilkat API to have been previously unlocked.
-- See Global Unlock Sample for sample code.

-- This first part is optional.  AWS authentication requires
-- the SHA-256 hash of the request body (i.e. the contents of the file
-- to be uploaded).  We can choose to pre-calculate the SHA-256 in a streaming fashion
-- and then provide it to the authenticator object.  This way, if the file is
-- extremely large, it never needs to completely reside in memory.
local crypt = chilkat.Crypt2.new()
crypt.encoding_mode = "hex"
crypt.hash_algorithm = "sha-256"
local fileToUploadPath = "qa_data/xml/hamlet.xml"
local hashStr = crypt:hash_file_enc(fileToUploadPath)

local rest = chilkat.Rest.new()

-- Connect to the Amazon AWS REST server.
local bTls = true
local port = 443
local bAutoReconnect = true
local success = rest:connect("s3.amazonaws.com", port, bTls, bAutoReconnect)

-- ----------------------------------------------------------------------------
-- Important: For buckets created in regions outside us-east-1,
-- there are three important changes that need to be made.
-- See Working with S3 Buckets in Non-us-east-1 Regions for the details.
-- ----------------------------------------------------------------------------

-- Provide AWS credentials for the REST call.
local authAws = chilkat.AuthAws.new()
authAws.access_key = "AWS_ACCESS_KEY"
authAws.secret_key = "AWS_SECRET_KEY"
authAws.service_name = "s3"
-- Provide the pre-computed SHA-256 here:
authAws.precomputed_sha256 = hashStr

success = rest:set_auth_aws(authAws)

-- Set the bucket name via the HOST header.
-- In this case, the bucket name is "chilkat100".
rest.host = "chilkat100.s3.amazonaws.com"

local fileStream = chilkat.Stream.new()
fileStream.source_file = fileToUploadPath

-- Upload to the S3 Storage service.
-- If the application provided the SHA-256 hash of the file contents (as shown above)
-- then file is streamed and never has to completely reside in memory.
-- If the application did NOT provide the SHA-256, then Chilkat will (internally)
-- load the entire file into memory, calculate the SHA-256, and then upload.
local responseStr = rest:full_request_stream("PUT", "/hamlet.xml", fileStream)
if not rest.last_method_success then
    print(rest.last_error_text)
    return
end

-- When successful, the S3 Storage service will respond with a 200 response code,
-- with an XML body.
if rest.response_status_code == 200 then
    print(responseStr)
    print("File uploaded.")
else
    -- Examine the request/response to see what happened.
    print("response status code = " .. rest.response_status_code)
    print("response status text = " .. rest.response_status_text)
    print("response header: " .. rest.response_header)
    print("response body: " .. responseStr)
    print("---")
    print("LastRequestStartLine: " .. rest.last_request_start_line)
    print("LastRequestHeader: " .. rest.last_request_header)
end