Perl

AWS S3 File Streaming Upload

See more Amazon S3 (new) Examples

Demonstrates how to do a streaming upload from a file to the AWS S3 storage service. The AWS authorization presents some difficulties when the REST request body is to be streamed from a file (or from some other source). The issue is that the SHA-256 hash of the file data must be calculated. There are only two possible ways to do this: (1) stream the file into memory in its entirety and calculate the SHA-256 hash prior to uploading, or (2) pre-calculate the SHA-256 in a streaming fashion, and then provide it to the AWS authentication object.

If the application does NOT pre-compute the SHA-256, then Chilkat (internally) is forced to stream into memory, calculate the SHA-256, and then upload from the in-memory copy of the file.

Chilkat2 Perl

use strict;
use warnings;
use Chilkat2;

# This example requires the Chilkat API to have been previously unlocked.
# See Global Unlock Sample for sample code.

# This first part is optional.  AWS authentication requires
# the SHA-256 hash of the request body (i.e. the contents of the file
# to be uploaded).  We can choose to pre-calculate the SHA-256 in a streaming fashion
# and then provide it to the authenticator object.  This way, if the file is
# extremely large, it never needs to completely reside in memory.
my $crypt = Chilkat2::Crypt2->new;
$crypt->EncodingMode('hex');
$crypt->HashAlgorithm('sha-256');
my $fileToUploadPath = 'qa_data/xml/hamlet.xml';
my $hashStr = $crypt->HashFileENC($fileToUploadPath);

my $rest = Chilkat2::Rest->new;

# Connect to the Amazon AWS REST server.
my $bTls = 1;
my $port = 443;
my $bAutoReconnect = 1;
my $success = $rest->Connect('s3.amazonaws.com', $port, $bTls, $bAutoReconnect);

# ----------------------------------------------------------------------------
# Important: For buckets created in regions outside us-east-1,
# there are three important changes that need to be made.
# See Working with S3 Buckets in Non-us-east-1 Regions for the details.
# ----------------------------------------------------------------------------

# Provide AWS credentials for the REST call.
my $authAws = Chilkat2::AuthAws->new;
$authAws->AccessKey('AWS_ACCESS_KEY');
$authAws->SecretKey('AWS_SECRET_KEY');
$authAws->ServiceName('s3');
# Provide the pre-computed SHA-256 here:
$authAws->PrecomputedSha256($hashStr);

$success = $rest->SetAuthAws($authAws);

# Set the bucket name via the HOST header.
# In this case, the bucket name is "chilkat100".
$rest->Host('chilkat100.s3.amazonaws.com');

my $fileStream = Chilkat2::Stream->new;
$fileStream->SourceFile($fileToUploadPath);

# Upload to the S3 Storage service.
# If the application provided the SHA-256 hash of the file contents (as shown above)
# then file is streamed and never has to completely reside in memory.
# If the application did NOT provide the SHA-256, then Chilkat will (internally)
# load the entire file into memory, calculate the SHA-256, and then upload.
my $responseStr = $rest->FullRequestStream('PUT', '/hamlet.xml', $fileStream);
if (!$rest->LastMethodSuccess) {
    print $rest->LastErrorText, "\n";
    exit;
}

# When successful, the S3 Storage service will respond with a 200 response code,
# with an XML body.
if ($rest->ResponseStatusCode == 200) {
    print $responseStr, "\n";
    print "File uploaded.\n";
} else {
    # Examine the request/response to see what happened.
    print 'response status code = ', $rest->ResponseStatusCode, "\n";
    print 'response status text = ', $rest->ResponseStatusText, "\n";
    print 'response header: ', $rest->ResponseHeader, "\n";
    print 'response body: ', $responseStr, "\n";
    print "---\n";
    print 'LastRequestStartLine: ', $rest->LastRequestStartLine, "\n";
    print 'LastRequestHeader: ', $rest->LastRequestHeader, "\n";
}