Chilkat Examples

ChilkatHOMEAndroid™Classic ASPCC++C#Mono C#.NET Core C#C# UWP/WinRTDataFlexDelphi ActiveXDelphi DLLVisual FoxProJavaLianjaMFCObjective-CPerlPHP ActiveXPHP ExtensionPowerBuilderPowerShellPureBasicCkPythonChilkat2-PythonRubySQL ServerSwift 2Swift 3/4TclUnicode CUnicode C++Visual Basic 6.0VB.NETVB.NET UWP/WinRTVBScriptXojo PluginNode.jsExcelGo

PureBasic Examples

Web API Categories

ASN.1
Amazon Glacier
Amazon S3
Amazon S3 (new)
Amazon SES
Amazon SNS
Amazon SQS
Async
Azure Cloud Storage
Azure Service Bus
Base64
Bounced Email
Box
CAdES
CSR
CSV
Certificates
Compression
DKIM / DomainKey
DSA
Diffie-Hellman
Digital Signatures
Dropbox
Dynamics CRM
ECC
Email Object
Encryption
FTP
FileAccess
Firebase
GMail REST API
Geolocation
Google APIs
Google Calendar
Google Cloud Storage
Google Drive
Google Photos
Google Sheets
Google Tasks
Gzip
HTML-to-XML/Text

HTTP
HTTP Misc
IMAP
JSON
JSON Web Encryption (JWE)
JSON Web Signatures (JWS)
JSON Web Token (JWT)
Java KeyStore (JKS)
MHT / HTML Email
MIME
Microsoft Graph
NTLM
OAuth1
OAuth2
OneDrive
OpenSSL
Outlook
PEM
PFX/P12
POP3
PRNG
REST
REST Misc
RSA
SCP
SFTP
SMTP
SSH
SSH Key
SSH Tunnel
SharePoint
Socket/SSL/TLS
Spider
Stream
Tar Archive
Upload
WebSocket
XAdES
XML
XML Digital Signatures
XMP
Zip
curl

 

 

 

(PureBasic) S3 Upload the Parts for a Multipart Upload

This example uploads a large file in parts. The multipart upload needs to have been first initiated prior to uploading the parts.

See http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html for more information about uploading parts.

Chilkat PureBasic Module Download

Chilkat PureBasic Module

IncludeFile "CkAuthAws.pb"
IncludeFile "CkFileAccess.pb"
IncludeFile "CkXml.pb"
IncludeFile "CkStream.pb"
IncludeFile "CkStringBuilder.pb"
IncludeFile "CkRest.pb"

Procedure ChilkatExample()

    ;  In the 1st step for uploading a large file, the multipart upload was initiated
    ;  as shown here: Initiate Multipart Upload

    ;  Other S3 Multipart Upload Examples:
    ;  Complete Multipart Upload
    ;  Abort Multipart Upload
    ;  List Parts

    ;  When we initiated the multipart upload, we saved the XML response to a file.  This
    ;  XML response contains the UploadId.  We'll begin by loading that XML and getting
    ;  the Upload ID.

    xmlInit.i = CkXml::ckCreate()
    If xmlInit.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    success.i = CkXml::ckLoadXmlFile(xmlInit,"s3_multipart_uploads/initiate.xml")
    If success <> 1
        Debug "Did not find the initiate.xml XML file."
        CkXml::ckDispose(xmlInit)
        ProcedureReturn
    EndIf

    uploadId.s = CkXml::ckGetChildContent(xmlInit,"UploadId")
    Debug "UploadId = " + uploadId

    ;  When uploading parts, we need to keep an XML record of each part number
    ;  and its corresponding ETag, which is received in the response for each part.
    ;  There can be up to 10000 parts, numbered 1 to 10000.
    ;  After all parts have been uploaded, the final step will be to complete
    ;  the multipart upload (see Complete Multipart Upload)

    ;  In this example, the large file we want to upload is somethingBig.zip
    fileToUploadPath.s = "s3_multipart_uploads/somethingBig.zip"

    ;  The minimum allowed part size is 5MB (5242880 bytes).  The last part can be smaller because
    ;  it will contain the remainder of the file.  (This minimum is enforced by the AWS service.)
    ;  We'll use the minimum allowed part size for this example.
    partSize.i = 5242880

    ;  Let's use Chilkat's FileAccess API to examine the file to be uploaded.  We'll get the size
    ;  of the file and find out how many parts will be needed, including the final "partial" part.
    fac.i = CkFileAccess::ckCreate()
    If fac.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    CkFileAccess::ckOpenForRead(fac,fileToUploadPath)

    ;  How many parts will there be if each part is 5242880 bytes?
    numParts.i = CkFileAccess::ckGetNumBlocks(fac,partSize)
    Debug "numParts = " + Str(numParts)
    CkFileAccess::ckFileClose(fac)

    ;  Imagine that we may be running this for the 1st time, or maybe we already
    ;  attempted to upload parts, and something failed. Maybe there was a network problem
    ;  the resulted in not all parts getting uploaded.  We'll write this code so that if run again,
    ;  it will upload whatever parts haven't yet been uploaded.

    ;  We'll keep a partsList.xml file to record the parts that have already been successfully
    ;  uploaded.  If this file does not yet exist, we'll create it..
    partsListFile.s = "s3_multipart_uploads/partsList.xml"
    partsListXml.i = CkXml::ckCreate()
    If partsListXml.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    If CkFileAccess::ckFileExists(fac,partsListFile) = 1
        CkXml::ckLoadXmlFile(partsListXml,partsListFile)
    EndIf

    ;  Make sure the top-level tag is "CompleteMultipartUpload"
    CkXml::setCkTag(partsListXml, "CompleteMultipartUpload")

    ;  --------------------------------------
    ;  Before entering the loop to upload parts,
    ;  setup the REST object with AWS authentication,
    ;  and make the initial connection.
    rest.i = CkRest::ckCreate()
    If rest.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    ;  Connect to the Amazon AWS REST server.
    bTls.i = 1
    port.i = 443
    bAutoReconnect.i = 1
    success = CkRest::ckConnect(rest,"s3.amazonaws.com",port,bTls,bAutoReconnect)

    ;  ----------------------------------------------------------------------------
    ;  Important: For buckets created in regions outside us-east-1,
    ;  there are three important changes that need to be made.
    ;  See Working with S3 Buckets in Non-us-east-1 Regions for the details.
    ;  ----------------------------------------------------------------------------

    ;  Provide AWS credentials for the REST call.
    authAws.i = CkAuthAws::ckCreate()
    If authAws.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    CkAuthAws::setCkAccessKey(authAws, "AWS_ACCESS_KEY")
    CkAuthAws::setCkSecretKey(authAws, "AWS_SECRET_KEY")
    CkAuthAws::setCkServiceName(authAws, "s3")
    success = CkRest::ckSetAuthAws(rest,authAws)

    ;  Set the bucket name via the HOST header.
    ;  In this case, the bucket name is "chilkat100".
    CkRest::setCkHost(rest, "chilkat100.s3.amazonaws.com")
    ;  --------------------------------------

    partNumber.i = 1
    sbPartNumber.i = CkStringBuilder::ckCreate()
    If sbPartNumber.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    While (partNumber <= numParts)
        Debug "---- " + Str(partNumber) + " ----"

        ;  This cumbersome way of converting an integer to a string is because
        ;  Chilkat examples are written in a script that is converted to many programming languages.
        ;  At this time, the translator does not have integer-to-string code generation capability..
        CkStringBuilder::ckClear(sbPartNumber)
        CkStringBuilder::ckAppendInt(sbPartNumber,partNumber)

        bPartAlreadyUploaded.i = 0

        ;  If there are no children, then the XML is empty and no parts have yet been uploaded.
        numUploadedParts.i = CkXml::ckNumChildren(partsListXml)
        If numUploadedParts > 0
            ;  If some parts have been uploaded, check to see if this particular part was already upload.
            ;  If so, then it can be skipped.

            ;  Position ourselves at the 1st record.
            xRec0.i = CkXml::ckGetChild(partsListXml,0)
            foundRec.i = CkXml::ckFindNextRecord(xRec0,"PartNumber",CkStringBuilder::ckGetAsString(sbPartNumber))
            If CkXml::ckLastMethodSuccess(xRec0) = 1
                bPartAlreadyUploaded = 1
                Debug "Part " + Str(partNumber) + " was previously uploaded."
                Debug CkXml::ckGetXml(foundRec)
                CkXml::ckDispose(foundRec)

            EndIf

            CkXml::ckDispose(xRec0)

        EndIf

        ;  If this part was not already uploaded, we need to upload.
        ;  Also update the partsListXml and save as each part is successfully uploaded.
        If bPartAlreadyUploaded = 0
            Debug "Uploading part " + Str(partNumber) + " ..."

            ;  Setup the stream source for the large file to be uploaded..
            fileStream.i = CkStream::ckCreate()
            If fileStream.i = 0
                Debug "Failed to create object."
                ProcedureReturn
            EndIf

            CkStream::setCkSourceFile(fileStream, fileToUploadPath)
            ;  The Chilkat Stream API has features to make uploading a parts
            ;  of a file easy.  Indicate the part size by setting the SourceFilePartSize
            ;  property.
            CkStream::setCkSourceFilePartSize(fileStream, partSize)

            ;  Our HTTP start line to upload a part will look like this:
            ;  PUT /ObjectName?partNumber=PartNumber&uploadId=UploadId HTTP/1.1

            ;  Set the query params.  We'll need partNumber and uploadId.
            ;  Make sure the query params from previous iterations are clear.
            CkRest::ckClearAllQueryParams(rest)
            CkRest::ckAddQueryParam(rest,"partNumber",CkStringBuilder::ckGetAsString(sbPartNumber))
            CkRest::ckAddQueryParam(rest,"uploadId",uploadId)

            ;  Upload this particular file part.
            ;  Tell the fileStream which part is being uploaded.
            ;  Our partNumber is 1-based (the 1st part is at index 1), but the fileStream's SourceFilePart
            ;  property is 0-based.  Therefore we use partNumber-1.
            CkStream::setCkSourceFilePart(fileStream, partNumber - 1)

            ;  Because the SourceFilePart and SourceFilePartSize properties are set, the stream will
            ;  will provide just that part of the file.
            responseStr.s = CkRest::ckFullRequestStream(rest,"PUT","/somethingBig.zip",fileStream)
            If CkRest::ckLastMethodSuccess(rest) <> 1
                Debug CkRest::ckLastErrorText(rest)
                CkXml::ckDispose(xmlInit)
                CkFileAccess::ckDispose(fac)
                CkXml::ckDispose(partsListXml)
                CkRest::ckDispose(rest)
                CkAuthAws::ckDispose(authAws)
                CkStringBuilder::ckDispose(sbPartNumber)
                CkStream::ckDispose(fileStream)
                ProcedureReturn
            EndIf

            If CkRest::ckResponseStatusCode(rest) <> 200
                ;  Examine the request/response to see what happened.
                Debug "response status code = " + Str(CkRest::ckResponseStatusCode(rest))
                Debug "response status text = " + CkRest::ckResponseStatusText(rest)
                Debug "response header: " + CkRest::ckResponseHeader(rest)
                Debug "response body: " + responseStr
                Debug "---"
                Debug "LastRequestStartLine: " + CkRest::ckLastRequestStartLine(rest)
                Debug "LastRequestHeader: " + CkRest::ckLastRequestHeader(rest)
                CkXml::ckDispose(xmlInit)
                CkFileAccess::ckDispose(fac)
                CkXml::ckDispose(partsListXml)
                CkRest::ckDispose(rest)
                CkAuthAws::ckDispose(authAws)
                CkStringBuilder::ckDispose(sbPartNumber)
                CkStream::ckDispose(fileStream)
                ProcedureReturn
            EndIf

            ;  OK, this part was uploaded..
            ;  The response will have a 0-length body.  The only information we need is the
            ;  ETag response header field.
            etag.s = CkRest::ckResponseHdrByName(rest,"ETag")
            ;  It should be present, but just in case there was no ETag header...
            If CkRest::ckLastMethodSuccess(rest) <> 1
                Debug "No ETag response header found!"
                Debug "response header: " + CkRest::ckResponseHeader(rest)
                CkXml::ckDispose(xmlInit)
                CkFileAccess::ckDispose(fac)
                CkXml::ckDispose(partsListXml)
                CkRest::ckDispose(rest)
                CkAuthAws::ckDispose(authAws)
                CkStringBuilder::ckDispose(sbPartNumber)
                CkStream::ckDispose(fileStream)
                ProcedureReturn
            EndIf

            ;  We need to add record to the partsListXml.
            ;  The record will look like this:
            ;  &lt;Part>
            ;    &lt;PartNumber>PartNumber&lt;/PartNumber>
            ;    &lt;ETag>ETag&lt;/ETag>
            ;  &lt;/Part>
            xPart.i = CkXml::ckNewChild(partsListXml,"Part","")
            CkXml::ckNewChildInt2(xPart,"PartNumber",partNumber)
            CkXml::ckNewChild2(xPart,"ETag",etag)
            CkXml::ckDispose(xPart)

            success = CkXml::ckSaveXml(partsListXml,partsListFile)
            If success <> 1
                Debug CkXml::ckLastErrorText(partsListXml)
                CkXml::ckDispose(xmlInit)
                CkFileAccess::ckDispose(fac)
                CkXml::ckDispose(partsListXml)
                CkRest::ckDispose(rest)
                CkAuthAws::ckDispose(authAws)
                CkStringBuilder::ckDispose(sbPartNumber)
                CkStream::ckDispose(fileStream)
                ProcedureReturn
            EndIf

            Debug "-- Part " + Str(partNumber) + " uploaded. ---------------------"
        EndIf

        partNumber = partNumber + 1
    Wend

    Debug "Finished.  All parts uploaded."


    CkXml::ckDispose(xmlInit)
    CkFileAccess::ckDispose(fac)
    CkXml::ckDispose(partsListXml)
    CkRest::ckDispose(rest)
    CkAuthAws::ckDispose(authAws)
    CkStringBuilder::ckDispose(sbPartNumber)
    CkStream::ckDispose(fileStream)


    ProcedureReturn
EndProcedure

 

© 2000-2019 Chilkat Software, Inc. All Rights Reserved.