PureBasic Requires Chilkat v11.7.0+

Expand Ligatures and Normalize Unicode in Extracted PDF Text

See more PDF Text Extraction Examples

Demonstrates the "expandLigatures" and "normalize" clean-up options. Typeset PDFs use single glyphs for "fi" and "fl"; by default they are expanded to their letters. Unicode normalization is NFC by default; NFKC also folds compatibility characters such as circled digits, the trademark sign and vulgar fractions into plain characters, and "none" leaves the text as the PDF has it. PureBasic

IncludeFile "CkPdf.pb"
IncludeFile "CkJsonObject.pb"

Procedure ChilkatExample()

    success.i = 0

    ;  This example requires the Chilkat API to have been previously unlocked.
    ;  See Global Unlock Sample for sample code.

    ;  The PDF used in this example is available at ligatures.pdf

    pdf.i = CkPdf::ckCreate()
    If pdf.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    success = CkPdf::ckLoadFile(pdf,"qa_data/pdf/ligatures.pdf")
    If success = 0
        Debug CkPdf::ckLastErrorText(pdf)
        CkPdf::ckDispose(pdf)
        ProcedureReturn
    EndIf

    json.i = CkJsonObject::ckCreate()
    If json.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    jsonOut.i = CkJsonObject::ckCreate()
    If jsonOut.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    ;  The first line is set with the fi and fl ligature glyphs; the second has circled digits,
    ;  a trademark sign and a vulgar fraction.
    success = CkPdf::ckExtractText(pdf,"text",json,jsonOut)
    If success = 0
        Debug CkPdf::ckLastErrorText(pdf)
        CkPdf::ckDispose(pdf)
        CkJsonObject::ckDispose(json)
        CkJsonObject::ckDispose(jsonOut)
        ProcedureReturn
    EndIf

    Debug "---- default (expandLigatures: true, normalize: NFC) ----"
    Debug CkJsonObject::ckStringOf(jsonOut,"text")

    ;  Keep the ligature characters as they are in the PDF.
    CkJsonObject::ckUpdateBool(json,"expandLigatures",0)
    CkJsonObject::ckUpdateString(json,"normalize","none")
    success = CkPdf::ckExtractText(pdf,"text",json,jsonOut)
    If success = 0
        Debug CkPdf::ckLastErrorText(pdf)
        CkPdf::ckDispose(pdf)
        CkJsonObject::ckDispose(json)
        CkJsonObject::ckDispose(jsonOut)
        ProcedureReturn
    EndIf

    Debug "---- expandLigatures: false, normalize: none ----"
    Debug CkJsonObject::ckStringOf(jsonOut,"text")

    ;  NFKC folds the compatibility characters too (and expands ligatures on its own).
    CkJsonObject::ckUpdateString(json,"normalize","NFKC")
    success = CkPdf::ckExtractText(pdf,"text",json,jsonOut)
    If success = 0
        Debug CkPdf::ckLastErrorText(pdf)
        CkPdf::ckDispose(pdf)
        CkJsonObject::ckDispose(json)
        CkJsonObject::ckDispose(jsonOut)
        ProcedureReturn
    EndIf

    Debug "---- normalize: NFKC ----"
    Debug CkJsonObject::ckStringOf(jsonOut,"text")

    ;  Sample output (shown here in ASCII: (fi) and (fl) stand for the single ligature characters, (1) and (2) for the circled digits,
    ;  (TM) for the trademark sign and 1/2 for the vulgar fraction; the NFKC output has plain "1", "2", "TM" and "1", fraction slash, "2"):

    ;  ---- default (expandLigatures: true, normalize: NFC) ----
    ;  Typeset Text
    ;  
    ;  The final flight of the season was officially confirmed on the fifth. Rooms (1) and (2) are reserved; the Harbor Cafe(TM) serves 1/2 portions.
    ;  ---- expandLigatures: false, normalize: none ----
    ;  Typeset Text
    ;  
    ;  The (fi)nal (fl)ight of the season was of(fi)cially con(fi)rmed on the (fi)fth. Rooms (1) and (2) are reserved; the Harbor Cafe(TM) serves 1/2 portions.
    ;  ---- normalize: NFKC ----
    ;  Typeset Text
    ;  
    ;  The final flight of the season was officially confirmed on the fifth. Rooms 1 and 2 are reserved; the Harbor CafeTM serves 1/2 portions.


    CkPdf::ckDispose(pdf)
    CkJsonObject::ckDispose(json)
    CkJsonObject::ckDispose(jsonOut)


    ProcedureReturn
EndProcedure