oletools and olefile should be dependencies

[decalage2]

I just noticed that CAPE includes copies of the files olefile.py, olevba, oleid, etc. (which is great, I'm glad they are useful)
As those tools are regularly updated to fix bugs and to support new formats and new obfuscation tricks, it would be better not to include old copies here, but to install the latest versions with pip.
Is there a specific reason why they are copied rather than imported?

[doomedraven]

maybe just to prevent when something is changed in oletools internals and to not break the rest, but apart of that no

[decalage2]

BTW I just noticed oletools is listed in requirements.txt, so it may also be installed outside of CAPE, ending up with two versions. I guess the copy could be removed from the lib directory, as long as import statements in CAPE point to the version installed by pip.

A specific version of oletools can be fixed in requirements.txt, to avoid breaking changes.

[doomedraven]

yes i added that as i have added vba2graph and now it accepts a file with macros so it can generate graph and it uses oletools, but yes make sense to update

[doomedraven]

PR is done for that, if @kevoreilly is fine with that after merge there is only need to remove unused files,

• msgextract can be handled by sflock
• remove pyparsing.py
• DridexURLDecoder - is kinda for historical data only as that is useless nowdays

the vbadeobf is done already in oletools as iocs

so if we cleanup that folder we can move vba2graph to common and wipe office folder

[kevoreilly]

All sounds good to me, merged PR.

[kevoreilly]

(@decalage2 many thanks for all your hard work with oletools by the way!)

[doomedraven]

ok fully moved everything to oletools and iocs extract in my last PR

[kevoreilly]

I'm having some issues which I think might be related...

officeresults["Metadata"] = meta.get_meta()
AttributeError: OleMetadata instance has no attribute 'get_meta'

This is on line 1108 of static.py. olefile is installed at version 0.44 (won't update).

[kevoreilly]

The get_meta() function is no longer present in olefile from 0.44, I will try and update static.py accordingly,

[doomedraven]

ok its here, fixing it :) https://github.com/decalage2/oletools/blob/master/oletools/olemeta.py#L87

[enzok]

I thought I commented with a warning about the get_meta function last night. Apparently my phone didn't send. I have that commit in my fork I will do a PR.

[doomedraven]

the fix is
replace

officeresults["Metadata"] = meta.get_meta()

with

meta.SUMMARY_ATTRIBS

just tested in local

[kevoreilly]

I think the meta.SUMMARY_ATTRIBS is the more concise, assuming they both perform the same?

[doomedraven]

meta.SUMMARY_ATTRIBS is just the dict as before it was, they just removed an extra subcall

[doomedraven]

meta.SUMMARY_ATTRIBS
['codepage', 'title', 'subject', 'author', 'keywords', 'comments', 'template', 'last_saved_by', 'revision_number', 'total_edit_time', 'last_printed', 'create_time', 'last_saved_time', 'num_pages', 'num_words', 'num_chars', 'thumbnail', 'creating_application', 'security']
```

[doomedraven]

yes is the same, at least looks like

[kevoreilly]

Ok I merged that one. I have found a problem in quarantine.py:

import lib.cuckoo.common.office.olefile as olefile

Is import olefile enough or is import oletools.thirdparty.olefile as olefile better?

[doomedraven]

oletools.thirdparty.olefile

[doomedraven]

im checking this one

CAPE/web/templates/analysis/static/_office.html

Line 74 in 4bd606a

<td>{{analysis.static.office.Metadata.DocumentSummaryInformation.company}}</td>

as the @enzok restore that structure, but maybe it just also requires a cleanup, will bring update in few mins

[enzok]

I forgot at some point I added support to handle Office metadata from newer XML .doc files. The SummaryInfo doesn't match up 1 for 1 from the OLE docs. @doomedraven has the better fix.

[doomedraven]

basically we need this

# attribute names for SummaryInformation stream properties:
    # (ordered by property id, starting at 1)
    SUMMARY_ATTRIBS = ['codepage', 'title', 'subject', 'author', 'keywords', 'comments',
        'template', 'last_saved_by', 'revision_number', 'total_edit_time',
        'last_printed', 'create_time', 'last_saved_time', 'num_pages',
        'num_words', 'num_chars', 'thumbnail', 'creating_application',
        'security']

    # attribute names for DocumentSummaryInformation stream properties:
    # (ordered by property id, starting at 1)
    DOCSUM_ATTRIBS = ['codepage_doc', 'category', 'presentation_target', 'bytes', 'lines', 'paragraphs',
        'slides', 'notes', 'hidden_slides', 'mm_clips',
        'scale_crop', 'heading_pairs', 'titles_of_parts', 'manager',
        'company', 'links_dirty', 'chars_with_spaces', 'unused', 'shared_doc',
        'link_base', 'hlinks', 'hlinks_changed', 'version', 'dig_sig',
        'content_type', 'content_status', 'language', 'doc_version']

im checking if something else, and we need to make sure it's forced to printable

[decalage2]

I am going to remove the copy of olefile from oletools.thirdparty quite soon (olefile will be a dependency of oletools, that needs to be installed on its own), so I would advise to add olefile to your requirements.txt, and to use import olefile instead of import oletools.thirdparty.olefile as olefile.

[doomedraven]

ok, thanks i will fix imports and update reqs.txt