Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Brunnhilde.py error: Is the path name current re: line 699 main()? #38

Closed
EagleRock1313 opened this issue Oct 31, 2018 · 16 comments
Closed

Brunnhilde.py error: Is the path name current re: line 699 main()? #38

EagleRock1313 opened this issue Oct 31, 2018 · 16 comments

Comments

@EagleRock1313
Copy link

Brunnhilde.py error: Is the path name current re: line 699 main()?

Downloading CSS and JS files from Github...
-1 / unknown

#Refer to function main()
#line 699

download assets

bootstrap_css_url = 'https://github.com/timothyryanwalsh/brunnhilde/blob/master/assets/css/bootstrap.min.css'
bootstrap_js_url = 'https://github.com/timothyryanwalsh/brunnhilde/blob/master/assets/js/bootstrap.min.js'
jquery_url = 'https://github.com/timothyryanwalsh/brunnhilde/blob/master/assets/js/jquery-3.3.1.slim.min.js'
popper_url = 'https://github.com/timothyryanwalsh/brunnhilde/blob/master/assets/js/popper.min.js'

print("\nDownloading CSS and JS files from Github...")
try:
    wget.download(bootstrap_css_url, os.path.join(css, 'bootstrap.min.css'))
    wget.download(bootstrap_js_url, os.path.join(js, 'bootstrap.min.js'))
    wget.download(jquery_url, os.path.join(js, 'jquery-3.3.1.slim.min.js'))
    wget.download(popper_url, os.path.join(js, 'popper.min.js'))

print("\nDownloads complete.")
@tw4l
Copy link
Owner

tw4l commented Nov 2, 2018

Hi, and thanks for creating an issue! I'm not quite sure that I understand the error you're encountering. The "-1/unknown" that is output to the terminal is from wget, and typically an indicator that the file download is in-progress.

I did a quick check of the JS and CSS URLs and they all seem to be accurate.

Is Brunnhilde getting hung up on this step for you?

@EagleRock1313
Copy link
Author

EagleRock1313 commented Nov 2, 2018 via email

@stefanabreitwieser
Copy link

Just a note to say that we had this error message since the last update too. It doesn't appear to have any impact on the final results(??)

Thanks Tim!

@tw4l
Copy link
Owner

tw4l commented Nov 2, 2018

Thank you, Sandy and Stefana!

Downloading the CSS and JavaScript files and saving them locally along the report is a new feature, because reports generated by previous versions of Brunnhilde were linking out to externally-hosted CSS and JS files, which was (ironically) a digital preservation risk for the outputs of a digital preservation utility. Less than ideal.

The behavior introduced in Brunnhilde 1.7.2 instead downloads these files from this Github repository each time and saves them in an "assets" directory in the Brunnhilde outputs, so that the HTML report can access them locally. The current implementation is less than ideal because:

  1. It requires an internet connection to run Brunnhilde, as there is not yet an option to cache these files somewhere on the local computer and just use those moving forward. (This is trickier than it sounds to implement, as pip installs Brunnhilde in different locations depending on which operating system you're running and how Python and pip are installed)

  2. The files are being downloaded by the Python wget library, which is what is outputting the "-1/unknown" to the terminal. This is actually normal behavior - aka not a bug - but it sure looks like one, as this issue and both of your comments attest.

To fix this for a future release, I propose:

  1. To introduce a new argument (or two) allowing users to save the CSS/JS assets to a local directory of their choosing, and then in subsequent runs of Brunnhilde to copy the assets from that directory rather than downloading them from Github every time.

  2. To replace that "-1/unknown" terminal message with more user-friendly status indicators like a typical progress bar or at least some variant of "Download in progress..."

What do you think?

@EagleRock1313
Copy link
Author

EagleRock1313 commented Nov 2, 2018 via email

@tw4l
Copy link
Owner

tw4l commented Nov 2, 2018

Hi Sandy,

The total size of the four CSS and JavaScript files is 277 KB (they are all minified text files, so they don't take much space).

I think a workaround to the permissions and updating issue would be to make the arguments explicit. Something like:

[First time]
brunnhilde.py --save-assets /path/to/local/directory

[Subsequent runs]
brunnhilde.py --load-assets /path/to/local/directory

And if a user does not use the --load-assets flag, Brunnhilde will just download them from Github as it currently does in 1.7.2.

Thoughts?

@EagleRock1313
Copy link
Author

EagleRock1313 commented Nov 2, 2018 via email

@tw4l tw4l mentioned this issue Nov 9, 2018
Merged
@tw4l
Copy link
Owner

tw4l commented Nov 9, 2018

Hi @EagleRock1313 and @stefanabreitwieser!

I believe this issue is fixed (at least for now) in release 1.8.0. I've also updated the package on PyPI, so you should be able to update with sudo pip install brunnhilde --upgrade.

Let me know how this works out for you and if the documentation is clear, and as always I'm happy for suggestions, criticisms, and pull requests :)

@tw4l tw4l closed this as completed Nov 9, 2018
@tw4l
Copy link
Owner

tw4l commented Nov 9, 2018

(I'm closing the issue to help me triage outstanding issues, but please feel free to re-open and comment here if you have feedback!)

@EagleRock1313
Copy link
Author

EagleRock1313 commented Nov 9, 2018 via email

@schmidtster
Copy link

I am trying to implement the solution as applied through this thread and README, saving the JS and CSS assets (--save_assets) to a local directory and referencing those when loading them (--load_assets). However, we are having trouble constructing the argument and outputting the results to the metadata directory, as listed in the guidelines.

We've tried multiple iterations of the following:

brunnhilde.py -b -d --hash MD5 -z /media/sf_.../...E01 --save_assets <directory_name> /media/sf_TEST-IMAGES/GK90/ metadata

brunnhilde.py --save_assets <directory_name> -b -d --hash MD5 -z /media/sf_.../...E01 /media/sf_TEST-IMAGES/GK90/ metadata

brunnhilde.py --load_assets <directory_name> -b -d --hash MD5 -z /media/sf_.../...E01 /media/sf_TEST-IMAGES/GK90/ > metadata

The last two seemed to pass the arguments, up to the metadata directory. However, the error message either says the directory already exists or no such directory exists. Both errors cancel the operation, preventing any of the checks/reports to be put in the metadata directory. Our general problem is that we want to download the assets and then copy them into an offline environment, reference them in the command line Brunnhilde, then (importantly) try running the Brunnhilde GUI and hope that it also references the local assets. Any help as to the specifics on how to run the command line would be greatly appreciated!

@tw4l tw4l reopened this Mar 4, 2019
@tw4l
Copy link
Owner

tw4l commented Mar 4, 2019

Hi @schmidtster, thank you for commenting. A few notes:

The --save_assets <directory_name> argument must be made before the 3 positional arguments (source, destination, and basename), which explains why the first iteration was giving you an error.

The <directory_name> must be replaced by the path to a local folder. When I do this on my Macbook, the terminal output I get specifies the --load_assets command to use moving forward. E.g.:

Tims-MBP:Desktop twalsh$ brunnhilde.py --save_assets save_test /Users/twalsh/PII_test_data/ . out

Downloading CSS and JS files from Github...

Downloads complete.

Brunnhilde assets saved locally. To use these in subsequent runs, use this argument: --load_assets "/Users/twalsh/Desktop/save_test"

Can you try doing this on your machine and copy the terminal output to this issue (or email it to me at timothyryanwalsh [at] gmail [dot] com)?

Finally, I'm realizing that I don't think the GUI has been updated to account for these new arguments, since this approach has been something of a stopgap before I have time to work on Brunnhilde 2, which should resolve this issue of the CSS/JS dependencies in a cleaner way. If being able to use this functionality with the GUI is an immediate need for you, I should be able to make those updates in the coming weeks.

@schmidtster
Copy link

schmidtster commented Mar 13, 2019 •
edited
Loading

Thank you for your help! I redid the arguments with --save_assets <directory_name> and instead of trying to pass source, destination, and basename, I just downloaded it to a local folder, as you suggested. Luckily that time I didn't get any errors.

After that, I played around with --load_assets command. The errors I was receiving were insufficient arguments or file/directory already existing. I will see about recreating the error and editing this comment to insert the errors. However, I did get the following argument to work:

brunnhilde.py --load_assets /home/bcadmin/Desktop/Forensics and Reporting/LOAD_ASSETS -b -d --hash MD5 -z /home/bcadmin/Desktop/GK10/image/GK10.E01 /home/bcadmin/Desktop/GK10 metadata

I renamed the assets folder to LOAD_ASSETS and was able to access them through BitCurator's Forensics and Reporting folder. I should have mentioned I am running this on a BitCurator instance.

In order to update the GUI, I took the liberty to insert the following lines into the main.py program on line 149:

# load local assets
self.process_list.append('--load_assets')
self.process_list.append('/home/bcadmin/Desktop/Forensics and Reporting/LOAD_ASSETS')

I put it right after appending brunnhilde.py to process_list. I was thinking the order the arguments are made in the list affect the GUI's construction of the command line argument, though this may not be the case, in which it wouldn't matter where we put this line. This solution only works for my case, because I know where the assets will be, but it wouldn't be universally acceptable.

If you could update the GUI to include this specific case, that would be great but isn't an immediate need. It will be a need later because I am creating documentation detailing the setup process for installing Brunnhilde GUI on BitCurator in an offline environment. Not having to modify the main.py file for every individual instance of the environment would be really helpful.

@tw4l
Copy link
Owner

tw4l commented Mar 13, 2019

Glad to hear this worked out for you! Closing this issue again for now, as I plan to address this issue as part of the Brunnhilde 2 rewrite.

@tw4l tw4l closed this as completed Mar 13, 2019
@tw4l
Copy link
Owner

tw4l commented Aug 1, 2020

Hi @EagleRock1313 @schmidtster. I thought you might like to know that I just published Brunnhilde 1.9.0, which removes the Bootstrap dependency from the HTML report and so no longer requires an internet connection or saving/loading assets locally. You can read more in the release notes: https://github.com/tw4l/brunnhilde/releases/tag/v1.9.0

@EagleRock1313
Copy link
Author

EagleRock1313 commented Aug 1, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tw4l @EagleRock1313 @stefanabreitwieser @schmidtster