Added option to embedart using an image URL [small PR] #4719

arsaboo · 2023-03-18T23:37:30Z

Description

Fixes #83

Added another option -u to provide an image URL. The plugin can now download the image, check if it is valid, and then embed the picture as cover art.

To Do

Documentation. (If you've add a new command-line flag, for example, find the appropriate page under docs/ to describe it.)
Changelog. (Add an entry to docs/changelog.rst near the top of the document.)
Tests. (Encouraged but not strictly required.)

JOJ0 · 2023-03-27T14:21:11Z

Just had a quick look: Maybe you could add error handling for:

http requests
file handling

Best you look around how others do stuff like that. E.g file handling in the smartplaylist plugin for an example and maybe fetchart itself for requests errors catching.

JOJ0 · 2023-03-27T14:32:55Z

Some example of handling file errors and also important: How to make use of syspath() normpath, bytestring_path functions, which is an importent aspect of how beets handles files: https://github.com/beetbox/beets/pull/4719/files#diff-5a7d57cfa16b2afc519d39be035fd1808000f256b044d8def53142c97e46196eR95-R101

I'm also only learning stuff like that with beets, so hope that is the right pointer.

Recently I worked on this plugin and copied how files are handled there:

beets/beetsplug/smartplaylist.py

Lines 221 to 234 in ed1202b

    
           if not pretend: 
        
               prefix = bytestring_path(self.config['prefix'].as_str()) 
        
               # Write all of the accumulated track lists to files. 
        
               for m3u in m3us: 
        
                   m3u_path = normpath(os.path.join(playlist_dir, 
        
                                       bytestring_path(m3u))) 
        
                   mkdirall(m3u_path) 
        
                   with open(syspath(m3u_path), 'wb') as f: 
        
                       for path in m3us[m3u]: 
        
                           if self.config['forward_slash'].get(): 
        
                               path = path_as_posix(path) 
        
                           if self.config['urlencode']: 
        
                               path = bytestring_path(pathname2url(path)) 
        
                           f.write(prefix + path + b'\n')

Well, actually errors are _not_handled here. In my opinion the with block needs to be wrapped into try/except and then an OSError should be catched and error class the utils package provides should be thrown.

I think like that is making sure a "nice" error is displayed and no ugly exception (the FileSystemError you can import from beets.utils package) https://github.com/beetbox/beets/pull/4399/files#diff-f20a2ab41ac3a3c83575cac2b871653999b0e4d28c27336a1f3897a004f5d6daR86-R93

arsaboo · 2023-03-27T18:50:35Z

@JOJ0 I have added error handling. I think the path-related comment is less relevant, as the file is saved in the temp folder and removed later, i.e., the path is less important here.

JOJ0 · 2023-03-27T19:48:47Z

Nice! You could use the exception provided in utils for the file handling. Maybe beets has one provided for requests/apis as well. Dont know..

JOJ0 · 2023-03-27T19:50:20Z

But not sure if it works with Image.open

JOJ0 · 2023-03-27T19:59:15Z

Ok I see you did something similar to what we have in fetchart, so I guess that's fine.

beets/beetsplug/fetchart.py

Lines 370 to 376 in ed1202b

    
           def get_image_urls(url, preferred_width=None): 
        
               try: 
        
                   response = self.request(url) 
        
               except requests.RequestException: 
        
                   self._log.debug('{}: error receiving response' 
        
                                   .format(self.NAME)) 
        
                   return

beetsplug/embedart.py

JOJ0 · 2023-03-28T08:18:52Z

beetsplug/embedart.py

+                        with open('temp.jpg', 'wb') as f:
+                            f.write(response.content)
+                    except Exception as e:
+                        self._log.error(f"Error writing file: {e}"


Here a few examples of how to use the file error exception class beets already provides: https://github.com/search?q=repo%3Abeetbox%2Fbeets%20FileSystemError&type=code

beets/beets/util/__init__.py

Lines 534 to 536 in ed1202b

except OSError as exc:

raise FilesystemError(exc, 'link', (path, dest),

traceback.format_exc())

This for example will result in a nicely human readable message like: "Error durint linking of paths ....."

Instead of the word "link" you could use "write" in your case. The error class will handle the rewording magic! Give it a try! It's a cool concept in beets. We don't have to reinvent the wheel :-)

Here the definition of what will happen when "write" is given exactly:

beets/beets/util/__init__.py

Lines 106 to 124 in ed1202b

def get_message(self):

# Use a nicer English phrasing for some specific verbs.

if self.verb in ('move', 'copy', 'rename'):

clause = 'while {} {} to {}'.format(

self._gerund(),

displayable_path(self.paths[0]),

displayable_path(self.paths[1])

)

elif self.verb in ('delete', 'write', 'create', 'read'):

clause = 'while {} {}'.format(

self._gerund(),

displayable_path(self.paths[0])

)

else:

clause = 'during {} of paths {}'.format(

self.verb, ', '.join(displayable_path(p) for p in self.paths)

)

return f'{self._reasonstr()} {clause}'

I followed the approach used in fetchart and playlist plugins.

beets/beetsplug/fetchart.py

Lines 352 to 353 in ed1202b

except util.FilesystemError as exc:

self._log.debug('error cleaning up tmp art: {}', exc)

I feel that the following may needlessly complicate things as we will have to define path and dest, which may not be needed here.

except OSError as exc: raise util.FilesystemError(exc, 'write', (path, dest), traceback.format_exc())

replace path+dest with one thing then

see the definition I posted: #4719 (comment)

I think you misunderstood: You should still catch oserror but then raise FilesystemError

Filesystemerror will translate your exception into human readable

caught OSError. let's hope this works.

arsaboo · 2023-04-14T12:29:55Z

Looks like tests are failing for unrelated reasons...

JOJ0 · 2023-04-18T12:42:14Z

I realised I can simply re-run failed tests. I think it was just some problems with the underlying cloud servers at github/ms/whatever. Rerun worked.

I think you left a debug print around that tempimg code. Maybe post some test runs with good and bad input values. I won't get to testing myself....

arsaboo · 2023-04-19T01:05:59Z

@JOJ0 Removed the debug print. All the test runs work fine:

$ beet embedart -u https://crapurl album:"Khauff"
embedart: HTTPSConnectionPool(host='crapurl', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f69ff7d9c30>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
$ beet embedart -u crapurl album:"Khauff"
embedart: Invalid URL 'crapurl': No scheme supplied. Perhaps you meant https://crapurl?
$ beet embedart -u https://www.google.com album:"Khauff"
embedart: Invalid image file

JOJ0 · 2023-04-21T06:07:47Z

@arsaboo sorry for the delay. I realised embed art is pretty well covered with unittests and it would make sense to add some for this new feature. I just pushed a test that checks for proper fetching and embedding of a jpeg file to your feature branch. I hope that is ok.

I know you once told me that writing tests is not in your bag of tricks yet. Here I'd like to encourage you to give it a try. You could copy test_embed_art_from_url_with_yes_input() and make a version from it that tests for a png file (The imported test helper supports that). Also have a look at the test helper code FetchImageHelper.mock_response(). It uses the responses library to mock a request for the passed URL and content_type.

A third copy of this test could test if we properly fail when no image is fetched (with something other than jpeg or png, the test helper would return nothing I think, and we'd get an invalid url error). We could test then if the media file has no image embeded.

hope that helps and get back to me with questions. Also @sampsyo or @Serene-Arc could you have a quick look if my test makes sense that way or if I'm not testing anything worthwhile. Just to make sure. Thanks!

docs/plugins/embedart.rst

arsaboo · 2023-04-21T12:42:41Z

@JOJ0 Thanks for the review and for adding a test. I added a few tests. Hopefully, this should be enough to get this merged.

As I mentioned earlier, tests are not something I am very comfortable with, and would rather not mess with them unless it is an ABSOLUTE MUST to get this PR merged. Feel free to modify the PR to add anything that you feel will improve the code quality.

and shorten second and third test a little by providing -y cli flag. Enough to test with interactive input once. Move all 3 tests to the very bottom of the test class.

JOJ0 · 2023-04-21T19:15:31Z

Oh, it should not feel like a burden. I just thought it makes sense and actually is a rather easy opportunity to learn something about tests. The beauty of beets is that almost everything that could be tested is found around the codebase somewhere. Big opportunities to learn about testing everywhere.

Also the nice thing about tests is, it makes you see thing you wouldn't even with damn thorough manual test runs. I just realised that actually we could download html files and try to embed, but in that case the error handling of embed to file function itself kicks in and handles it. So I think that's good enough:

$ python -m unittest test.test_embedart.EmbedartCliTest.test_embed_art_from_url_not_image

Sending event: database_change
no user configuration found at /tmp/tmpq280n_md/config.yaml
data directory: /tmp/tmpq280n_md
plugin paths: 
Sending event: pluginload
embedart: embedding /tmp/image.html
embedart: not embedding image of unsupported type: image/x-None
Sending event: cli_exit
.
----------------------------------------------------------------------
Ran 1 test in 0.064s

OK

In a real world test this looks like this:

$ beet -vv embedart -u http://chat.peek-a-boo.at/index.html chimpo dial

Sending event: library_opened

Chimpo - On The Dial - 01 - Rig Doctor (2022/UK)  [APHA026] f:FLAC a:01-01/Discogs |Electronic, Drum n Bass, Jungle
Chimpo - On The Dial - 02 - Luv NRG (2022/UK)  [APHA026] f:FLAC a:01-01/Discogs |Electronic, Drum n Bass, Jungle
Chimpo - On The Dial - 03 - Buzz Army (2022/UK)  [APHA026] f:FLAC a:01-01/Discogs |Electronic, Drum n Bass, Jungle
Chimpo - On The Dial - 04 - Broad (2022/UK)  [APHA026] f:FLAC a:01-01/Discogs |Electronic, Drum n Bass, Jungle
Modify artwork for 4 files (Y/n)? y
embedart: embedding /tmp/image.html
embedart: not embedding image of unsupported type: image/x-None
embedart: embedding /tmp/image.html
embedart: not embedding image of unsupported type: image/x-None
embedart: embedding /tmp/image.html
embedart: not embedding image of unsupported type: image/x-None
embedart: embedding /tmp/image.html
embedart: not embedding image of unsupported type: image/x-None
Sending event: cli_exit

Makes it more clear what we are doing.

JOJ0 · 2023-04-21T19:28:07Z

Oh I see you mentioned above that this is actually expected: #4719 (comment)

I needed a test to read your comment haha ;-)

Thanks a lot for the patience and all! We went from almost no error handling to pretty bulletproof error handling and even have some unittesting. So, yeah, I think it was all worth it. Thanks again! Great! Now I can fix my sometimes funny random album art ;-)

Merging...

arsaboo · 2023-04-21T19:43:05Z

Thanks @JOJ0 for your patience and for guiding me through this.

BTW, where/how are you running python -m unittest test.test_embedart.EmbedartCliTest.test_embed_art_from_url_not_image? I will explore running tests inside VS Code (which is where I code)

JOJ0 · 2023-04-21T19:52:59Z

Thanks @JOJ0 for your patience and for guiding me through this.
Your are very welcome!

BTW, where/how are you running python -m unittest test.test_embedart.EmbedartCliTest.test_embed_art_from_url_not_image? I will explore running tests inside VS Code (which is where I code)

Simply run it in the terminal. From the root of the repo. In vscode terminal or regular terminal.

arsaboo added 6 commits March 18, 2023 18:23

Add option to import using URL

f051d67

Update embedart.py

feeeb40

Update embedart.py

4dd1654

Update embedart.py

81835b0

Update embedart.py

64c2ae1

Update embedart.py

59d5d23

arsaboo changed the title ~~Added option to embedart using a URL~~ Added option to embedart using an image URL Mar 18, 2023

arsaboo added 2 commits March 19, 2023 09:15

Move import inside the function

2d942ce

Update parser options

2060e04

arsaboo closed this Mar 20, 2023

arsaboo reopened this Mar 20, 2023

arsaboo closed this Mar 20, 2023

arsaboo reopened this Mar 20, 2023

arsaboo closed this Mar 23, 2023

arsaboo reopened this Mar 23, 2023

arsaboo changed the title ~~Added option to embedart using an image URL~~ Added option to embedart using an image URL [small PR] Mar 23, 2023

arsaboo mentioned this pull request Mar 25, 2023

fetchart: Use MusicBrainz' Bandcamp/Discogs/... association to fetch art #4707

Closed

Add additional error handling

2e2a3bf

JOJ0 reviewed Mar 27, 2023

View reviewed changes

beetsplug/embedart.py Outdated Show resolved Hide resolved

arsaboo added 2 commits March 27, 2023 19:28

More error handling

a4b4df8

Update documents and add changelog entry

6626b76

JOJ0 reviewed Mar 28, 2023

View reviewed changes

arsaboo added 2 commits March 28, 2023 09:22

Add FilesystemError handling

2fcdb8c

Raise OSError

3b39a4f

arsaboo reopened this Apr 14, 2023

arsaboo closed this Apr 15, 2023

arsaboo reopened this Apr 15, 2023

arsaboo closed this Apr 16, 2023

arsaboo reopened this Apr 16, 2023

Remove logging

bb2ce02

arsaboo closed this Apr 19, 2023

arsaboo reopened this Apr 19, 2023

arsaboo closed this Apr 19, 2023

arsaboo reopened this Apr 19, 2023

Add test for embedart -u fetching jpeg from url

3dd32c9

JOJ0 approved these changes Apr 21, 2023

View reviewed changes

JOJ0 reviewed Apr 21, 2023

View reviewed changes

docs/plugins/embedart.rst Show resolved Hide resolved

arsaboo added 3 commits April 21, 2023 07:58

Updated language

d48eab9

Add test for png image

919f326

Add test for invalid URL

47cb11b

Fix embedart -u test names

f3d4929

and shorten second and third test a little by providing -y cli flag. Enough to test with interactive input once. Move all 3 tests to the very bottom of the test class.

JOJ0 force-pushed the embedart_url branch from 13656fd to f3d4929 Compare April 21, 2023 18:12

Fix text/html file ending in embedart -u test

56adba7

Makes it more clear what we are doing.

JOJ0 merged commit f6b0311 into beetbox:master Apr 21, 2023

arsaboo deleted the embedart_url branch April 21, 2023 20:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added option to embedart using an image URL [small PR] #4719

Added option to embedart using an image URL [small PR] #4719

arsaboo commented Mar 18, 2023 •

edited by JOJ0

Loading

JOJ0 commented Mar 27, 2023 •

edited

Loading

JOJ0 commented Mar 27, 2023 •

edited

Loading

arsaboo commented Mar 27, 2023 •

edited

Loading

JOJ0 commented Mar 27, 2023

JOJ0 commented Mar 27, 2023

JOJ0 commented Mar 27, 2023

JOJ0 Mar 28, 2023

JOJ0 Mar 28, 2023 •

edited

Loading

JOJ0 Mar 28, 2023

arsaboo Mar 28, 2023 •

edited

Loading

JOJ0 Mar 28, 2023 •

edited

Loading

JOJ0 Mar 28, 2023

arsaboo Mar 28, 2023

arsaboo commented Apr 14, 2023

JOJ0 commented Apr 18, 2023 •

edited

Loading

arsaboo commented Apr 19, 2023

JOJ0 commented Apr 21, 2023 •

edited

Loading

arsaboo commented Apr 21, 2023 •

edited

Loading

JOJ0 commented Apr 21, 2023 •

edited

Loading

JOJ0 commented Apr 21, 2023 •

edited

Loading

arsaboo commented Apr 21, 2023

JOJ0 commented Apr 21, 2023

	except OSError as exc:
	raise FilesystemError(exc, 'link', (path, dest),
	traceback.format_exc())

	def get_message(self):
	# Use a nicer English phrasing for some specific verbs.
	if self.verb in ('move', 'copy', 'rename'):
	clause = 'while {} {} to {}'.format(
	self._gerund(),
	displayable_path(self.paths[0]),
	displayable_path(self.paths[1])
	)
	elif self.verb in ('delete', 'write', 'create', 'read'):
	clause = 'while {} {}'.format(
	self._gerund(),
	displayable_path(self.paths[0])
	)
	else:
	clause = 'during {} of paths {}'.format(
	self.verb, ', '.join(displayable_path(p) for p in self.paths)
	)

	return f'{self._reasonstr()} {clause}'

	except util.FilesystemError as exc:
	self._log.debug('error cleaning up tmp art: {}', exc)

Added option to embedart using an image URL [small PR] #4719

Added option to embedart using an image URL [small PR] #4719

Conversation

arsaboo commented Mar 18, 2023 • edited by JOJ0 Loading

Description

To Do

JOJ0 commented Mar 27, 2023 • edited Loading

JOJ0 commented Mar 27, 2023 • edited Loading

arsaboo commented Mar 27, 2023 • edited Loading

JOJ0 commented Mar 27, 2023

JOJ0 commented Mar 27, 2023

JOJ0 commented Mar 27, 2023

JOJ0 Mar 28, 2023

Choose a reason for hiding this comment

JOJ0 Mar 28, 2023 • edited Loading

Choose a reason for hiding this comment

JOJ0 Mar 28, 2023

Choose a reason for hiding this comment

arsaboo Mar 28, 2023 • edited Loading

Choose a reason for hiding this comment

JOJ0 Mar 28, 2023 • edited Loading

Choose a reason for hiding this comment

JOJ0 Mar 28, 2023

Choose a reason for hiding this comment

arsaboo Mar 28, 2023

Choose a reason for hiding this comment

arsaboo commented Apr 14, 2023

JOJ0 commented Apr 18, 2023 • edited Loading

arsaboo commented Apr 19, 2023

JOJ0 commented Apr 21, 2023 • edited Loading

arsaboo commented Apr 21, 2023 • edited Loading

JOJ0 commented Apr 21, 2023 • edited Loading

JOJ0 commented Apr 21, 2023 • edited Loading

arsaboo commented Apr 21, 2023

JOJ0 commented Apr 21, 2023

arsaboo commented Mar 18, 2023 •

edited by JOJ0

Loading

JOJ0 commented Mar 27, 2023 •

edited

Loading

JOJ0 commented Mar 27, 2023 •

edited

Loading

arsaboo commented Mar 27, 2023 •

edited

Loading

JOJ0 Mar 28, 2023 •

edited

Loading

arsaboo Mar 28, 2023 •

edited

Loading

JOJ0 Mar 28, 2023 •

edited

Loading

JOJ0 commented Apr 18, 2023 •

edited

Loading

JOJ0 commented Apr 21, 2023 •

edited

Loading

arsaboo commented Apr 21, 2023 •

edited

Loading

JOJ0 commented Apr 21, 2023 •

edited

Loading

JOJ0 commented Apr 21, 2023 •

edited

Loading