♻️ Split Onigumo module to smaller parts

[Glutexo]

• URLs file reader
• HTTP client, response extractor
• Integrating downloader

[Glutexo]

What parts? Ideas, @nappex?

I see that we can extract the parts related to HTTP. That is, get_url and get_body. But is it necessary? Maybe not now, when there are still just a handful of short and straightforward methods.

If you don’t object, I will close this issue for now.

[Glutexo]

Renaming the Onigumo module to Downloader may help. The original name hints it is a general-purpose module. The methods inside are, however, specific, related to downloading. See #118.

[Glutexo]

I am closing this issue. Feel free to re-open if you have a clear idea of what to extract from the module or want to spike about it.

[nappex]

i suppose that main idea was to seperate code to more func. if you remember we did a lot work inside one func but i feel we ve already fixed the problem. yes i agree it is a general problem, kind of refactor which can be done any time...

[Glutexo]

I’ve put more thought into this and see three possible descendants of the Downloader module:

• URLs file reader. The input file may become more complex than a simple list of addresses.
• HTTP client. A wrapper around the HTTPoison library issuing a request and extracting the response body. *
• The Downloader. This integration part would put all the functionality together.

* I see the “and” there, which may point to the module doing two things instead of one. I think, however, that the wrapping around HTTPoison is so thin that it does not deserve further extraction.

[Glutexo]

I reopened the issue and added the proposed parts to the description. I am looking forward to your feedback, @nappex.

[Glutexo]

I realized that there is the file name creating function. It looks separate, but now I think it belongs to the integration Downloader module.