[Feature Request] Add support for lesson and workshop module

[nkfs]

Avoid duplicates

• Bug is not already reported in another issue

Describe the bug

Php files are not being downloaded, only the md files even though external files are selected.

2020-09-14 22:31:44  DEBUG  {results_handler}  Detected moodle version: 2019111803
2020-09-14 22:31:46  WARNING  {cookie_handler}  Moodle cookie has expired, an attempt is made to generate a new cookie.
2020-09-14 22:31:53  DEBUG  {moodle_service}  Checking for changes...
2020-09-14 22:31:53  DEBUG  {main}  Start downloading changed files...
2020-09-14 22:31:57  ERROR  {download_service}  Error while trying to download file: URLTarget (File (module_id: 7377770, section_name: Week 1, module_name: Assignment 3: Assessm..., content_filepath: /, content_filename: h
ttps://moodleaddress.com/mod/workshop/view.php?id=12345, content_fileurl:h
ttps://moodleaddress.com/mod/workshop/view.php?id=12345, content_filesize: 0, content_timemodified: 0, module_modname: cookie_mod, content_isexternalfile: True, saved_to: class1/Week 1 - assignments /https꞉／／moodleaddress.com／mod／workshop／view.php？id=12345, time_stamp: 0, modified: False, moved: False, deleted: False, notified: False, hash: None), False, None)

[C0D3D3V]

Thats a link that is parsed from a description. As requested in #44.
It only tries to download files. I can change it so it fallback to the html webpage, if no file could be downloaded.

[C0D3D3V]

I still have to fix something. I'll let you know as soon as I have tested it successfully.

[C0D3D3V]

You can test it with the latest code, in a testing enviroment (don't destroy your working enviroment :D). As this is a development branch... some files may be downloaded again, because I changed the names of the modules again.

[nkfs]

I just tested it, and it downloaded a bunch of .desktop files which are just links to more php files, but none of the content is being downloaded still. It managed to grab a single php file out of the hundreds there should be.

[nkfs]

actually looking closer, even the .md file it's downloading is only the first couple lines on the page right before the next bolded section / heading. Maybe that's why it's not parsing the "next" link on the lessons.

[C0D3D3V]

Is your moodle public?

[C0D3D3V]

the rest of the php files are moodle links?

[nkfs]

no, it's not a public moodle. and yes, each php file is just the "next" page of the lesson. I've logged responses, is there a way to get more debug output?

[nkfs]

Looking through the "responses.log", it appears all the md files are actually just the descriptions of each week / lesson. So maybe I'm not actually downloading any content at all. I've gone through these lessons already, does it matter to the app whether it's been viewed before? I can still see the content using the mobile app to verify the api

[C0D3D3V]

yes the markdown files are the descriptions of every file and section (as you in the --config explained)

[C0D3D3V]

"next" pages should not downloaded at all... I never saw a plugin that sets next pages in the description xD thats so stupid

[C0D3D3V]

It does not matter that you viewed a file ... The downloader has its own database.

[C0D3D3V]

You need to give me more information what is happening. Is there anything missing? Or are there to much files? I can add an option, to make it optional to parse links from description. Normaly only files from the api are downloaded

[C0D3D3V]

you can also contact me by mail or telegram...

[nkfs]

maybe I am misunderstanding the tool. I was hoping it would download the entire course, all the readings, assignments, and links. It downloaded all the pdf attachments correctly, and submissions / responses. But none of the content. What's your telegram? We can take this offline.

[C0D3D3V]

what content?
telegram username: c0d3d3v

[C0D3D3V]

It is fully capable of downloading files of any type (The module name in moodle for that is resource). The downloader structures the files the way it is structured on the course page. It uses the moodle app API in first place, but in the issue-41 branche it is now also capable of using a browser cookie, to download also not suported modules.

More supported modules:

• folder are directroies on the course page. They are fully downloaded and correctly structured.
• url are links to different websites or moodle itself. If the website contains a file or a video it tries to download it.
• index_mod is a mod I made up, it combines all modules that points to a index.html page, and that shoud be downloaded or contains a file/video. Currently thats the page and moodecvideo module. (and some special links parsed from descriptions)
• cookie_mod is the second mod I made up. It combines all moduls that needs a browser cookie to work. Currently that is only the kalvidres module (videos from Kalvidre) and special links parsed from descriptions.
• data is the module for databases . That are big modules in that teacher and sudents can upload files with lot of meta datas.
• assign contains a assignment, files for the assignment, submissions of the teacher or student, grades, and more descriptions.

It will download all descriptions in markdown format of files, sections and where ever a description is written. Most of the times these descriptions are annoying. But you can delete all local files as you want, they wont be redownloaded, so you can keep the interesting descriptions (markdown files have the extension .md).

Descriptions are now parsed for links, and these are traded as url mod if it is not clear that it is one of the other mod.

If you miss any importend file, you need to give me more information, specialy the responses.log would be good (but only over telegram), it can contain private information.

You can decide to not to download some of the modules, you only need to go through the additional configuration, with --config.

[C0D3D3V]

It seems that you can download lessons only if you start an attempt. I don't know what kind of effects this can have. That's also the reason why I don't download quizzes, because I don't want to start the attempt . If an attempt is limited in time it could cause problems for the student. But I will test how it looks like with completed lessons.

[C0D3D3V]

I would be happy if someone implements it and makes a Pull Request. Due to my studies I will not be able to implement this in the near future.

[C0D3D3V]

I created a wiki entry with all the API methods we need to download lessons.

[C0D3D3V]

I have implemented the first half to download lessons. The second half would be to also download the reports, that is, the ratings that can be viewed at /mod/lesson/report.php. Would you like the reports to be downloaded as well?

You can test the function now and give me feedback.

I'm going to have a look at the workshop mod, because of which you actually opened the issue.

[C0D3D3V]

I created a wiki entry with all the API methods we need to download workshops.

[C0D3D3V]

If you want that reports of lessons get also downloaded, then please open a new issue