Automatically regenerate modified assets when building

[oscarlevin]

Each time pretext build is run, we will create a hash of each type of component for which the user might want to generate assets. Then these are each compared to the hash saved from the last build. If any are different, we regenerate that sort of asset. (This was suggested by @bnmnetp; please correct me if I'm missing something.)

A couple questions for @StevenClontz and other interested parties:

1. Where should the list of hashes be stored in the project? It could go in generated assets in a hidden folder or file, but it really doesn't need to be copied over for other projects. I guess, neither does the webwork-representations.xml though. Or maybe a build-support folder, that also contains build logs? Other ideas?
2. What should the default behavior be when building? A warning? Actually building the updated assets? We can have a flag that builds without regenerating, but note this is a fairly drastic change in behavior for a minor release. Or we make the -g flag only generate what is needed?

[bnmnetp]

It would be cool to get this implemented.

The way I was envisioning it, would be that it would more or less eliminate the need for the generate subcommand or the -g flag to build.

When you do a build, the cli checks the hash table using the unique id of each generated asset it encounters.

If the identifier isn't in the table then you know you have to generate it.
Once its generated then you store the md5 hash of the source in the table

if the identifier is already in the table then compute the md5 hash of the source and compare against what is stored. If they are different then you regenerate this one, and store the new hash value in the table.

if the computed md5 matches then you just move on.

At the end of the build you would pickle or json encode the hash table and save it. Makes sense to me that it would just be in the generated assets folder. Its not going to take up much disk space.

[oscarlevin]

Thanks for the hints. I think I can get this done very soon.

Your outline is more fine-grained than my original plan; I was going to simply hash the complete set of each type of asset that would be generated, and then regenerate all of those if needed. Eventually, having this work with individual images would be nice though.

I'm not sure exactly how that should be done though, for assets that don't have a unique id. I am pretty sure that the pre-processor step puts unique ids on everything, but should we really run that twice? Or maybe this is worth collaborating with pretext core to make the preprocessor's output available for parsing in python. I think I will continue this line of attack on -dev.

[oscarlevin]

This is now done, via #447.

Closing for now, but possible improvements for later:

• Use the assembled source so that fewer assets need to be regenerated each time (possibly including this as a feature in Core)
• Check for the generated asset files as well, in case those were deleted.