Use metadata keys last-modified and etag

[jnioche]

Allow their use globally via configuration as well as overriding the de/activation on a per URL basis with use.Etag or use.ModifiedSince.

[jnioche]

To use etag and last-modified, you now just need to specify

 metadata.persist:
   - etag
   - last-modified

in the crawl configuration. This way, the key values will be persisted if returned by the server and used the next time round when querying it.

This can also be set on a per URL basis by setting the K/V in the metadata.