Fetching HTML source if no description is given

[cbleek]

@fedys @TiSiE

adding a jobosting without a fulltext to solr makes no sense. So what can we do, if a job is activated and no html is given.

My idea is to simply fetch the html. If fetching fails, the job is not inserted.

Can we/should we do this in the solr module?

Keep in mind that all the time consuming tasks like fetching pages, inserting into solr, sending mails... should be moved into some queing system

[fedys]

I am not sure if the Solr module should be responsible for fetching an html. In my opinion the code which inserts an active Job should be responsible for a data completeness. If there is no description field in the import specification I would fetch an html in the SimpleImport module.

[cbleek]

@TiSiE do you agree?

[TiSiE]

Yes, I do agree.

[cbleek]

this will be done in yawik/SimpleImport#4