Processing node for preview datasets

[zelima]

We need new processing node to define one more processing flow (preview datasets). For that, we need to implement a class inheriting from BaseProcessingNode, and add that class name to the ORDERED_NODE_CLASSES array. new processing node will iterate over all resources and add processors that are generating first 10000 rows and views to the ones that have type derived/json

Acceptance criteria

• after able to run load_previews and load_views from Processors for getting preview #23

Tasks

• define new class for preview processing node class DerivedPreviewProcessingNode that will inherit DerivedFormatProcessingNode
• in the DerivedPreviewProcessingNode override get_artifacts and Iterate over existing artefacts (resources)
• check for the ones that are derived/json (or csv?)
• and pass the processors that we will need to get the desired resources and views.
• add DerivedPreviewProcessingNode to the ORDERED_NODE_CLASSES in correct position (last in the list)

Analysis

Example how refactored basic_nodes.py may look like

class DerivedFormatProcessingNode(BaseProcessingNode):

  def get_artifacts(self):
    for artifact in self.available_artifacts:
      if artifact.datahub_type == 'derived/csv':
        datahub_type = 'derived/preview' 
        resource_name = artifact.resource_name + '_preview.'
        output = ProcessingArtifact(
        datahub_type, resource_name,
        [artifact], [],
        [(new processors + old goes here)],
                True
            )
            yield output

class DerivedPreviewProcessingNode(...):
  def __init__(self):
    ... # (all same)

[zelima]

FIXED