Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple saves in pipeline #2311

Closed
aclews opened this issue Dec 13, 2018 · 6 comments
Closed

Multiple saves in pipeline #2311

aclews opened this issue Dec 13, 2018 · 6 comments

Comments

@aclews
Copy link

aclews commented Dec 13, 2018

If I take the simple split (via tags) and merge (via inputs) flow below, with 3 QC outputs, I only get the last writer to output

Please can PDAL be updated to include multiple saves

{
"pipeline":[
{
"type":"readers.las",
"filename":"filename.las",
"tag":"E000_Readers_End"
},
{
"inputs":["E000_Readers_End"],
"type":"filters.range",
"limits":"Z![130:140]",
"tag":"S010_ClassifyZ_01of02"
},
{
"inputs":["S010_ClassifyZ_01of02"],
"type":"writers.las",
"filename":"Filename_QC_01of02.las"
}
{
"inputs":["E000_Readers_End"],
"type":"filters.range",
"limits":"Z[130:140]"
},
{
"type":"filters.assign",
"assignment":"Classification[:]=3",
"tag":"S010_ClassifyZ_02of02"
},
{
"inputs":["S010_ClassifyZ_02of02"],
"type":"writers.las",
"filename":"Filename_QC_02of02.las"
}
{
"type":"filters.merge",
"tag":"E010",
"inputs":["S010_ClassifyZ_01of02","S010_ClassifyZ_02of02"]
},
{
"inputs":["E010"],
"type":"writers.las",
"filename":"Filename_Merge.las"
}
]
}

@abellgithub
Copy link
Contributor

PDAL has never fully executed pipelines with multiple endpoints. Take a look at #1537 and #1386. That said, you can have a pipeline with multiple writers, they just can't be separate endpoints. For example, you can have something like:

Reader -> Filter -> Writer -> Filter -> Writer

You can also chain writers to get two types of output:

Reader -> Writer -> Writer

What you can't do is split the pipeline and get proper execution of all branches. As you found, this doesn't work:

Reader -> Filter -> Filter -> Filter -> Writer
                     |             |         
                     v             v
                  Writer        Writer

I've opened #2313 to address this situation.

@jessenestler
Copy link

@abellgithub I have a pipeline that's very similar to the example you gave:

Reader -> Filter -> GDALWriter1 -> Filter -> GDALWriter2

Is there a way to specify filenames to both writers as options to the pipeline? Something like:

pdal pipeline pipeline.json --readers.las.filename="input.las" --writers.gdal.filename="output1.tif" --writers.gdal.filename="output2.tif"

FWIW I've tried exactly this but it gives me an PDAL: writers.gdal: Attempted to set value twice for argument 'filename'. error. Happy to open a separate issue if it's merited.

@abellgithub
Copy link
Contributor

@jessenestler Please see "option substitution" here: http://pdal.io/apps/pipeline.html#substitutions

@jessenestler
Copy link

Perfect, exactly what I needed, thank you.

@hadallen
Copy link

hadallen commented Sep 4, 2024

@jessenestler Please see "option substitution" here: http://pdal.io/apps/pipeline.html#substitutions

https://pdal.io/en/latest/apps/pipeline.html#substitutions

Updated link for those searching for this - I had trouble finding it as I was looking in the other "Pipeline" page :P

@hobu
Copy link
Member

hobu commented Sep 4, 2024

grumble I see we need to fix the link redirects. I opened a ticket at #4491 to deal with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants