-
Notifications
You must be signed in to change notification settings - Fork 614
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
COCO Reader pixelwise_masks Emtpy Output #5404
Comments
polygon_masks=True returns valid poly and vert data btw. In the meantime, I will probably use this to render image with a plugin that runs over these with cv::fillConvexPoly. Could probably be interesting see how well that works compared to doing it manually in COCOReader::PixelwiseMasks, and potentially replace it... |
If In regards to the segmentation image itself, it is initialized with zeros, but the poly masks may not cover all pixels, and there might be a zero class, so there will be incorrect loss applied to areas that should be I might still try testing throughput of the current RLE implementation and cv::fillConvexPoly to simplify the codebase and remove pycocotools may be useful. RLE is not the prettiest thing to look at, I'll have to have a deeper understanding to figure out how to change from class to mask id. |
Current plan is operator rasterize_vert_poly that rasterizes from vert/poly list, will basically follow the meta algorithm that works in normal python below. mask = np.zeros(image.shape[:-1], dtype=np.int32)
vert_cpu *= mask.shape
vert_cpu = vert_cpu.astype(np.int32)
for i, poly_ in enumerate(poly_cpu, 1):
points = vert_cpu[poly_[1] : poly_[2]]
mask = cv2.fillConvexPoly(mask, points, i) |
Without actually benchmarking any performance difference, I have a plugin that works as expected. |
Hi @5had3z, Thank you for reporting the bug. Yes, indeed it is about the mishandling of the |
I had my blinders on and didn't stop to think I need panoptic, not instance segmentation, so I need to make a COCO panoptic reader plugin anyway. I think there is some overlap with my existing cityscapes plugins, if there is interest reading coco panoptic format I can upstream this, but there will probably have to be some lenghty design discussions. FYI the post-hoc rasterize Verticies + Poly is relatively simple to implement, however I haven't benchmarked check if its eggregiously slower than the existing impl. using ConstTensor = dali::ConstSampleView<dali::CPUBackend>;
using Tensor = dali::SampleView<dali::CPUBackend>;
void rasterizeVertPoly(ConstTensor polyTensor, ConstTensor vertTensor, Tensor maskTensor, bool normCoords)
{
struct Poly
{
int maskIdx;
int startIdx;
int endIdx;
};
const auto outShape = maskTensor.shape();
cv::Mat maskMat(outShape[0], outShape[1], CV_32S, maskTensor.raw_mutable_data());
maskMat.setTo(-1); // FIXME user defined void area
std::span polyData(static_cast<const Poly*>(polyTensor.raw_data()), polyTensor.shape()[0]);
std::span vertData(static_cast<const cv::Point2f*>(vertTensor.raw_data()), vertTensor.shape()[0]);
std::vector<cv::Point> vertices(vertData.size()); // Transform poly points from float to int
if (normCoords)
{
std::transform(std::execution::unseq, vertData.begin(), vertData.end(), vertices.begin(),
[h = outShape[0], w = outShape[1]](cv::Point2f p) { return cv::Point(p.x * w, p.y * h); });
}
else
{
std::transform(std::execution::unseq, vertData.begin(), vertData.end(), vertices.begin(),
[](cv::Point2f p) { return static_cast<cv::Point>(p); });
}
for (const auto& polyPoint : polyData)
{
cv::Point* start = vertices.data() + polyPoint.startIdx;
const int nPoints = polyPoint.endIdx - polyPoint.startIdx;
cv::fillConvexPoly(maskMat, start, nPoints, polyPoint.maskIdx);
}
}
template <>
void RasterizeVertPoly<dali::CPUBackend>::RunImpl(dali::Workspace& ws)
{
const auto& polyIn = ws.Input<dali::CPUBackend>(0);
const auto& vertIn = ws.Input<dali::CPUBackend>(1);
auto& maskOut = ws.Output<dali::CPUBackend>(0);
auto& tPool = ws.GetThreadPool();
for (int sampleId = 0; sampleId < ws.GetRequestedBatchSize(0); ++sampleId)
{
tPool.AddWork([&, sampleId](int)
{ rasterizeVertPoly(polyIn[sampleId], vertIn[sampleId], maskOut[sampleId], mNormalizedCoords); });
}
tPool.RunAll();
} |
Hi @5had3z, Thank you for sharing your implementation. The current one is an external contribution and was not benchmarked thoroughly as well. If you have some spare time feel free to compare both your implementations to see which one yields better perf results.
|
Version
1.35.0
Describe the bug.
When enabling pixelwise_masks (with intention to use for panoptic sementation), the output image is only zeros (with sometimes a stray non-zero pixel, which is also concerning).
I've tried this with both the normal coco dataset, and with examples in DALI_extra, both have the same behaviour.
I'm also not sure what the format of pixelwise_masks is meant to be. The index of the segment?
Minimum reproducible example
Relevant log output
No response
Other/Misc.
COCO Reader tutorial also needs to be updated to
from nvidia.dali.types import DALIImageType
instead ofimport nvidia.dali.types as types
with its usage infn.decoders.image
.Check for duplicates
The text was updated successfully, but these errors were encountered: