Per page totals strategy

[donmurta]

So i'm trying to build up a report that needs some per page aggregation and currently am using a Decoration to build the table (header, content, and footer) with a grid to produce the rows - which is working swimmingly with with the exception of getting my head around how to have a per page summary be reflected in the footer (right now it produces the aggregation of the entire report not a per page total - which makes sense as the content generation happens in a single shot).

Any ideas? Am I missing something obvious? Seems like this is somewhat similar to what's going on with the page numbers - but at a different containment level. Is this what slots will accomplish?

Thanks!

[MarcinZiabek]

Hello, thank you for your comment. I am afraid, it is not supported at the moment. However, I definitely would like to take care of it in the future, once other, higher-prioritized features are done.

If your layout is relatively simple, e.g. the height of each element in the table is constant, and the table always has the same amount of space on the page, it is possible to achieve your case. You can first calculate how many elements will fit on the page, and then chunk your items into groups. This way, you can generate a series of tables with desired subtotal footers, where each table corresponds to a single page.

Your requirement is very interesting. Basically, it creates a somewhat circular dependency between table content and the subtotal part. One cannot be easily calculated without the other. Unless we introduce some additional limiting factors, e.g. subtotal size needs to be constant. I would like to implement it in a most generic way. I don't want to force the developer to follow any visual hierarchy or design. If anybody else needs such functionality, please let me know.

For almost a month, I am thinking about preparing a Dynamic element, where the developer will get general information, e.g. available space and a container. And then, based on the context, it will be possible to generate dynamic content. It is similar to the Canvas element but with access to all QuestPDF goodies. However, it is not simple, as the library uses some internal mechanisms to distribute context, measure stuff, reset elements, etc. I will think about it over time.

@donmurta Can you please propose, how the API may look like in your opinion? We can start working on it and see how it goes :)

[donmurta]

Hey thanks for the response and a suggested workaround @MarcinZiabek. Though not ideal, it will suffice if I can't get the requirement dropped or reworked as I am questioning the end user usefulness of the data being presented this way and will have to get some feedback on if this is a requirement just because it's there in my product already or not. I am well aware this isn't a simple problem so thanks for the consideration!

In terms of how this would actually work, I think there are three parts to it:

• Tagging containers/elements with user data (eg tag the grid (row) with the aggregated totals so far)
• A page break event in which I could retrieve the container that caused the page break and access said tagged data
• A the bare minimum a text element that can generate its contents at a deferred composition time (a fully editable container that composes dynamically would be nice, but I understand the impracticality of this with respect to reflow). I would think an Func with closure over the report would do the trick - but again am aware of the potential for reflow when the text grows too large. My guess is that page numbers would do this in edge cases anyway, but the chance of that happening is likely very limited.

To be honest though... this seems kinda fragile and relies on a few thing working just right and assumptions on what containers will cause page breaks. The more I think about it the more it seems like a kludge on an already pretty clean interface and seems a little hokey. Nonetheless, I'll try and give it a little more consideration over the next week.

Thanks for your time!

[MarcinZiabek]

I know that there are libraries that provide such functionality, e.g. some old Oracle reporting service. Still, to be honest, I have no idea how to achieve something like this with any HTML-to-pdf converter.

As you pointed out, the biggest issue is with the reflow algorithm. For example, to provide support for page numbers (especially the total number of pages), the library needs to recalculate the layout algorithm twice. And there are still corner cases that may produce incorrect results. In some cases (assuming that the library wants perfection, which is not true for QuestPDF), you may be even stuck in an infinite loop. The need of calculating the layouting algorithm twice has forced me to introduce an element resettable state (e.g. the Stack element depends on the binary tree structure, the Text element on the internal queue) as well as additional caching support.

I would like to provide a set of simple and generic elements, e.g. padding or alignment. The page sub-totals is on the opposite side of the spectrum, it is clearly very specific. If there is anything that will take us closer to such a functionality, it would be a Dynamic element. Where the responsibility of the layout consistency would be slightly moved to the developer side. Meaning, the developer would need to generate a set of free-floating containers, then measure their sizes and manually calculate how many of them will be visible on the page. Then, it will be possible to construct a Fluent API tree to describe the content. I have even tried to implement a prototype yesterday but I already see multiple difficult places.

[MarcinZiabek]

@donmurta I have been working on the prototype of the dynamic component. Please remember, this is a very rough implementation and it also needs to wait for other features to be published first (it utilizes functionality that is still in the experimental phase). Therefore, I don't expect it live any time soon but I would like to cooperate with you on the API :)

https://github.com/QuestPDF/QuestPDF/tree/dynamic

This is how to create a Dynamic component:

public class TableWithSubtotals : IDynamic
{
    private ICollection<int> Values { get; }
    private Queue<int> ValuesQueue { get; set; }

    public TableWithSubtotals(ICollection<int> values)
    {
        Values = values;
    }
    
    // layout algorithm is calculated twice, we need a way to reset the queue of numbers
    public void Reset()
    {
        ValuesQueue = new Queue<int>(Values);
    }

    public bool Compose(DynamicContext context, IContainer container)
    {
        var internalQueue = new Queue<int>(ValuesQueue);
        
        container.Box().Border(2).Background(Colors.Grey.Lighten3).Stack(stack =>
        {
            // an expected height of the "summary" footer
            var summaryHeight = 40f;
            
            var totalHeight = summaryHeight;
            var total = 0;
            
            while (internalQueue.Any())
            {
                var value = internalQueue.Peek();

                // create an unattached container with any custom content
                var structure = context.Content(content =>
                {
                    content
                        .Padding(10)
                        .Text(value);
                });

                // measure newly generated container against available space
                var structureHeight = structure.Measure().Height;

                // check if there is enough space for this container
                if (totalHeight + structureHeight > context.AvailableSize.Height)
                    break;

                // if yes, update the height and sum
                internalQueue.Dequeue();
                totalHeight += structureHeight;
                total += value;
 
                // then attach the container to the stack
                stack.Item().Border(1).Element(structure);
            }
            
            // create structure with the sub-page summary
            stack
                .Item()
                .ShowEntire()
                .Border(2)
                .Background(Colors.Grey.Lighten1)
                .Padding(10)
                .Text($"Total: {total}", TextStyle.Default.SemiBold());
        });

        // in measure step: keep current queue structure
        // in draw step: remove drawn elements from the queue
        if (context.IsDrawStep)
            ValuesQueue = internalQueue;
        
        // return true, if there are more items to be drawn on the next page
        return internalQueue.Any();
    }
}

And this is how to use it:

var values = Enumerable.Range(0, 15).ToList();

container
    .Background(Colors.White)
    .Padding(25)
    .Decoration(decoration =>
    {
        decoration
            .Header()
            .PaddingBottom(5)
            .Text(text =>
            {
                text.DefaultTextStyle(TextStyle.Default.SemiBold().Color(Colors.Blue.Darken2).Size(16));
                text.Span("Page ");
                text.CurrentPageNumber();
                text.Span(" of ");
                text.TotalPages();
            });
        
        decoration
            .Content()
            .Dynamic(new TableWithSubtotals(values));
    });

Results:

What I am the most concerned about, is how to design an API. Clearly, the responsibility of calculating the proper layout is moved from the library to the developer. And not everything is obvious at this stage. It needs to be super clear how everything works otherwise nobody will be able to use this functionality. The library can provide some helper methods to deal with most layouting tasks, e.g. to take elements until their total height exceeds some threshold. What do you think?

[donmurta]

Oh wow @MarcinZiabek this much surpassed my expectations!

From a consumer point of view - this is exactly what I'd need, especially with respect to having temporary components that may or may not get consumed.

Behaviour wise, its pretty (functionally) clear to me for the most part especially with respect to the provided example, with the exception of IsDrawStep. If the Component is able to be Reset() why would it care if it's being drawn or measured - as an implementer I'd expect the composition behaviour to be the same regardless.

Naming wise I'd expect the interface to be an IDynamicComponent as it follows a similar signature to IComponent, as well as DynamicContext.Content to return an IContainer which would then expose Measure() publicly.

[MarcinZiabek]

Thank you for your feedback :)

Behaviour wise, its pretty (functionally) clear to me for the most part especially with respect to the provided example, with the exception of IsDrawStep. If the Component is able to be Reset() why would it care if it's being drawn or measured - as an implementer I'd expect the composition behaviour to be the same regardless.

This is not that simple. Some elements may want to compose DynamicComponent multiple times per page, testing various size constraints. For example, the Row element will first measure all its children, then find a child with a maximum height to finally draw all children with this one, consistent height. Therefore, the Compose method may be called multiple times and the developer needs some flag to determine if the state should be mutated (Measure vs Draw phases). I know that the IsDrawStep is far from being perfect but it serves its purpose and we just need to find something better.

Naming wise I'd expect the interface to be an IDynamicComponent as it follows a similar signature to IComponent, as well as DynamicContext.Content to return an IContainer which would then expose Measure() publicly.

Yes, IDynamicComponent makes sense :)

However, the DynamicContext.Content returns special type of IContainer that hides the complexity of measuring containers, especially hiding not needed paging capability - as we call the IDynamicComponent every page.

[donmurta]

Fair enough, and no it never is that simple :) Just started looking at the internals of the library so my comments reflect a black box view mostly - thanks for your patience.

Some elements may want to compose DynamicComponent multiple times per page, testing various size constraints.

Wouldn't they just compose it and call Reset() between compositions in this case? I guess where I'm coming from here is that the library should know when it wants to reevaluate something (and already does), where as the consumer can compose and entirely different element based on this flag - which seems bad. Additionally when measuring the example - the composition is never going to reflect the second page as the queue is never going to be reduced, not sure this would matter for the most part if the dynamic component is consistent but could see it falling apart if it wasn't.

[MarcinZiabek]

Hello @donmurta :)

A small update from my side. I have tried to improve the API and prepare a more sophisticated example, something that may be used in the documentation at some point. Please take a look and let me know what do you think:

public class OrderItem
{
    public string ItemName { get; set; } = Placeholders.Label();
    public int Price { get; set; } = Placeholders.Random.Next(1, 11) * 10;
    public int Count { get; set; } = Placeholders.Random.Next(1, 11);
}

public class OrdersTable: IDynamicComponent
{
    private ICollection<OrderItem> Items { get; }
    private ICollection<OrderItem> ItemsLeft { get; set; }
    
    public OrdersTable(ICollection<OrderItem> items)
    {
        Items = items;
    }
    
    public void Compose(DynamicContext context, IDynamicContainer container)
    {
        if (context.Operation == DynamicLayoutOperation.Reset)
        {
            ItemsLeft = new List<OrderItem>(Items);
            return;
        }

        var header = ComposeHeader(context);
        var sampleFooter = ComposeFooter(context, Enumerable.Empty<OrderItem>());
        var decorationHeight = header.Size.Height + sampleFooter.Size.Height;
        
        var rows = GetItemsForPage(context, decorationHeight).ToList();
        var footer = ComposeFooter(context, rows.Select(x => x.Item));

        if (ItemsLeft.Count > rows.Count)
            container.HasMoreContent();
        
        if (context.Operation == DynamicLayoutOperation.Draw)
            ItemsLeft = ItemsLeft.Skip(rows.Count).ToList();

        container.Box().Decoration(decoration =>
        {
            decoration.Header().Element(header);
            
            decoration.Content().Box().Stack(stack =>
            {
                foreach (var row in rows)
                    stack.Item().Element(row.Element);
            });

            decoration.Footer().Element(footer);
        });
    }

    private IDynamicElement ComposeHeader(DynamicContext context)
    {
        return context.CreateElement(element =>
        {
            element
                .BorderBottom(1)
                .BorderColor(Colors.Grey.Darken2)
                .Padding(5)
                .Row(row =>
                {
                    var textStyle = TextStyle.Default.SemiBold();

                    row.ConstantColumn(30).Text("#", textStyle);
                    row.RelativeColumn().Text("Item name", textStyle);
                    row.ConstantColumn(50).AlignRight().Text("Count", textStyle);
                    row.ConstantColumn(50).AlignRight().Text("Price", textStyle);
                    row.ConstantColumn(50).AlignRight().Text("Total", textStyle);
                });
        });
    }
    
    private IDynamicElement ComposeFooter(DynamicContext context, IEnumerable<OrderItem> items)
    {
        var total = items.Sum(x => x.Count * x.Price);

        return context.CreateElement(element =>
        {
            element
                .Padding(5)
                .AlignRight()
                .Text($"Subtotal: {total}$", TextStyle.Default.Size(14).SemiBold());
        });
    }
    
    private IEnumerable<(OrderItem Item, IDynamicElement Element)> GetItemsForPage(DynamicContext context, float decorationHeight)
    {
        var totalHeight = decorationHeight;
        var counter = Items.Count - ItemsLeft.Count + 1;
        
        foreach (var orderItem in ItemsLeft)
        {
            var element = context.CreateElement(content =>
            {
                content
                    .BorderBottom(1)
                    .BorderColor(Colors.Grey.Lighten2)
                    .Padding(5)
                    .Row(row =>
                    {
                        row.ConstantColumn(30).Text(counter++);
                        row.RelativeColumn().Text(orderItem.ItemName);
                        row.ConstantColumn(50).AlignRight().Text(orderItem.Count);
                        row.ConstantColumn(50).AlignRight().Text($"{orderItem.Price}$");
                        row.ConstantColumn(50).AlignRight().Text($"{orderItem.Count*orderItem.Price}$");
                    });
            });

            var elementHeight = element.Size.Height;
                
            if (totalHeight + elementHeight > context.AvailableSize.Height)
                break;
                
            totalHeight += elementHeight;
            yield return (orderItem, element);
        }
    }
}

---

1. Changed name of the method used for creating unattached elements to CreateElement().
2. Simplified measuring element's size. Now the size is just available as a property.
3. In the example, there is no assumption regarding the height of the header and footer. They are additionally measured to stay safe.
4. Creating/measuring/selecting table rows has been moved to a separate method.
5. The context provides now type of current operation.

• Reset - the entire document is being re-rendered. The state of the component should be reset.
• Measure - the component is being measured. The produced container will not be included in the final structure. Don't modify the state.
• Draw - the component is being drawn based on the available space. Please modify the state, as the next operation will take place on the next page.

[donmurta]

Again I'm quite impressed by the changes @MarcinZiabek! I think 1,2,3 & 4 all make it simpler to consume and a lot easier to understand what's going on.

5 - Something still doesn't feel quite right about this as Resetting seems to be a completely different operation than draw and measure - though I do prefer this for the delineation between draw and measure vs the boolean property. Anyway - feels like nitpicking, it still seems like a good implementation and the difference between draw and measure is a lot more clear now too.

On another note I needed a charting/graphing solution on a few of my reports. Grabbed Microcharts and was able to drop it into the canvas element and had it rendering graphs in about 10 minutes. It really is a joy working with this library - and am looking forward to seeing what else it can do!

[MarcinZiabek]

@donmurta I am so happy to hear that the integration with a charting library was easy! I would really like to have a documentation section that shows how to perform such integration. The goal is to show other developers that it is possible and give some hints on how to start :)

What I need is:

1. A code example in the QuestPdf.Examples project.
2. A short article describing the chart implementation in the documentation.

Do you think, you can help me with this?

[donmurta]

Ya I can do that, though it will be a week or so due to other commitments. I’ll setup a PR once it’s done. Cheers!

[MarcinZiabek]

As such tutorial is not related to any release in particular, please feel free to create one in your spare time, no rush required :) Once again,thank you :)

[MarcinZiabek]

Yes, I agree, there is still a possibility to improve. Once other parts of the library are publicly available, let's come back to this discussion and make it even better :) Time is needed to think about all corner cases and potential optimizations :)

[donmurta]

No problem - thanks again!

[MarcinZiabek]

Hello 😁 I have just released the QuestPDF 2022.5 version that includes this feature, finally! Please take a look here

[donmurta]

Awesome! Thanks for the update Marcin, looks fantastic from an initial glance at the documentation, I’ll hope to get into integrating the new version this weekend. Regards, Donald Murta

…

On May 9, 2022, at 9:07 AM, Marcin Ziąbek ***@***.***> wrote:  Hello 😁 I have just released the QuestPDF 2022.5 version that includes this feature, finally! Please take a look here — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[MarcinZiabek]

That's great! I am very sorry that it took so much time. This time I have no excuses 🤣

[crabouif]

Hello, first I would like to say that is the best awesome PDF generator I found until now ! Thanks for that !!
I just have a problem related to this thread.
I would like to make a report that makes sum of item amount in each page but cumulative.

• page 1 -> 11 items -> sum 11
• page 2 -> 10 items -> sum 21
• etc.

I tried to pass a dictionary in the dynamic to store the amount by page and give that to my page footer, but I have strange behavior:

• page 1 -> 11 items -> sum 11
• page 2 -> 24 items -> sum 48 => Here is the problem...

I added some trace to understand behavior, it seems it's a bit chaotic:

Page 1 - 11 rows - Sum = 11
Page 1 - 22 rows - Sum = 22
Page 2 - 22 rows - Sum = 22
Page 2 - 22 rows - Sum = 22
Page 3 - 22 rows - Sum = 22
Page 3 - 5 rows - Sum = 5
Page 4 - 5 rows - Sum = 5
Page 1 - 13 rows - Sum = 13
Page 1 - 24 rows - Sum = 24
Page 2 - 24 rows - Sum = 24
Page 2 - 23 rows - Sum = 23
Page 3 - 23 rows - Sum = 23

Here is the piece of code:

        public DynamicComponentComposeResult Compose(DynamicContext context)
        {
            var header = ComposeHeader(context);
            var decorationHeight = header.Size.Height;

            var rows = GetServicesForPage(context, decorationHeight).ToList();
            if (context.PageNumber > 0)
            {
                File.AppendAllLines("printing.log", new[] { $"Page {context.PageNumber} - {rows.Count} rows - Sum = {rows.Sum(row => row.Service.Amount)}" });
                AmountsByPage[context.PageNumber] = rows.Sum(row => row.Service.Amount);
            }

            var content = context.CreateElement(
                container =>
                {
                    container.MinimalBox().Decoration(
                        decoration =>
                        {
                            decoration.Before().Element(header);
                            decoration.Content().Column(
                                column =>
                                {
                                    foreach (var row in rows)
                                        column.Item().Element(row.Element);
                                });
                        });
                });

            State = new LineTableState
            {
                ShownServicesCount = State.ShownServicesCount + rows.Count
            };

            return new DynamicComponentComposeResult
            {
                Content = content,
                HasMoreContent = State.ShownServicesCount < Services.Count
            };
        }

To have the accumulated amount in each pages I do that in the footer (also IDynamicComponent)

var amount = Enumerable.Range(1, currentPage).Aggregate(0m, (sum, page) => sum + _amountsByPage[page]);

And the PDF result is:
Page 1 => Correct

Page 2 => Wrong, should be 37

Regards.
Pascal