Statistics of Xournal++ files (.xopp)

[eldipa]

For the developers and curious people. Do you know how many strokes are in a typical Xournal++ file? How the pen pressure is distributed? How large are the images that people attach to?

Working on #937 , before designing the new file format, the first step is to know the current one.

Here are some plots from a 48 files dataset collected from 9 different public sources.

How large are the texts, images and Tex-images?

As expected images and Tex-images are quite heavy, ~10K and ~100K. Note however that they are stored as Base64 so the numbers are ~1.33 times the real sizes.

Texts are much smaller but they are quite more numerous.

Note: both axes are in log scale

How many coordinates a stroke has?

Most of the strokes have around 100 coordinates but some outliers almost reach 1000!

Do you see the weird pattern of missing vertical lines in the left part of plot? This is because the strokes have a even number of coords (they are x-y points)

Note: the x axis is in log scale

And how many strokes do we have?

If you ask per page you will get this:

If you ask in total per sample file you will get this (x axis in log scale):

Some pages have more than 1000 strokes and some samples docs have more than 10000 strokes (and it may reach 100000)

Talking of strokes, how are theirs x-y coords distributed?

Mean and standard deviation for x-coord (y-coord is similar):

Is any relationship between x and y?

No strong correlation but there are some features: it looks that certain y values are more common than others (look carefully the plot and you will see some darker horizontal lines)

You may also spotted some weirdness: why there is a missing chunk in the plot? in the range of (600 - 800) x and y axes?

My intuition says that most of the pages are 800x600 and sometimes the pages are in landscape orientation giving the apparent size of 600x800.

What about the pen pressure?

The darker point at mean and std equals to 0 corresponds to the strokes that they don't have pressure points

Distribution of background page types.

Looks like solid is most common but pdf is not far away.

Do you want contribute?

Share your .xopp docs and I'll add it to the dataset. With more samples we can get a better understanding of how you and others use Xournal++ and we can design the software better.

[netlimpopo]

Hello!
I sent you some .xopp files. To minimise data consumption I generally use a lot of text and geometric shapes. I might not be a stereotypical user I guess. I am mainly use Xournal++ for teaching maths in high school, especially when there are figures to draw. I prepare quite a few text parts in advance. Here are my files:
https://schlosstraunsee-my.sharepoint.com/:f:/g/personal/a_zettler_schlosstraunsee_at/EgSrpe_kUXlCoUrOnBooovMBymPllcjz6XSPQ3x7jnO5Tg?e=Fd0oj0
Thank you for looking into the new fileformat and the interesting stats so far.

[eldipa]

Thanks for the files! I will add them to the second analysis round.

[LuAViVA]

Hi

I´m a student and use xournal a lot. Almost all my notes are in xournal and I do homeworks, practice and much more in it. I gathered some documents in here:

https://1drv.ms/f/s!ApoA9il2jxFHg_F2kld8L9sLZc0taA?e=jqJdal

With big documents it starts to bug and slow down, I usually download what I had to a pdf and create a new document with that so it doesn´t slow down as much. Hope it helps.

[eldipa]

Hey thanks for the files!,

Yes I had the same issue than you. I'm studying maths/proba and doing a bunch of exercises. Xournal++ is really user friendly but with more notes/exercises/text that I write the files get bigger and there is too much lag eventually.

Most of the problems are around how Xournal++ writes the .xopp file. We cannot do much in its current form but we can design a better format and eventually make Xournal++ take advantage of it.

[rolandlo]

Sorry for interrupting the interesting discussion here, but can we just clearly distinguish between Xournal and Xournal++? Although Xournal++ is a rewrite of Xournal, the two codebases are almost completely distinct and what applies to Xournal++ may not apply to Xournal and vice versa. I guess this discussion is all about Xournal++, but I could be wrong.

[eldipa]

You are right, the discussion is about Xournal++. I guessed that it was implicitly implied but it is better to be explicit and avoid any confusion.

I updated the title and my comments.

[Badhunter0303]

An example of the way I use xournal++ to study math

https://drive.google.com/file/d/18jYy4cM3EE3j547lDJvDggMNepvljqAQ/view?usp=drive_link

[eldipa]

Thanks!

[Badhunter0303]

8° Ano Mat. Bianchini 2015 Cap. 1 - Retas e ângulos Atividades.pdf

[eldipa]

Hi! Thanks for the contribution! Do you have the .xopp file too? We use more that file than the .pdf to get better statistics.

[Badhunter0303]

@eldipa how do I share the xopp file?

[vladQll]

I don't know if what I'm going to ask this linked , but if it's about the pressure sensitivity but when I write I often have the feeling that it doesn't feel so natural , previously I had a wacom ctl4100 tablet and the strokes came out distorted , or when I tried to write with a small calligraphy they came out big strokes or if I tried to write fast like on paper , the strokes didn't have the shape I wanted , then I found out that it could be related to the response times of the pen , the wacom I mentioned had 133PPS, so I bought another graphic tablet a little more expensive a Huion dial 2 that claims to have 300PPS and to my surprise it improved a little, but I still notice that some strokes visually look distorted, I do not want to be wrong because I do not know what would be the cause, or maybe it has to do with some issue of smoothing, or rendering tambien i don't know why whenever i start or finish drawing some letters start or end with a sharp point like an l or p .
but for example here I attach some images of xournal++ and an online whiteboard excalidraw

xournal++:

Excalidraw:

[eldipa]

Hi @vladQll , you will get better chance to have an better answer if you create a new discussion or you open a new issue. Give it a try!

Nevertheless let me give you a possible explanation of what you are seeing in Xournal++ and Excalidraw. I'm not an expert on this but my intuition says that it is not a problem with the pressure of the pen but how the programs interpret the input.

In Excalidraw it seems that it takes the input (from your pen) and smooths it to look more like a hand-made calligraphy. I saw this in other programs like Krita.
In Xournal++ it is not happening such processing. I don't know if there is a setting to enable the feature or if such it is even implemented.

Again, you will have better chances opening a new discussion or a new issue.

[ahcox]

Would it be possible to share the set of files?

[eldipa]

Absolutely: https://github.com/eldipa/xoz-dataset

Caveat: the readme in that repository describes how I realized the analysis of the files. However, I wrote it a few weeks after doing the real analysis and the scripts were slightly tweaked in time to time so what the readme says it may not work out of the box.