Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use unicode characters to display tables #36857

Merged
merged 3 commits into from
Jan 14, 2024
Merged

use unicode characters to display tables #36857

merged 3 commits into from
Jan 14, 2024

Conversation

fchapoton
Copy link
Contributor

@fchapoton fchapoton commented Dec 10, 2023 •
edited

This is using unicode box-drawing characters to make better-looking tables.

📝 Checklist

  • The title is concise, informative, and self-explanatory.
  • The description explains in detail what this PR is about.

⌛ Dependencies

@kwankyu
Copy link
Collaborator

kwankyu commented Dec 11, 2023

This looks nice. But pdf doc building failure is genuine. pdftex with inputenc package cannot handle those unicode characters.

@kwankyu
Copy link
Collaborator

kwankyu commented Dec 11, 2023 •
edited

The chatbot is useful here too. I like solution 4. We may switch to xetex.

The error you're encountering with \usepackage[utf8]{inputenc} typically occurs when there's a Unicode character in your text that isn't recognized by LaTeX. The character ╞ corresponds to Unicode U+255E, which might not be supported by the inputenc package.

To resolve this issue, you can try a few things:

  1. Replace the Character: Identify where this character is in your text and replace it with an appropriate alternative or remove it if it's not necessary.

  2. Use a Different Encoding: Instead of utf8, try using a different encoding that might support this character. For example, you could try utf8x or latin1 to see if they handle the character better.

  3. Escape the Character: Sometimes, you can escape problematic characters using LaTeX's escape sequences. For example, you might try replacing ╞ with \textSFNC or another suitable LaTeX command.

  4. Use a Unicode-aware TeX Engine: Consider switching to a TeX engine that supports Unicode natively, like XeTeX or LuaTeX. These engines can handle a wider range of Unicode characters without requiring special packages like inputenc.

@kwankyu
Copy link
Collaborator

kwankyu commented Dec 11, 2023

Related ticket:

#18370

@kwankyu kwankyu mentioned this pull request Dec 11, 2023
Merged
5 tasks
@fchapoton
Copy link
Contributor Author

fchapoton commented Dec 12, 2023 •
edited

We also have

$ git grep -c "DeclareUnicode" src
src/doc/fr/tutorial/conf.py:1
src/doc/ru/tutorial/conf.py:1
src/sage_docbuild/conf.py:160

@kwankyu
Copy link
Collaborator

kwankyu commented Dec 12, 2023

Yes. They may be deleted.

@kwankyu
Copy link
Collaborator

kwankyu commented Dec 12, 2023 •
edited

We also have

$ git grep -c "DeclareUnicode" src
src/doc/fr/tutorial/conf.py:1
src/doc/ru/tutorial/conf.py:1
src/sage_docbuild/conf.py:160

"DeclareUnicode" in the first two files are already commented out and not used. I deleted them in 708c100

A bunch of "DeclareUnicode" in src/sage_docbuild/conf.py is conditioned on \ifPDFTeX. So we may leave them untouched.

@dimpase
Copy link
Member

dimpase commented Dec 12, 2023

@fchapoton - I think we should just switch to xelatex/lualatex pdf builder, AND also drop (pdf)latex builder (as inferior, obsolete 30-years old tech). I advocate for this on #36861 (@kwankyu wants to keep (pdf)latex builder - but I don't see why)

@kwankyu
Copy link
Collaborator

kwankyu commented Dec 14, 2023

#36861 works well with this PR. You may use it as a dependency of this PR.

vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 22, 2023
sagemathgh-36861: Use lualatex for better unicode support
92aeb3f 
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

to support sagemath#36857.

There have been numerous problems due to limited support of unicode in
pdflatex. I see no reason why we should not switch to modern tex engine
lualatex, as suggested in

https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-
latex_engine

We set lualatex as the default engine for
- building the sage documentation, in `sage_docbuild/`
- rendering pdf images, in `src/sage/misc/latex.py`

In  `src/sage/misc/latex.py`, the default engine is determined by
availability of the engines  with the order of preference: lualatex,
xelatex, pdflatex.

Along the way, we
- remove latex-related code deprecated in sagemath#32650.
- make lots of cosmetic edits in `src/sage/misc/latex.py`
- add support "lualatex" as an alternative to "[pdf|xe]latex" in
`src/sage/misc/latex.py`

We add optional (dummy) packages `texlive_luatex`, `free_fonts`, `xindy`
as new dependencies of `sagemath_doc_pdf` in addition to `texlive`
optional package. It seems that depending on the version and the
platform, `texlive` may already include the new dependencies. If not,
you need to install the new dependencies (as `_recommended` by the dummy
package)

On Ubuntu, the requirement is fulfilled by
```
sudo apt install texlive texlive-luatex fonts-freefont-otf xindy
```

The built pdf doc is available here: https://deploy-livedoc--
sagemath.netlify.app

To test `view` via lualatex, for example,
```
latex.extra_preamble(r"\usepackage{fontspec}\setmainfont{Arial}\setmonof
ont{Arial}")
view('Εύρηκα')
```
you should build sage on your local platform with this PR. You may need
to install `texlive-full` before testing. It seems that `texlive` (2019)
is not enough even for the `view` via pdftex.

sage-devel thread seeking Unicode testers:
https://groups.google.com/g/sage-devel/c/tG2LK6Jvw0I


<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36861
Reported by: Kwankyu Lee
Reviewer(s): Dima Pasechnik, Kwankyu Lee, Matthias Köppe
vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 23, 2023
sagemathgh-36861: Use lualatex for better unicode support
20079a2 
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

to support sagemath#36857.

There have been numerous problems due to limited support of unicode in
pdflatex. I see no reason why we should not switch to modern tex engine
lualatex, as suggested in

https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-
latex_engine

We set lualatex as the default engine for
- building the sage documentation, in `sage_docbuild/`
- rendering pdf images, in `src/sage/misc/latex.py`

In  `src/sage/misc/latex.py`, the default engine is determined by
availability of the engines  with the order of preference: lualatex,
xelatex, pdflatex.

Along the way, we
- remove latex-related code deprecated in sagemath#32650.
- make lots of cosmetic edits in `src/sage/misc/latex.py`
- add support "lualatex" as an alternative to "[pdf|xe]latex" in
`src/sage/misc/latex.py`

We add optional (dummy) packages `texlive_luatex`, `free_fonts`, `xindy`
as new dependencies of `sagemath_doc_pdf` in addition to `texlive`
optional package. It seems that depending on the version and the
platform, `texlive` may already include the new dependencies. If not,
you need to install the new dependencies (as `_recommended` by the dummy
package)

On Ubuntu, the requirement is fulfilled by
```
sudo apt install texlive texlive-luatex fonts-freefont-otf xindy
```

The built pdf doc is available here: https://deploy-livedoc--
sagemath.netlify.app

To test `view` via lualatex, for example,
```
latex.extra_preamble(r"\usepackage{fontspec}\setmainfont{Arial}\setmonof
ont{Arial}")
view('Εύρηκα')
```
you should build sage on your local platform with this PR. You may need
to install `texlive-full` before testing. It seems that `texlive` (2019)
is not enough even for the `view` via pdftex.

sage-devel thread seeking Unicode testers:
https://groups.google.com/g/sage-devel/c/tG2LK6Jvw0I


<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
Resolves sagemath#18370
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36861
Reported by: Kwankyu Lee
Reviewer(s): Dima Pasechnik, Kwankyu Lee, Matthias Köppe
vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 24, 2023
sagemathgh-36861: Use lualatex for better unicode support
d35ab3a 
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

to support sagemath#36857.

There have been numerous problems due to limited support of unicode in
pdflatex. I see no reason why we should not switch to modern tex engine
lualatex, as suggested in

https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-
latex_engine

We set lualatex as the default engine for
- building the sage documentation, in `sage_docbuild/`
- rendering pdf images, in `src/sage/misc/latex.py`

In  `src/sage/misc/latex.py`, the default engine is determined by
availability of the engines  with the order of preference: lualatex,
xelatex, pdflatex.

Along the way, we
- remove latex-related code deprecated in sagemath#32650.
- make lots of cosmetic edits in `src/sage/misc/latex.py`
- add support "lualatex" as an alternative to "[pdf|xe]latex" in
`src/sage/misc/latex.py`

We add optional (dummy) packages `texlive_luatex`, `free_fonts`, `xindy`
as new dependencies of `sagemath_doc_pdf` in addition to `texlive`
optional package. It seems that depending on the version and the
platform, `texlive` may already include the new dependencies. If not,
you need to install the new dependencies (as `_recommended` by the dummy
package)

On Ubuntu, the requirement is fulfilled by
```
sudo apt install texlive texlive-luatex fonts-freefont-otf xindy
```

The built pdf doc is available here: https://deploy-livedoc--
sagemath.netlify.app

To test `view` via lualatex, for example,
```
latex.extra_preamble(r"\usepackage{fontspec}\setmainfont{Arial}\setmonof
ont{Arial}")
view('Εύρηκα')
```
you should build sage on your local platform with this PR. You may need
to install `texlive-full` before testing. It seems that `texlive` (2019)
is not enough even for the `view` via pdftex.

sage-devel thread seeking Unicode testers:
https://groups.google.com/g/sage-devel/c/tG2LK6Jvw0I


<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
Resolves sagemath#18370
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36861
Reported by: Kwankyu Lee
Reviewer(s): Dima Pasechnik, Kwankyu Lee, Matthias Köppe
vbraun pushed a commit to vbraun/sage that referenced this pull request Dec 26, 2023
sagemathgh-36861: Use lualatex for better unicode support
8702356 
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

to support sagemath#36857.

There have been numerous problems due to limited support of unicode in
pdflatex. I see no reason why we should not switch to modern tex engine
lualatex, as suggested in

https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-
latex_engine

We set lualatex as the default engine for
- building the sage documentation, in `sage_docbuild/`
- rendering pdf images, in `src/sage/misc/latex.py`

In  `src/sage/misc/latex.py`, the default engine is determined by
availability of the engines  with the order of preference: lualatex,
xelatex, pdflatex.

Along the way, we
- remove latex-related code deprecated in sagemath#32650.
- make lots of cosmetic edits in `src/sage/misc/latex.py`
- add support "lualatex" as an alternative to "[pdf|xe]latex" in
`src/sage/misc/latex.py`

We add optional (dummy) packages `texlive_luatex`, `free_fonts`, `xindy`
as new dependencies of `sagemath_doc_pdf` in addition to `texlive`
optional package. It seems that depending on the version and the
platform, `texlive` may already include the new dependencies. If not,
you need to install the new dependencies (as `_recommended` by the dummy
package)

On Ubuntu, the requirement is fulfilled by
```
sudo apt install texlive texlive-luatex fonts-freefont-otf xindy
```

The built pdf doc is available here: https://deploy-livedoc--
sagemath.netlify.app

To test `view` via lualatex, for example,
```
latex.extra_preamble(r"\usepackage{fontspec}\setmainfont{Arial}\setmonof
ont{Arial}")
view('Εύρηκα')
```
you should build sage on your local platform with this PR. You may need
to install `texlive-full` before testing. It seems that `texlive` (2019)
is not enough even for the `view` via pdftex.

sage-devel thread seeking Unicode testers:
https://groups.google.com/g/sage-devel/c/tG2LK6Jvw0I


<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
Resolves sagemath#18370
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36861
Reported by: Kwankyu Lee
Reviewer(s): Dima Pasechnik, Kwankyu Lee, Matthias Köppe
@github-actions GitHub Actions
Copy link

Documentation preview for this PR (built with commit d03c94c; changes) is ready! 🎉

kwankyu
kwankyu approved these changes Dec 27, 2023
View reviewed changes
Copy link
Collaborator

@kwankyu kwankyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

vbraun pushed a commit to vbraun/sage that referenced this pull request Jan 2, 2024
sagemathgh-36857: use unicode characters to display tables
73468b7 
    
This is using unicode box-drawing characters to make better-looking
tables.

### 📝 Checklist

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.

### ⌛ Dependencies

- sagemath#36861

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
    
URL: sagemath#36857
Reported by: Frédéric Chapoton
Reviewer(s): Kwankyu Lee
vbraun pushed a commit to vbraun/sage that referenced this pull request Jan 5, 2024
sagemathgh-36857: use unicode characters to display tables
97ce48c 
    
This is using unicode box-drawing characters to make better-looking
tables.

### 📝 Checklist

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.

### ⌛ Dependencies

- sagemath#36861

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
    
URL: sagemath#36857
Reported by: Frédéric Chapoton
Reviewer(s): Kwankyu Lee
@vbraun vbraun merged commit 281be08 into sagemath:develop Jan 14, 2024
19 checks passed
@mkoeppe mkoeppe added this to the sage-10.3 milestone Jan 14, 2024
@fchapoton fchapoton deleted the unicode_art_table branch January 15, 2024 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kwankyu kwankyu kwankyu approved these changes

Assignees
No one assigned
Labels
c: documentation
Projects
None yet
Milestone
sage-10.3
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@fchapoton @kwankyu @dimpase @vbraun @mkoeppe