Text formatting

[Exagone313]

Hi,

I worked a bit to support text formatting for ** to <strong> and * to <em>. You can find a demo here.

I'm wondering if such feature would be acceptable? I would submit a PR if yes. Here is my current diff, that works only for messages sent on the current instance (this code isn't ready for prod, I'd need some help for that):

diff --git a/app/lib/formatter.rb b/app/lib/formatter.rb
index da7ad202..9cf07400 100644
--- a/app/lib/formatter.rb
+++ b/app/lib/formatter.rb
@@ -14,6 +14,7 @@ class Formatter
 
     html = status.text
     html = encode(html)
+    html = strong_em_html(html)
     html = simple_format(html, {}, sanitize: false)
     html = html.gsub(/\n/, '')
     html = link_urls(html)
@@ -36,6 +37,7 @@ class Formatter
     return reformat(account.note) unless account.local?
 
     html = encode(account.note)
+    html = strong_em_html(html)
     html = link_urls(html)
     html = link_accounts(html)
     html = link_hashtags(html)
@@ -98,4 +100,8 @@ class Formatter
   def mention_html(match, account)
     "#{match.split('@').first}<a href=\"#{TagManager.instance.url_for(account)}\" class=\"h-card u-url p-nickname mention\">@<span>#{account.username}</span></a>"
   end
+
+  def strong_em_html(html)
+    html.gsub(/(^| )[*][*]([^*]+)[*][*]( |$)/, '\1<strong>\2</strong>\3').gsub(/(^| )[*]([^*]+)[*]( |$)/, '\1<em>\2</em>\3')
+  end
 end

---

• I searched or browsed the repo’s other issues to ensure this is not a duplicate.

[sjml]

Why not do a more complete support of Markdown? Ultimately this is something that could live at the UI level instead of the server so different clients (including the web) could adapt as needed.

[krainboltgreene]

Agreed, lets put this at the UI level at least and then perhaps handle markdown.

[valentin2105]

It would be awesome to have code support too. (without syntax for now).

[diomed]

send a PR. can't hurt.
there's also some other people interested in this, including me:

#761

[Exagone313]

I don't agree with any different format for links. It's not handy on phone and one could try phishing.

[sjml]

That's a good point. Would it be confusing, though, to have Markdown minus the linking syntax?

[krainboltgreene]

Markdown already does that by the way. You can either use raw links (which the browser will pick up if nothing else does) *or* use the link syntax.

…

On Thu, Apr 6, 2017 at 2:48 PM, Shane Liesegang ***@***.***> wrote: That's a good point. Would it be confusing, though, to have Markdown minus the linking syntax? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#853 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUb2UZllWynFUfSEof2S0HLBs-HeGmxks5rtV3FgaJpZM4MzBMz> .

-- Kurtis Rainbolt-Greene, Software Developer Founder of Difference Engineers 2985 San Marino St. Los Angeles, CA 90006 202-643-2263

[sjml]

Right; I think @Exagone313 was implying, though, that the link syntax could be used for phishing and thus we may want to avoid it.

Lots of other sites use Markdown, though, without this being an urgent issue. Maybe not worth the hassle at this phase.

[krainboltgreene]

Lets use this as the central point for any sort of formatting of messages on the UI.

Duplicates/Similar: #1305 #761 #1031

[krainboltgreene]

@Gargron We get a lot of requests for this and there are a lot of ways to solve this issue, so this is going to need your approval if mastodon core is going to support this natively.

[Spunkie]

This was previous posted in #761, moving it here.

I just want to 2nd @diomed & @valentin2105 here, inline code,

code blocks,

and quote blocks would be of daily use to me.

From the conversations I've had with maintainers of other large open source projects(like chocolatey) the piecemeal approach to implementing markdown will only lead to headaches down the line though. Either support markdown in its entirety or not at all.

If you are going to implement markdown it would be wise to offload the work to a library that is already CommonMark compliant. Most large services seem to be moving towards it commonmark, even github's own spec is based off it now.

Also some have mentioned in the various threads that supporting markdown will lead to visual clutter but from my experience using gitter.im I've seen that a splash of markdown can increase readability of small format content 100 fold. I've been made a markdown believer if you will. Ya sure there will be some people that abuse it or go overboard but that is why we have the mute/block function.

[zacanger]

If anyone's interested, I have a small (purely client-side) renderer working here. It's a very small subset of Markdown, because of comments made on this issue and the other related ones. Justification for the decisions I made in it are in comments. I'm planning to try it out as a Chromium extension sometime in the next few days and see how it goes.

[nykh]

By the way I have not fully understand @Exagone313 's suggestion that link syntax can be used for phishing. The site I mentioned (Plurk) has link for so long and have not had a problem with that at all. Just render the link so that it is distinctive enough. People should have enough savviness to hover and peek the URL before clicking. But I kind of understand the caution.

@zacanger 's implementation looks quite good. Although I feel eating up the \n (for markdown compliance, obviously) is not very user-friendly in this context...

[zacanger]

If you're talking about the last line (.replace(/\n\s*\n/g, '\n')), consecutive newlines are already collapsed in toots (not sure where in the code that happens, but I tested it out here). If you're talking about something else, I misunderstood :$.

[zacanger]

Something that I hadn't thought about that might be important: I have no idea how GNU Social and other apps consume posts. RSS? Would this break things (or make things look really funky) for them? Does it matter (would small amounts of Markdown syntax showing up in feeds be a big deal to anyone)?

[krainboltgreene]

It's the OStatus protocol.

Here's my bid on this issue:

1. Markdown components, ignore []() syntax (easy)
2. Plain text pushes (easy)
3. Syntax highlighting for code (hard)

[zacanger]

Sorry, I meant the specific format, not the protocol. I looked it up, seems to be Atom-ish underneath?

I have a POC here using a tiny renderer I put together that has a shorthand option (* _ ~ instead of ** __ ~~) and doesn't allow links and images. ol, ul, and blockquote work in the renderer but are broken in Mastodon because of inserted brs, but I can work on that some more if it seems like a good direction to be going in. Thoughts? I'm especially interested in how this might affect GS/other apps ( @rainyday would you be interested in reviewing if this makes it to a PR? ).

[ghost]

I'm just some rando who has submitted a few PR's but in my opinion some limited subset of markdown, like the features dealing strong em ul ol and blockquote wouldn't be problematic on their own since the markdown syntax is pretty much identical to how those features are represented in plaintext normally. However, to my knowledge existing ostatus applications only parse br, a and span (and p in the case of mastodon which is the cause of current formatting issues between mastodon and other servers). So formatting would probably have to either be done client side in the frontend and every other client application or by adding an additional mastadon-specific element to the atom document (like it currently does with mastodon:scope) that contains a rendered version of the post content which would increase the size of activity streams.

Both solutions have their downsides so it kinda depends on the goals of the mastadon devs which, if any, of the solutions they'ed want to implement.

[denysvitali]

For anyone wondering: I'm working on this issue

[Gargron]

@denysvitali Are you using anything like Kramdown or RedCarpet? I don't know if those implementations allow to selectively disable Markdown features, but using an established library would be preferable

[denysvitali]

@Gargron Actually I'm not using any library, just regex that IMHO are smaller and faster. But I agree on your point about the stability. I'm testing this method atm, we'll see how it goes

[zacanger]

Is it preferred that this happens on the server instead of client?

[denysvitali]

@Gargron We're testing it right now at dev.mastodonti.co, if you want you can join and test it. (Registrations are open for a few minutes)

Edit:
You can also see the results here

[wiktor-k]

Just for reference ActivityPub has special field (source) for markup/source text used by the author:

{
  "@context": ["https://www.w3.org/ns/activitystreams",
               {"@language": "en"}],
  "type": "Note",
  "id": "http://postparty.example/p/2415",
  "content": "<p>I <em>really</em> like strawberries!</p>",
  "source": {
    "content": "I *really* like strawberries!",
    "mediaType": "text/markdown"}
}

The entire section is worth reading: https://www.w3.org/TR/activitypub/#source-property

[kizu]

I'd want a subset of markdown on mastodon. While I'd agree that it doesn't need to have all its features, there are things that can be useful even in microblogging: bold, italic, inline code blocks, lists, quotes.

One of the things why they're good (and I didn't see it mentioned there) — they're good for accessibility, as screen readers would read those with proper semantics.

And the mentioned above subset have perfect fallbacks to plaintext, so those who don't like them could have the plaintext as an option without losing anything.

Implementation-wise, I'd say that it should be stored as is, as the implementation of a subset shouldn't be really hard.

Mastodon already makes paragraphs out of toots with line breaks, and making proper semantic lists, blockquotes and other lighter formatting would be very nice and accessible as well.

[wiktor-k]

Another thing worth linking here: XMPP clients like Conversations use a very limited subset of Markdown called Message Styling that cover most of use cases (at least for me): code blocks and code spans, bold and italics.

[koyuawsmbrtn]

I am thinking about this issue all the time. Just markdown support with small heading font sizes like in diaspora* would be extremely nice. It's time to format toots to make them look beautiful. Also this would overcome the issue to be limited to no more than four images in a toot since you can host them at Imgur or something and put useful descriptions into them. Also excluding images in the character count in Markdown should also be a thing since images also aren't bound to character limits.

[Exagone313]

I think I should close the issue, everything has been explained on e.g. #10787

[mashirozx]

My private version of Mastodon mashirozx/mastodon , which support the full set of Markdown grammar, you can see demo toot here

You can try it easily using the database of latest Mastodon release and switch back at anytime.

Also available on Docker Hub

I won't create pull request currently to this official repository due to the reason they discussed in #10787 , and also for the reason that I think the code and feature should be enhanced before pull request. If you like please help me enhancing the code and feature.

[ghost]

Links when? I'd love to see being able to link for example my GitHub or a website using a clean my GitHub instead of https://github.com/valentino1337
Or better yet, linking images with ![] instead of having to upload them. That would save a lot of work for the user, server space for the instance and bandwidth for both

[timmc]

Image embedding would be a whole different thing from text formatting -- you'd need an image proxy API to go along with that, since loading directly from the user's client reveals their IP address.

[mashirozx]

@valentino1337 You may try this: mashirozx@7014efb

Just insert any media URL (Mastodon will download them as remote media attachment) with short code:

IMAGE: [https://example.com/example.png]
IMAGE: [https://example.com/example.png] {file name if you need}
VIDEO: [https://example.com/example.mp4]

Just needs to git cherry-pick it, no need to migrate database.

Try it here: https://hello.2heng.xin

[alexanderadam]

Well, while the idea might be born here in 2017 Twitter might get this feature faster (formatting depends on your client though).