awk for Regular Expressions Query

[FelipeRearden]

Hello @Taitava and Friends!

This is a post to show how to use Shell Commands to select/copy content from a note using the awk command.

User Case

I have .md files generated from a App (DEVONthink) that I use for highlight Web Articles. This App let me highlight web articles and generate a .md file only with the highlights.

This App generates a URL from every highlight that I do. This way, I can track the origin of the highlight inside the original article in the App
[L4](x-devonthink-item://C5A952D4-8CB9-4CA9-9B3A-BC397B08C1F6?line=3)

The problem is that the url schema is not placed that I want. I need to select all these URL and transfer to another placed in the note.

I am on MacOS and Obsidian 0.12.15

AWK

After a long search I found a command called awk that is used to select text inside a note based on a bunch of criteria.

For my need I need to copy all URL to the clipboard to be pasted in the end of the note

What worked for me is this:

awk 'match($0, /regex/) {print substr($0, RSTART, RLENGTH)}' "/{{file_path:absolute}}" | pbcopy

• match() >>> apply the regex query in the whole document $0
• {print substr} >>> select only the regex query instead of the whole line where the regex found somenthing
• pbcopy >>> copy to clipboard

In my case:
awk 'match($0, /\[L[0-9]*\]\(x-devonthink-item:\/\/[a-zA-Z0-9_]*-[a-zA-Z0-9_]*-[a-zA-Z0-9_]*-[a-zA-Z0-9_]*-[a-zA-Z0-9_]*\?line\=[0-9]*\)/) {print substr($0, RSTART, RLENGTH)}' "/{{file_path:absolute}}" | pbcopy

This is the result:

---

If you think there is a better way to do this, please let me know!!!

• I'm just a newbie and don't have much experience with regex and bash.

---

I hope it helps someone with the same workflow.

Have a great day!

[Taitava]

Nice! I'm trying to figure out this bit by bit, so I'll do a simple summary in my own words, just so that I can analyse this a bit:

1. Preparations before executing a shell command: Quotes from web articles are stored with DEVONthink to markdown files.
2. AWK is used to read links from the markdown file that you have currently open in Obsidian (= that's where {{file_path:absolute}} refers to).
3. The result of AWK is piped (= "transferred", not the best veb here) to the clipboard. This is where the shell command stops.
4. Afterwards, you have the links in your clipboard, and you will edit the file manually in Obsidian to do the following: paste the links into the end of the file. If I understood correctly, the links were not removed from the MD file by AWK? So you need to remove the links manually from their old places? Could AWK remove them for you?

Can you please do small improvements:

• Make DEVONthink to be a link to their website.
• Make the first mention of AWK to be a link to AWK website.
• Mention that {{file_path:absolute}} is a variable provided by this plugin, which gives a path to a file that is currently open and active in Obsidian. If there are newcomers (or someone just lands here from another planet, i.e. a Google search), they can easily see that {{file_path:absolute}} is not a natural part of a shell command, and it won't work in a normal terminal. You can add this for example to the list you wrote:

  • match() >>> apply the regex query in the whole document $0
  • {print substr} >>> select only the regex query instead of the whole line where the regex found somenthing
  • pbcopy >>> copy to clipboard

Have a great day, too! 🙂 You have clearly come up with ideas that let you use this plugin in ways that would have never come to my mind! 😎

[FelipeRearden]

@Taitava !!!!!

The whole story about this is here in the post for the fantastic Apply Patterns plugin. That’s why this awk Shell Commands is used.

If I understood correctly, the links were not removed from the MD file by AWK?

No, you are right. I just select and copy to clipboard.

So you need to remove the links manually from their old places?

After this step, I have another regex in Apply Patterns that erases only (x-devonthink-item://C5A952D4-8CB9-4CA9-9B3A-BC397B08C1F6?line=3) to make the line clean.

Could AWK remove them for you?

As far as I know, yes we can do a find and delete with AWK (but I don’t know how)

---

I happy to share this with you !!! Now you are seeing why this is so important to me :)