Importing should register in Recent

[twMat]

Importing, say, an image via drag'n drop should be reflected in the Recent list.

Edit: Here is a mockup that handles the problems identified in the following discussion.

[sukima]

There is already an import list provided. Would this be to enhance that list because it needs to be recent or not.

I could see an counter argument that you don't want to change recent list during a import. But instead to add to recent as you click on your chosen tiddlers from the import summary page.

[pmario]

imo both behaviours make sense for file based TiddlyWikis. ... The server version needs to update the modified field on import, for consistency reasons.

[twMat]

IMO, the import list is of value - but it is not instead of the Recent listing and, as far as I understand, the import list gets replaced for each new import so it's not possible to see its history. (That may be desirable though?)

I try to look at things from a pragmatic perspective; the Recent list is there (I think) to easify access by using our "sense of chronolgy" for when it was we dealt with something. It would be (is!) odd to force the user to remember exactly how a tiddler came into existence; the ease of access should be regardless if it was created from scratch, copy-pasted field-by-field, or dragged in. What's the point in separating these?

I could see an counter argument that you don't want to change recent list during a import. But instead to add to recent as you click on your chosen tiddlers from the import summary page.

I'm curious on which that counter argument would be. And are you suggesting titles should be added to Recent merely by clicking on them (i.e not by editing them)?

[sukima]

Good point. To clarify I was playing devil's advocate. So my personal opinion is this could be a useful feature worth looking into.

[Jermolene]

Importing, say, an image via drag'n drop should be reflected in the Recent list.

That implies that importing a tiddler would reset its modified time. That seems destructive; in some cases, the last modification date of a tiddler is useful metadata about when a tiddler was last updated.

We could perhaps set the created/creator/modified/modifier fields if they are not present on the incoming tiddler? But I'm concerned that there are other consequences too: importing a plugin would change the modified date, which would mean that the plugin tiddler would no longer be 100% the same as the original source.

[inmysocks]

Would it be hard to add a tiddler that holds the titles of the previous n most recently imported tiddlers in its list field? That way there wouldn't be any changes to imported tiddlers.

You could add something like this to the import button to update the list:

<$set name=UpdatedList filter='[[$:/RecentlyImported]list[]]{CurrentImportedTiddlerList}+[last[n]]'>
<$action-setfield $tiddler='$:/RecentlyImported' $field='list' $value=<<UpdatedList>>/>
</$set>

You would have to replace {CurrentImportedTiddlerList} with something that actually works in this context, and I am assuming that there is some way to access the list of tiddlers waiting to be imported.

[Jermolene]

I'm not massively keen on the new "Recently Imported" tab. I think we can refine the import handling to improve the behaviour. For example, we could perhaps add created/creator/modified/modifier fields to imported items that are not already tiddlers (so, images dragged from the filesystem would show up in Recent).

[felixhayashi]

Importing, say, an image via drag'n drop should be reflected in the Recent list.

TL;DR I am with @twMat

[Jermolene]

@felixhayashi what do you think of the concerns I raised:

• It seems destructive for importing a tiddler to reset its modified time; in some cases, the last modification date of a tiddler is useful metadata about when a tiddler was last updated
• importing a plugin would change the modified date, which would mean that the plugin tiddler would no longer be 100% the same as the original source

[felixhayashi]

Hi @Jermolene

sorry, I was in haste and didn't have time to read.

Both your points are strong arguments!
I changed my mind and I am on your side now :)

Still Mat's point is valid:

I try to look at things from a pragmatic perspective; the Recent list is there (I think) to easify access by using our "sense of chronolgy" for when it was we dealt with something.//

I like your idea of a compromise to modify the behaviour so that at least images dragged from the filesystem would show up in recent. But in any other case, I would not change the modification date! They are important e.g. when working with colleagues and exchanging tiddlers because you have to know how recent the information is.

Maybe another field added that stores at what time data (tiddler/images) entered the wiki would be nice. This would go along well with created and modified. Maybe overkill though.

-Felix

[sukima]

How about an easy way to reopen the $:/Import tiddler. Or possibly have the $:/Import be a sub-tab of the 'More' tab. maybe right under Shadows. Thoughts?

[Jermolene]

Mmmm. The trouble is that despite my two objections above, there is a compelling logic to the idea that a recently imported item should appear in the "Recent" list.

One possibility would be a slightly more complicated rule: add a modified time if the tiddler is not a system tiddler and does not already have a modified field.

That behaviour might still be confusing, though: a newly dragged tiddler with a modified date from another wiki wouldn't appear at the top of the "Recent" list.

[felixhayashi]

Related: https://groups.google.com/d/msgid/tiddlywiki/f9255b04-a8b7-482f-894f-b837e54c42d4%40googlegroups.com?utm_medium=email&utm_source=footer

[tobibeer]

Maybe $:/Import could show a log of all imports, not just the most recent. For example, importing SomeTiddler there could be some system tiddler...

$:/temp/imported/SomeTiddler

...holding the import date as its creation date and perhaps other import related information, should there be any (serialisation, etc).

As for the recent tab, I would prefer a slider called "recently imported" which would show the same import log.

[twMat]

OK, I think I solved it UI-wise! Mockup: http://import-in-recent.tiddlyspot.com/

Please recall from the above (old) discussion that the important thing is to have direct access to imported tiddlers. The mockup features access to both //just// imported tiddlers //and// previously imported tiddlers that the user wants keep shortcuts to.

[pmario]

I think, that's plugin territory. ... The bundler plugin implements 1 way to handle imported tiddlers. You may want to add a different way. ... Users should have the choice.

see: Enable Import Bundle https://wikilabs.github.io/editions/bundler/#%24%3A%2Fplugins%2Fwikilabs%2Fbundler%2Fui%2FControlPanel%2Fbundler-config

[twMat]

I think, that's plugin territory. ... The bundler plugin implements 1 way to handle imported tiddlers. You may want to add a different way. ... Users should have the choice.

Non sequitur. Surely a plugin can't be expected to care for all other plugins that people may have designed? If e.g bundles was part of standard distro, then I'd agree but as it is...

As for if it should be a plugin; In your opinion, what kind of solution would fulfill the OP that is suitable for native TW? I'd really appreciate an answer on this (from you or anyone) because we keep butting heads on this. You (and many) are way more knowledgeable than me when it comes to coding so maybe it is something I don't get. My argument, for the current case as a native solution is this:

For the OP "Imports should register in Recent", it is - as far as I can tell - either #2530 (i.e to modify the imported tids modified date) ...or it is to "build" something like in the mockup. So, in your opinion, is there anything that is "built" and qualifies for standard distro? Or is there a third route? What in that case?

[twMat]

Or is there a third route? What in that case?

Actually, there is, but only in theory: If plugins were easily accessible + also #3952 , I'd agree that standard distro should contain the absolute minimum of bells and whistles.

[inmysocks]

It wouldn't be difficult to add an option to.update the created and modified fields, but it would either require a system setting or a toggle on the import tiddler. I think the easiest solution is a view template on the import list that adds an imported field an the import tiddler.

It isn't something I have had a use for, but it seems simple enough as a plugin or just a single view template tiddler. Having it available on tiddlywiki.com as an example could take care of the access part.

[pmario]

There are several use-cases to set a time stamp at import.

Eg: If I d&d import a file, as a user I'd expect the "created and modified" stamp to be equal to the OS file timestamp. ...

If it is a .tid file, I'd like to have the choice, if I want to overwrite the existing info. Default should be: Use existing info, instead of overwriting it.

If the imported file is an image and contains exif information I'd like to have a choice if either the file datestamp is used or the EXIF info is used. If exif is there I'd like to have a list of fields that should be created. ....

As you can see, the minimal useful function is: "check the file info if available and use it" ... That can be part of the core. ... The rest is definitely plugin territory.

---

BUT you are right. The core import mechanism needs to be improved. ... Because if it detects a file that shouldn't be imported, it completely removes it from the import tiddler. So even if you "check" the import button, it can't be imported. Since it doesn't exist anymore. This will be fixed in v5.2.0

[twMat]

OK, so the probably universal need to have imported tids both easily accessed and not modified, is not for native TW then... :-/

If the goal is a minimal standard distro then I agree.
If the goal is UX then I don't.

[pensive-fuzz]

That implies that importing a tiddler would reset its modified time. That seems destructive; in some cases, the last modification date of a tiddler is useful metadata about when a tiddler was last updated.

We could perhaps set the created/creator/modified/modifier fields if they are not present on the incoming tiddler? But I'm concerned that there are other consequences too: importing a plugin would change the modified date, which would mean that the plugin tiddler would no longer be 100% the same as the original source.

I'm in the process of getting around to upgrading a personal TiddlyWiki Classic to TiddlyWiki 5. After importing I noticed that some of the tiddlers weren't in the recent list I opened the file in a text editor and noticed that those tiddlers had a created filed but no modified field, as they were never modified after initial creation.

I find it completely counterintuitive having tiddlers that were visible in the Timeline no longer appear in the equivalent Recent list after migrating. As such, personally I'd like to see TiddlyWiki either copy an existing created field into a missing modified field or have the Recent list utilize the created field when the modified field is missing instead of ignoring the tiddler.

[bulrush15]

I don't want just an import list (which is a tiddler itself) i'd also like to see images on my Recent list in the sidebar, or have an option to show images in the Recent list.

Also, tiddlers that I drag and drop from one TW to another do not show up in the Recent list in the sidebar. I'd like an option to show those as well in the Recent list.

If importing a tiddler resets it's modified time to now that would be fine with me.

I don't need the modified date on a plugin I import to be modified, I'm just talking about adding to the Recent list things like images, tiddlers I dragged from another TW.

Thanks!

[AnthonyMuscio]

Folks,

Without seeing all the detail of this issue here is a possible resolution.

Add to the import process the creation of an import-date on every tiddler with the value of "now" however do this without touching the created and modified date.

Keep in mind with a plugin this will only apply to the plugin tiddler itself, not its shadows.

Add to the recent tiddlers tab the inclusion of tiddlers with an import-date replacing modified date when it exists. A little tricky but possible.

Thus the import-date will always be changed on import, so no matter if it is exported with tiddlers. You can always see IF and When a tiddler was imported. You could even find which tiddlers were modified since import or not. Perhaps a final improvement would be a imported-by field.

If you are not interested in the above simply logging every tiddler title, import time and current user to a single import log would be sufficient but not necessarily part of the recent tab.

Regards
Tony

[Jermolene]

Hi @bulrush15

I don't want just an import list (which is a tiddler itself) i'd also like to see images on my Recent list in the sidebar, or have an option to show images in the Recent list.

Images are shown in the recent list if they have a modified field.

Also, tiddlers that I drag and drop from one TW to another do not show up in the Recent list in the sidebar. I'd like an option to show those as well in the Recent list.

Drag and drop is just a shortcut for importing.

If importing a tiddler resets it's modified time to now that would be fine with me.

That wouldn't be backwards compatible, but could be added as an option.

I don't need the modified date on a plugin I import to be modified, I'm just talking about adding to the Recent list things like images, tiddlers I dragged from another TW.

[AnthonyMuscio]

@bulrush15 you say;

I don't need the modified date on a plugin I import to be modified, I'm just talking about adding to the Recent list things like images, tiddlers I dragged from another TW.

But you do realise recent = recently modified by definition?, not recently opened or created but recently changed. For good reason import retains the modified date of the tiddlers you import, not the date they were imported.

You can just edit a tiddler immediate after import if you want to get it to appear in the recently modified list.

Also installing the bundler plugin you can set it to capture the tiddler titles of everything imported and lists tiddlers from first imported to last. It would be trivial to make this clickable links.

Regards
Tony

[MartyGentillon]

Also, it would be nice if we could remember where imported tiddlers come from. This has particular applicability to updating plugins. Right now, I have a bunch of plugins, and, unless I manually track where they came from, I have no idea where they came from.

[AnthonyMuscio]

Returning to this issue I realised I had developed a solution (fromm y perspective) and it was to simply allow after import a button to appear titled "save import record" which copies the list of tiddlers into a time date stamped import record. The side bar has a tab "imports" listing the imports, click to open and review the list of imported tiddlers. If you wish you can rename the time date import record to a meaningful name.

If you like this I can publish it.

[CodaCodr]

Interesting idea. I'd have thought GG would be the best place, Tones.

[AnthonyMuscio]

Interesting idea. I'd have thought GG would be the best place, Tones.

My Question is would it be sufficient to address Importing should register in Recent #1554 by providing a recent imports tab?

if twMat thought so we could close this

[twMat]

Thanks for your thoughts @AnthonyMuscio

My Question is would it be sufficient to address Importing should register in Recent #1554 by providing a recent imports tab?

My request is for a core or std distro solution because I believe the issue is of common interest. So for this I don't think a permanent and dedicated sidebar tab is a good idea because that'd take up too much real estate for an issue that is not that frequent, and IMO the registration should be automatic because you can't really tell in advance if you need to see it in the list or not.

[AnthonyMuscio]

twMat,

I would not dispute the standard distro but typically we demonstrate it before inclusion. If you describe your requirements perhaps I can make a solution. One that does not use any additional real-estate. I mean it could be a tools button that opens an Import Manager.

We can extend the import mechanism to always save to an import record, and then the "record tiddler itself" will appear in the recent tab, with the tiddlers imported named within it? This allows the imported tiddlers to remain with their own modified and created date with the import record appearing in the recent tab.

If placed in the standard distribution the import records could be temp tiddlers unless the user requests they be retained, but presently the recent tab does not show system tiddlers.

What do you think?
Tony

[pmario]

As I wrote, the bundler plugin has a functionality, that lets you create an import.bundle tiddler that contains a list of the imported tiddlers. When TW v5.2.0 is released it will also be possible to add a "comment" line to the import.bundle tiddler, that shows the import date.

So it will be possible to know, when each tiddler has been imported. New elements will be added to the import.bundle when a new import happens see: https://wikilabs.github.io/editions/bundler/#%24%3A%2Fplugins%2Fwikilabs%2Fbundler%2Fui%2FControlPanel%2Fbundler-config

[twMat]

@AnthonyMuscio

We can extend the import mechanism to always save to an import record, and then the "record tiddler itself" will appear in the recent tab, with the tiddlers imported named within it? This allows the imported tiddlers to remain with their own modified and created date with the import record appearing in the recent tab.

Interesting. I see both pluses and minuses with this idea. First, I interpret your idea to mean that, in the Recent list, the "import record" tiddler title shows and there is a little "expand arrow" next to it (like in ToC macros) to show a dropdown list with the imported titles (...because it would not be a good solution if one has to actually open an import tiddler to access its listing). Now, in actual use, this would mean that one has a title in mind (formulated exactly or, more typically, only vaguely) and one scans the Recent list but doesn't find it. So then one expands the import title(s) and hopefully finds it therein. But this also means that if you're looking for a title that really isn't in the Recent list anymore, then you're possibly mislead to snoop around in the import record dropdowns in vain. Currently, it is directly obvious if a title is in the Recent list or not and it is a pity to compromise this directness. And as you note, what should be done with the import record tiddlers when they are too old for the Recent list? Maybe it makes sense, as you suggest, to make them as temp tiddlers because "recent" kind of implies "this session" anyway.

...but more than anything I have a feeling this behaviour and UI is "too special" for something that is not an everyday issue. Especially if you consider that the problem with "missing in Recent" only concerns some imported tiddlers. (Besides, would the "import record" list all titles or just the ones relevant for this issue?) So while the idea has elegance, I feel the use case is fringe enough to disqualify a special UI. It should ideally be more invisible.

[twMat]

How about this:

The import mechanism adds all tiddlers that lack a created field to a separate list (let's call it the "undated tiddlers list"). These titles are associated with the time of the import. And the Recent list simply lists those titles along with the regular recent titles. Perhaps the "undated tiddlers list" is a temp tiddler so it only survives the current session?

Thoughts?

[Jermolene]

Perhaps the clearest solution would be a new sidebar tab called "Imported" that lists the recently imported tiddlers. We could keep track of them in the list field of a state tiddler like $:/state/imported.

There are several options for where to place the "Imported" tab:

1. Next to "Recent" as a new top-level sidebar tab
2. Under "More" as a secondary sidebar tab
3. Split the "Recent" tab into two with a nested tabset to choose between the current "Modified" and "Imported"

[pmario]

Perhaps the clearest solution would be a new sidebar tab called "Imported" that lists the recently imported tiddlers. We could keep track of them in the list field of a state tiddler like $:/state/imported.

There are several options for where to place the "Imported" tab:

1. Next to "Recent" as a new top-level sidebar tab

2. Under "More" as a secondary sidebar tab

3. Split the "Recent" tab into two with a nested tabset to choose between the current "Modified" and "Imported"

What you describe here is almost exactly, what my bundler plugin does.

If you open the link you will see the configuration page.

• If the "import.bundle" will be enabled, evey import is appended to the list in the text area of the tiddler.
• With v5.2.0 there will be some extra comment lines, which are not possible atm.
• The last setting lets you choose, where the "Bundles" tab will be shown. Prominent or hidden under More
• Since the tiddler is meant to be used to manage imported tiddlers it is a common tiddler and not a system-tiddler

It would be possible to make the import tiddler title configurable defaulting to "import.bundle"

---

It may be interesting to explore a possibility to assign "end user defined" filters, that show additional infos to tiddlers.
eg: [all[current]!has[modified]then[no modified field]] ... If this filter returns a value, the import tiddler is marked with that text. It should be possible to "stack" those filters. May be several lines in a config tiddler. 1 possible user text per line.

[CodaCodr]

3. Split the "Recent" tab into two with a nested tabset to choose between the current "Modified" and "Imported"

👍

[AnthonyMuscio]

To me the created and modified dates of tiddlers should be sacrosanct, what ever is imported should come with these values or not, they should not be changed by the import process. This allows the true created an modified date be available.

Remember we often import plugins or json files which in turn contain tiddlers each with their own dates and to modify these results in a shadow/tiddler pair.

I have being inspired by Mario's bundler plugin, but feel we can enhance the bundling and simultaneously simplify it. For example allow Any tiddler to contain a package-filter field (contains filter and/or list) that can act as a bundle, so there is no need for a bundle tiddler.

As per my most recent suggestion, it would be a simple matter to generate a tiddler with todays date and time eg "Import Record 8/8/2021 11:58" containing the list of tiddler imported in the body text. These tiddlers could be tagged or have a field set so all import records can be found. The user can happily rename this Import record tiddler to a more meaningful title if desired eg from "Import Record 8/8/2021 11:58" to Import "Editor Toolbar addons". It would not hurt also including these titles in the list field as per Jeremy's suggestion, such that the import titles will be available for programmatic use if required. Import record tiddlers can reliably use their created date as the import time.

• Import record tiddlers (not the tiddlers there in) will appear in the recent tab when they are imported/modified
• Searching for any tiddlername will return both the tiddler itself and the import record in which it is listed, perhaps even multiple import records.
• Such import records could accumulate indefinitely, without much impact (but the optional Import Records tab could allow deletion of any import record and also provide a total bytes used by all import records).
• No one need even know the the import records exist
• A tools button could open the import records tiddler in the story as well, this "Import records Tiddler" could contain additional features like only search within import records, or show imported tiddlers by import date(s) and provide the toggle to display the import records tiddler as a tab.
• The import records tiddler/tab could meet Mats desire with a "more button" to list the tiddlers within each batch.
• It would also be possible to create a list of tiddlers by there presence in an import record and sortby the import records created date, rather than their own created date.

This proposed method could solve 95% of the requirements as well as provide avenues for additional Import Records management, including but not limited to;

• Import as a JSON file without expanding into tiddlers and treat the JSON tiddler also as an Import record
• Convert an Import record to a plugin of the named tiddlers for subsequent export/distribution
• Ability to export tiddlers according to their being listed in an import record (re-export with updates applied)
• The ability to import additional tiddlers by dropping them on a named import record to import them whilst adding them to the import record. Allowing bundling as you import.

Regards
Tony

[pmario]

@Jermelone wrote

3. Split the "Recent" tab into two with a nested tabset to choose between the current "Modified" and "Imported"

I did have a new idea, where we don't need to split anything. ... The $:/core/macros/timeline tiddler already contains a "timeline-title" macro with the text: <!-- Override this macro with a global macro of the same name if you need to change how titles are displayed on the timeline -->

With v5.2.0 it will be possible to simplify the timeline macro code as follows:

<!-- Override one or both of the following 2 macros with a global macro 
     of the same name if you need to change how titles and/or links are displayed on the timeline -->
\define timeline-title() <$view field="title"/>
\define timeline-link() <$link to={{!!title}}><<timeline-title>></$link>
\define timeline(limit:"100",format:"DDth MMM YYYY",subfilter:"",dateField:"modified")
<div class="tc-timeline">
... snip ...

Since we can have "comments" in front of macro definitions now.

The code changes above will allow anyone to completely redefine the links in the timeline-macro without the need to modify the core macro itself.

See the attached code, which ONLY works with the prerelease, since it needs pragma comments.

test-recent-link.zip

It looks like this when open:

and like this when folded

[pmario]

I will create a PR with some small changes to $:/core/macros/timeline soon.

@Jermolene ... Should I do a similar thing for the Open-tab?

[pmario]

@twMat ... What do you think? ... The 2 PRs should be the first step towards more flexibility

[AnthonyMuscio]

twMat

Mario's suggestions for the recent tab support my Import records method, I can proceed with that if you want it, or may for myself?

[twMat]

@pmario - thank you for experimenting! That is a slick UI and, as Tony notes, it aligns well with his vision to which I expressed a concern that:

[...] if you're looking for a title that really isn't in the Recent list anymore, then you're possibly mislead to snoop around in the import record dropdowns in vain. Currently, it is directly obvious if a title is in the Recent list or not and it is a pity to compromise this directness.

I have a feeling the "hidden" tiddlers might cause some friction in the workflow but I obviously don't know. But I did add the following idea previously (albeit phrased in another way):

The Recent timeline lists imported tiddlers - including tiddlers with missing created/modified dats - by using the modified date of the very "Imported tiddlers" tiddler as a surrogate date. (I note that $:/Import lacks created/modified, so either this is added or there are other $:/Import type tids that persist after new imports)! This also incidentally (and conveniently) puts the imported and dateless tids at the very top of Recent.

This does not compromise the directness of the current Recent list. One potential problem might be that the UI is too smooth: The user might not realize that those tiddlers miss modified fields. Maybe there could be some notification next to these titles in the list.

[pmario]

@twMat ... I did forget, that v5.2.0 will allow us to easily manipulate tiddlers at import time using the th-importing-tiddler hook. So it will be easy for plugin authors to add the created and/or modified field to tiddlers that have no such field.

I'll integrate this function into my bundler plugin. ... It should be easy to create it as a stand-alone plugin with some configuration tiddlers too. ...

The import-workflow I do have in mind is:

• There are 2 config-values for created and modified, which can be defined by the user
• There is a global setting: don't modify missing date fields -> default
  • .. use <<now>> for missing "date-fields"
  • .. use "created AND modified" from config-tiddlers ... for missing fields
  • .. use "created from config" and <<now>> for modified
  • .. if modified is defined and created is missing, both fields will be set to modified-date
• There may be a separate checkbox for "force overwrite date-fields " ...

I think that should cover all possibilities that make sense. IMO It doesn't make sense to have a created date that is newer than modified. ... If the user sets "both to config" they can do whatever they want. .. So they are responsible that it makes sense.