List and Export linked files #370

leoheck · 2022-10-05T21:37:48Z

This is an attempt to list unique linked files of the assembly.
I also export files into a zip package with the name of the file + asm4_zip in the assembly folder.

…zip package.

…9% of the time)

hasecilu · 2022-10-05T23:57:21Z

I tried to load the changes but no icons are loaded in the WB, there is something in InitGui.py that breaks it.
Edit: ok, I'll check it but i have a problem with dependencies

leoheck · 2022-10-06T14:16:00Z

Thanks. It happens that Freecad has a different file path for the file being loaded when using the GUI and the CLI. I improved the behavior of the zip exporter.

Now, I don't have issues with InitGui.py
Could you check if your copy it is updated and does not have changes?

hasecilu · 2022-10-06T16:50:38Z

Is anytree, I have it in my system but not in FreeCAD's enviroment

leoheck · 2022-10-06T18:26:08Z

Did you do/try this?

sudo -H pip install anytree

leoheck · 2022-10-06T18:27:10Z

I installed it this way and it was working.

hasecilu · 2022-10-06T18:29:39Z

Ok, I solved it copying the anytree folder to the A4 folder.

The icon of List Linked Files is white/light gray and very similiar the the BOM buttons, maybe change to a similar color as List Linked Files (Tree) to have them "grouped" visually? Both commands work fine. There are typos in the svg files they say: Liked.

I got some errors with Export FIles and I think it's because those files were used more than once in the Assembly, so the function tries to add them again.

13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/T8_leadscrew/T8_leadscrew_500mm.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/SHF08/SHF08.FCStd
13:22:32  /snap/freecad-realthunder/70/usr/lib/python3.8/zipfile.py:1517: UserWarning: Duplicate name: 'cad/FreeCAD-library/SHF08/SHF08.FCStd'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/KFL08/KFL08.FCStd
13:22:32  /snap/freecad-realthunder/70/usr/lib/python3.8/zipfile.py:1517: UserWarning: Duplicate name: 'cad/FreeCAD-library/KFL08/KFL08.FCStd'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
13:22:32  ASM4> [ZIP] Adding file: sub-assemblies/Electronics.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/Board4electronics.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/step/Arduino_CNC_shield.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/step/Fuente_12V_10A.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/Slot_T_nuts/Slot_T_nuts.FCStd
13:22:32  /snap/freecad-realthunder/70/usr/lib/python3.8/zipfile.py:1517: UserWarning: Duplicate name: 'cad/FreeCAD-library/Slot_T_nuts/Slot_T_nuts.FCStd'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
13:22:32  ASM4> [ZIP] Adding file: cad/Spacer_20.FCStd
13:22:32  ASM4> [ZIP] Adding file: sub-assemblies/SW_X.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/step/Limit_SW_PCB.FCStd
13:22:32  /snap/freecad-realthunder/70/usr/lib/python3.8/zipfile.py:1517: UserWarning: Duplicate name: 'cad/step/Limit_SW_PCB.FCStd'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
13:22:32  ASM4> [ZIP] Adding file: sub-assemblies/Green.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FDM/Light.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/step/Green_indicator.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/NEMA/NEMA_17_with_connector.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/Steel L-Bracket for NEMA 17.FCStd
13:22:32  /snap/freecad-realthunder/70/usr/lib/python3.8/zipfile.py:1517: UserWarning: Duplicate name: 'cad/Steel L-Bracket for NEMA 17.FCStd'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
13:22:32  ASM4> [ZIP] Adding file: cad/FDM/NEMA17_support.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/Coupling/Beam-coupling-8mm-5mm.FCStd
13:22:32  /snap/freecad-realthunder/70/usr/lib/python3.8/zipfile.py:1517: UserWarning: Duplicate name: 'cad/FreeCAD-library/Coupling/Beam-coupling-8mm-5mm.FCStd'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
13:22:32  ASM4> [ZIP] Adding file: sub-assemblies/3D-printed_corner.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FDM/Big_Corner/Big_Corner.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/Slot_T_nuts/Slot_T_nuts.FCStd
13:22:32  ASM4> [ZIP] Adding file: sub-assemblies/Bed_full.FCStd
13:22:32  ASM4> [ZIP] Adding file: sub-assemblies/Bed_top.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/Plate_03.FCStd
13:22:32  ASM4> [ZIP] Adding file: sub-assemblies/Bed_bottom.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/SK08/SK08.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/KP08/KP08.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/8mmx500_rod.FCStd
13:22:32  /snap/freecad-realthunder/70/usr/lib/python3.8/zipfile.py:1517: UserWarning: Duplicate name: 'cad/8mmx500_rod.FCStd'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
13:22:32  ASM4> [ZIP] Adding file: cad/FreeCAD-library/T8_leadscrew/T8_leadscrew_500mm.FCStd
13:22:32  /snap/freecad-realthunder/70/usr/lib/python3.8/zipfile.py:1517: UserWarning: Duplicate name: 'cad/FreeCAD-library/T8_leadscrew/T8_leadscrew_500mm.FCStd'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
13:22:32  ASM4> [ZIP] Adding file: sub-assemblies/SW_Y.FCStd
13:22:32  ASM4> [ZIP] Adding file: cad/FDM/LSW_support_Y.FCStd

hasecilu · 2022-10-06T18:38:14Z

Did you do/try this?
sudo -H pip install anytree

I have just tried this and did not work in both snap and AppImage

leoheck · 2022-10-06T19:10:55Z

hm, I thought those "UserWarning: Duplicate name:" were just a warning, this may not be the case. They have to be added to preserve the hierarchy when unpacking the zip. I could rename all items before inserting in the package, and then I did not have errors, but this is not the fix since they won't be renamed back to the original when unpacking the zip. I am investigating this thing.

hasecilu · 2022-10-06T19:29:44Z

They still manage to be packed in the zip files, even with the unzip command I get messages asking me to replace the same files.

leoheck · 2022-10-06T19:31:35Z

try to extract it and load the main assembly, please.

hasecilu · 2022-10-06T19:56:58Z

For some reason in the list the googley eyes, Spacer 6.35 and V slot profile 370 mm are not detected and the files are not saved in the zip.

The assembly in the zip file is almost complete but due to above error there are errors about file not found

leoheck · 2022-10-06T21:00:03Z

It seems there is no duplicated files, only in the zip

Examining this case here:

17:54:17  ASM4> [ZIP] Adding file: cad/Steel L-Bracket for NEMA 17.FCStd
17:54:17  /tmp/.mount_FreeCARwPJVr/usr/lib/python3.10/zipfile.py:1517: UserWarning: Duplicate name: 'cad/Steel L-Bracket for NEMA 17.FCStd'

Then we see that there is only one file with this name:

➜  diy-pcb-cnc git:(testing) ✗ find -name "*Steel L-Bracket for NEMA 17.FCStd*"
./cad/Steel L-Bracket for NEMA 17.FCStd

This is the whole list of files.

17:58:01  ASM4> Listing uniq linked files
17:58:01    1 - ASSembly.FCStd
17:58:01    2 - cad/8mmx300_rod.FCStd
17:58:01    3 - cad/8mmx500_rod.FCStd
17:58:01    4 - cad/Board4electronics.FCStd
17:58:01    5 - cad/CNC_mount/CNC_mount_V2.FCStd
17:58:01    6 - cad/Drill_bit.FCStd
17:58:01    7 - cad/FDM/Big_Corner/Big_Corner.FCStd
17:58:01    8 - cad/FDM/Emergency_button_case.FCStd
17:58:01    9 - cad/FDM/LSW_support_Y.FCStd
17:58:01   10 - cad/FDM/LSW_support_Z.FCStd
17:58:01   11 - cad/FDM/Light.FCStd
17:58:01   12 - cad/FDM/NEMA17_support.FCStd
17:58:01   13 - cad/FreeCAD-library/2020_V-slot_Al_extrusion/2020x450_V_slot_profile.FCStd
17:58:01   14 - cad/FreeCAD-library/2020_V-slot_Al_extrusion/2020x490_V_slot_profile.FCStd
17:58:01   15 - cad/FreeCAD-library/Corner/2020_corner_bracket.FCStd
17:58:01   16 - cad/FreeCAD-library/Coupling/Beam-coupling-8mm-5mm.FCStd
17:58:01   17 - cad/FreeCAD-library/Electrical Switches/EmergencyButton_LAY37.fcstd
17:58:01   18 - cad/FreeCAD-library/KFL08/KFL08.FCStd
17:58:01   19 - cad/FreeCAD-library/KP08/KP08.FCStd
17:58:01   20 - cad/FreeCAD-library/NEMA/NEMA_17_with_connector.FCStd
17:58:01   21 - cad/FreeCAD-library/SC8UU/SC8UU.FCStd
17:58:01   22 - cad/FreeCAD-library/SHF08/SHF08.FCStd
17:58:01   23 - cad/FreeCAD-library/SK08/SK08.FCStd
17:58:01   24 - cad/FreeCAD-library/Slot_T_nuts/Slot_T_nuts.FCStd
17:58:01   25 - cad/FreeCAD-library/T8_housing_bracket/T8_housing_bracket.FCStd
17:58:01   26 - cad/FreeCAD-library/T8_leadscrew/T8_leadscrew_350mm.FCStd
17:58:01   27 - cad/FreeCAD-library/T8_leadscrew/T8_leadscrew_500mm.FCStd
17:58:01   28 - cad/FreeCAD-library/T8_screw_nut/Anti_backlash_nut/T8_antibacklash_nut_simplified.FCStd
17:58:01   29 - cad/Mototool.FCStd
17:58:01   30 - cad/Plate_03.FCStd
17:58:01   31 - cad/Plate_04.FCStd
17:58:01   32 - cad/Plate_05.FCStd
17:58:01   33 - cad/Side_wood.FCStd
17:58:01   34 - cad/Spacer_20.FCStd
17:58:01   35 - cad/Steel L-Bracket for NEMA 17.FCStd <============ THIS ONE
17:58:01   36 - cad/Timing_belt_GT2.FCStd
17:58:01   37 - cad/Timing_pulley_GT2_20.FCStd
17:58:01   38 - cad/Timing_pulley_GT2_60.FCStd
17:58:01   39 - cad/step/Arduino_CNC_shield.FCStd
17:58:01   40 - cad/step/Fuente_12V_10A.FCStd
17:58:01   41 - cad/step/Green_indicator.FCStd
17:58:01   42 - cad/step/Limit_SW_PCB.FCStd
17:58:01   43 - sub-assemblies/3D-printed_corner.FCStd
17:58:01   44 - sub-assemblies/Base.FCStd
17:58:01   45 - sub-assemblies/Bed_bottom.FCStd
17:58:01   46 - sub-assemblies/Bed_full.FCStd
17:58:01   47 - sub-assemblies/Bed_top.FCStd
17:58:01   48 - sub-assemblies/Corner.FCStd
17:58:01   49 - sub-assemblies/Electronics.FCStd
17:58:01   50 - sub-assemblies/Emergency.FCStd
17:58:01   51 - sub-assemblies/Green.FCStd
17:58:01   52 - sub-assemblies/SW_X.FCStd
17:58:01   53 - sub-assemblies/SW_Y.FCStd
17:58:01   54 - sub-assemblies/SW_Z.FCStd
17:58:01   55 - sub-assemblies/Vertical_frame_XZ.FCStd
17:58:01   56 - sub-assemblies/XZ.FCStd
17:58:01   57 - sub-assemblies/Z_axis.FCStd

However, the zip, as you showed, shows it twice

leoheck · 2022-10-06T21:15:23Z

Ah, the list of files I was using did not have unique files. It is fixed now.
Zip is being created. But there are some issues when opening the main assembly file.

leoheck · 2022-10-06T21:18:32Z

These files are not being restored

Linked file: ../cad/Spacer_6.35.FCStd
Linked file: ../cad/FreeCAD-library/2020_V-slot_Al_extrusion/2020x370_V_slot_profile.FCStd
Linked file: ../cad/Googly_eyes.FCStd

But they exist

➜  diy-pcb-cnc git:(testing) ✗ find -name Googly_eyes.FCStd
./cad/Googly_eyes.FCStd

➜  diy-pcb-cnc git:(testing) ✗ find -name 2020x370_V_slot_profile.FCStd 
./cad/FreeCAD-library/2020_V-slot_Al_extrusion/2020x370_V_slot_profile.FCStd

➜  diy-pcb-cnc git:(testing) ✗ find -name Spacer_6.35.FCStd
./cad/Spacer_6.35.FCStd

Maybe my parser is not finding them.

leoheck · 2022-10-06T21:19:40Z

Yeah, they are not in the above list with 57 items.

leoheck · 2022-10-06T21:24:29Z

Bed_full has spacers
And these are links from links.
I was not expecting nested links yet.

hasecilu · 2022-10-06T21:29:54Z

The Import Part from A4 imports the external files as Links and Links have the Element Count property so I changed it and that generates all those sub-links

leoheck · 2022-10-06T21:31:06Z

I found your Bed_top has issues.
What is the most updated code you have? Is it on master or is it still in the testing branch?

hasecilu · 2022-10-06T21:32:46Z

These files are not being restored

Linked file: ../cad/Spacer_6.35.FCStd
Linked file: ../cad/FreeCAD-library/2020_V-slot_Al_extrusion/2020x370_V_slot_profile.FCStd
Linked file: ../cad/Googly_eyes.FCStd

Those files had the Element Count property used.

leoheck · 2022-10-06T21:33:49Z

but look what my FC says

leoheck · 2022-10-06T21:34:53Z

There is no linked object

leoheck · 2022-10-06T21:36:25Z

ah, the file is missing, please, check if it was added to the repo.

hasecilu · 2022-10-06T21:37:48Z

It is in the repo but it's not in the zip file

leoheck · 2022-10-06T21:38:06Z

yeah, I was looking the wrong file, sorry

leoheck · 2022-10-09T21:11:51Z

@hasecilu took some time but now it exports all files of the main assembly.
Now, considering your repo, if you try to export a sub-assembly. It does not work yet. Some paths need to be manipulated since the current flow only handles nested files/folders and not files and folders that are not inside of the same folder...

An example of that would be Bed_top.FCStd where most of the files come from outside ../cad

hasecilu · 2022-10-09T23:57:22Z

Cool, it's working!

An example of that would be Bed_top.FCStd where most of the files come from outside ../cad

I'm not sure if there are file location guidelines for (sub) assemblies, moving cad inside of sub-assemblies could be a workaround but don´t know if there is a way to move linked files without breaking the links in assemblies.

leoheck · 2022-10-10T02:13:29Z

I was thinking that I could export the parent folder that has both assembly and its parts.

So if the assembly and related files are like this:

myass.FCSd
../partslib/part1.FCstd

Then it would normalize paths, considering the folder that has both (all) the parts.
It may work even if the paths are not related, I guess.

Also, some things to do

Improve toolbar images
Add a dialog so the user can choose the zip name and path while exporting
Allow this to work with any kind of assembly (assemblies that do not have parts nested).

…n its folder

leoheck · 2022-10-10T03:43:57Z

Hey, @hasecilu now it exports assemblies with files in other folders. So it works with the Bed_top.FCSd the way I was thinking before. And the elegant thing would be to add a symlink to the exported assembly inside the root folder when the main assembly is not in the root folder. I will implement this tomorrow.

hasecilu · 2022-10-10T04:10:44Z

Tested and working!

And the elegant thing would be to add a symlink to the exported assembly inside the root folder when the main assembly is not in the root folder. I will implement this tomorrow.

Sounds nice, could be useful when a assembly is made of many subassemblies and you need to find the main one.

leoheck · 2022-10-10T15:28:35Z

I got it working, but it happens that FreeCad can not handle symlinks since the internal links won't be adjusted if a symlink is being used. It could be a bug on Freecad or just something that was not thought of yet.

Then since symlink is in another place, the linked files cannot be found anymore.

Freecad could identify a symlink is being used to load the file from the original place, instead.

leoheck · 2022-10-10T15:34:47Z

I've shared this finding here, let's see how your peers are going to react.
https://forum.freecadweb.org/viewtopic.php?f=3&t=72491

leoheck · 2022-10-10T15:44:13Z

I've pushed the code since it is harmless to the export process.
If the assembly being exported is not in the root path of the zip package, it creates a symlink (you can test it with the Bed_top)
The only issue is that Freecad opens it but the linked files are not in the right place anymore. (this should be fixed on Freecad side, I gues)

hasecilu · 2022-10-10T17:08:29Z

In Link>Linked object property the path of the file is absolute and depending where you extract the zip file the absolute path of the exported zip file will change.

In my case I extracted in .../DIY-PCB-CNC/sub-assemblies/Bed_top_asm4/ but the symlink is expecting files in another directory .../DIY-PCB-CNC/sub-assemblies/cad/Plate_03.FCStd.

If I extract in: $HOME/Documents/test/Bed_top_asm4 it's expecting files in $HOME/Documents/test/cad/Plate_03.FCStd.

Basically is skipping the Bed_top_asm4 folder which is where the zip file was extracted.

I think the new icons need a color bg, the green and yellow of before were fine.

leoheck · 2022-10-10T17:32:10Z

Ah, I will have to tweak paths a bit. This feedback is what I was expecting, thanks.

The icons are a bit blurry still, I may have to improve them better, just have to learn how. But check this new set for now, please.

hasecilu · 2022-10-10T17:45:03Z

Probably they are blurry because of the icon size, when bigger they look crisper

leoheck · 2022-10-10T18:51:44Z

In my case I extracted in .../DIY-PCB-CNC/sub-assemblies/Bed_top_asm4/ but the symlink is expecting files in another directory .../DIY-PCB-CNC/sub-assemblies/cad/Plate_03.FCStd.

Ah, yes. This is the issue I am talking about.
When you open a symlink, Freecad should follow the link, and then open the original file since it may have Linked files that have hardcoded or fixed paths.

leoheck · 2022-10-10T18:52:38Z

So the symlink is being created correctly. However, Freecad cannot open it fine, when it has Linked parts. Unless you are talking about a different problem.

leoheck · 2022-10-10T19:31:45Z

Now the user can select where the zip is exported and how to name it.

hasecilu · 2022-10-10T20:40:15Z

So the symlink is being created correctly. However, Freecad cannot open it fine, when it has Linked parts. Unless you are talking about a different problem.

It's the same issue. It looks like there is no differentiation between the files:

$ file ass.FCStd
ass.FCStd: symbolic link to subass/ass.FCStd
$ file subass/ass.FCStd
subass/ass.FCStd: Zip archive data, at least v2.0 to extract, compression method=deflate

leoheck added 2 commits October 5, 2022 18:34

Add buttons to list linked files (list/tree) and to export them to a …

30c07ef

…zip package.

Improve temporary icon

201d118

leoheck mentioned this pull request Oct 5, 2022

New features (list and export linked files) #369

Closed

Fix exporting zip when not in the same directory (may be the cae on 9…

68661ff

…9% of the time)

Handles the directories better

e0e8a15

leoheck added 2 commits October 6, 2022 18:08

change order of buttons

b4d8ee5

Fix files list to be exported (remove duplicates)

600b049

Fix missing files to export

05064e9

Fix paths when exporting a assembly that does not have nested parts i…

5e278d7

…n its folder

Add symlink to the assembly if it is not in the root of the zip

4569045

leoheck added 2 commits October 10, 2022 12:52

Update zip icon

ce4c2ae

Change buttons to show linked files as list and tree

2176e87

Improve background

1a36772

leoheck added 2 commits October 10, 2022 16:00

Improve toolbar icon name and tooltip

c294b46

User selects where to export the zip file and how to name it

28cf702

Fix wrong name of Report View

afd802f

leoheck mentioned this pull request Oct 11, 2022

[Bug] App::Link breaks when opening a Symlinked FCStd FreeCAD/FreeCAD#7596

Open

2 tasks

Fix wrong dict key and add Asm4_PartDesign bodies

d39212e

leoheck closed this Feb 22, 2023

leoheck deleted the list_linked_files branch May 29, 2023 18:40

List and Export linked files #370

List and Export linked files #370

Conversation

leoheck commented Oct 5, 2022

hasecilu commented Oct 5, 2022 • edited

leoheck commented Oct 6, 2022

hasecilu commented Oct 6, 2022

leoheck commented Oct 6, 2022

leoheck commented Oct 6, 2022

hasecilu commented Oct 6, 2022

hasecilu commented Oct 6, 2022

leoheck commented Oct 6, 2022

hasecilu commented Oct 6, 2022

leoheck commented Oct 6, 2022

hasecilu commented Oct 6, 2022 • edited

leoheck commented Oct 6, 2022

leoheck commented Oct 6, 2022

leoheck commented Oct 6, 2022

leoheck commented Oct 6, 2022

leoheck commented Oct 6, 2022

hasecilu commented Oct 6, 2022

leoheck commented Oct 6, 2022 • edited

hasecilu commented Oct 6, 2022

leoheck commented Oct 6, 2022

leoheck commented Oct 6, 2022

leoheck commented Oct 6, 2022 • edited

hasecilu commented Oct 6, 2022

leoheck commented Oct 6, 2022

leoheck commented Oct 9, 2022 • edited

hasecilu commented Oct 9, 2022

leoheck commented Oct 10, 2022

leoheck commented Oct 10, 2022

hasecilu commented Oct 10, 2022

leoheck commented Oct 10, 2022

leoheck commented Oct 10, 2022

leoheck commented Oct 10, 2022

hasecilu commented Oct 10, 2022

leoheck commented Oct 10, 2022

hasecilu commented Oct 10, 2022

leoheck commented Oct 10, 2022

leoheck commented Oct 10, 2022

leoheck commented Oct 10, 2022 • edited

hasecilu commented Oct 10, 2022

hasecilu commented Oct 5, 2022 •

edited

hasecilu commented Oct 6, 2022 •

edited

leoheck commented Oct 6, 2022 •

edited

leoheck commented Oct 6, 2022 •

edited

leoheck commented Oct 9, 2022 •

edited

leoheck commented Oct 10, 2022 •

edited