JFLarvoire
released this
Significantly improved pysetup.bat, and the other python management scripts for Windows
pysetup.bat now defaults to configuring the py.exe launcher as the default command for executing Python scripts.
When multiple instances of Python are configured correctly, it's possible to simultaneously run both Python 2.7 and Python 3.x scripts, by typing just their name (without the .py extension) at the command prompt. I'm preparing a blog post explaining how all that should be done, and how to fix common problems. (Spoiler: SysToolsLib's pysetup.bat will fix everything automatically for you!)
pysetup.bat, pip.bat and python.bat now have a common Python instance selection method, with instance #0 the current default. (Preferably py.exe as explained above.) Use option -? to get more information.
Other changes and new tools
- Renamed lessive.exe as trim.exe. (The French name meant nothing to anybody except for me!)
- Renamed the Bash directory as Shell, as most of the scripts there are actually generic Posix Shell scripts.
- Added an mcd function for Linux, alter-ego of the mcd.bat script for Windows. (Create a directory, and enter it, in a single command.)
- A new -C option in dirc.exe for Windows, displaying the file compression ratio.
- regx.bat: Added options -se and -ue to easily manage System and User Environment variables.
- which.exe does not report App Exec Link targets by default anymore, until we know how to reliably use that target with the other link parameters.
Important bug fixes
- Fixed the -z option in sector.exe, where a regression prevented erasing hard disks.
- Fixed dirsize which reported 0 on some Unix partitions.
And several other smaller improvements and bug fixes.
Assets
3
It's been a while since I last made a release, so here's a gift for Christmas!
New scripts
- distrib: Identify the Unix distribution and version. Works in any Posix shell, and in any Unix derivative.
- htmldec.tcl: Decode text strings that contain HTML entities. For use as a filter script with 1clip.exe/2clip.exe/12.bat.
- htmlenc.tcl: Encode text strings with the 3 basic HTML entities for & < >. For use as a filter script with 1clip.exe/2clip.exe/12.bat.
New features in existing scripts and programs:
- which.exe, dirc.exe, truename.exe now detect UWP applications execution links, and display their target.
- dirc.exe now reports WSL 2 symbolic links, and their targets. For WSL 1, it just reports it's an .
- which.exe now identifies pwsh.exe as PowerShell, and uses PowerShell rules when invoked inside pwsh.exe.
- TimeX.bat can now measure durations > 1 day, even over midnight, any day in the year.
- 2clip.exe, conv.exe now auto-detect UTF-8 and UTF-16 input, even without a BOM.
- 1clip.exe, 2clip.exe have new -8 and -16 options to force output to be UTF-8 or UTF-16.
- update.exe accepts target argument "D:=" as meaning "Same path as the source, but on drive D:"
- ipcfg.bat improved the network adapter name detection, and defined "VPN" as the alias for Pulse Secure VPN.
- conv.exe, detab.exe, lessive.exe, remplace.exe now accept -= as an alias for -same, meaning update a file in place.
Under the hood, in the MsvcLibX library
- The Windows build and release generation can now be done from the project root directory, like for Unix.
- All the Windows executables built go into the bin subdirectory in the project root directory.
- Functions readlink(), readdir(), lstat() now handle UWP App. Exec. links and WSL Linux symlinks as if they were standard symlinks.
- Added GNU C library routine asprintf(). Now used for debug output, where it avoid lots of fixed size buffers allocations.
Notes about UWP App. Exec. links
Universal Windows Platform (UWP) applications are new in Windows 10.
They are applications that can run on any types of Windows devices. (PCs, Phones, Xbox, IoT devices, etc)
Circa 2017, Windows 10 introduced a new kind of NTFS link, tagged as an IO_REPARSE_TAG_APPEXECLINK, or AppExecLink here for short.
Most UWP applications installations create an AppExecLink to the actual executable in "%LOCALAPPDATA%\Microsoft\WindowsApps".
This directory is in the Windows 10 PATH, and the links there avoid having to add each application's path to your global PATH.
These AppExecLinks differ from normal NTFS symbolic links in that they contain more than just a pointer to the executable.
I think the additional information helps Windows' CreateTask() speed up the UWP apps startup.
Anyway, 3 years later, Windows' cmd.exe and PowerShell 5.1 still report AppExecLinks as 0-bytes files.
Only the new PowerShell 7.0 correctly reports them as links.
Until now, the System Tools Library was not doing any better than cmd.exe.
But at last the new version released here handles them correctly.
For example if you have installed the Windows Terminal, which.exe now finds it correctly, and truename.exe and dirc.exe also:
C:\JFL\Temp>which wt
C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.4.3243.0_x64__8wekyb3d8bbwe\wt.exe
C:\JFL\Temp>which -v wt
# C:\Users\Larvoire\AppData\Local\Microsoft\WindowsApps\wt.exe # UWP App. Exec. Link
C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.4.3243.0_x64__8wekyb3d8bbwe\wt.exe
C:\JFL\Temp>which -l -v wt
# 2020-11-20 13:26:46 0 C:\Users\Larvoire\AppData\Local\Microsoft\WindowsApps\wt.exe -> C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.4.3243.0_x64__8wekyb3d8bbwe\wt.exe # UWP App. Exec. Link
2020-11-20 13:26:44 92672 C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.4.3243.0_x64__8wekyb3d8bbwe\wt.exe
C:\JFL\Temp>truename C:\Users\Larvoire\AppData\Local\Microsoft\WindowsApps\wt.exe
C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.4.3243.0_x64__8wekyb3d8bbwe\wt.exe
C:\JFL\Temp>dirc C:\Users\Larvoire\AppData\Local\Microsoft\WindowsApps\wt.exe
C:\Users\Larvoire\AppData\Local\Microsoft\WindowsApps
wt.exe -> C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.4.3243.0_x64__8wekyb3d8bbwe\wt.exe <APPEXECL> 2020-11-20 13:26:46
1 files or directories listed.
C:\JFL\Temp>
Assets
3
This release has much better support for Linux, and for the first time supports MacOS.
The build and installation procedures are the same for MacOS as for Linux, collectively referred to as Unix.
Other significant improvements
- All C tools now build in Unix without warnings.
- Unix
make installis now more standard, using the Unixinstallcommand internally.
It can also be dry-runned by runningmake -n install - install: New script for installing select tools in Unix. (Contrary to
make installwhich installs everything.) - update.exe: Several new options, and a few old ones renamed for consistency:
Renamed switches -e|--erase as -c|--clean. (By analogy withmake clean.)
Renamed option -T|-resettime as -R|-resettime.
Added switches -T|--tree, independent of -E|--noempty. (As the 2018 change linking the two features did more harm than good.)
Renamed options -S|--showdest as -D|--dest, and added -S|--source to explicit the default behavior.
Added options -C|--command to display the equivalent shell commands. (And thus display both the source and dest. files.)
Added option -B|--nobak to skip backup and temporary files. (*.bak, ~, ##) - dirc.exe: Added option -B to skip backup and temporary files.
- halve.bat: New filtering function for use with 12.bat.
- sml2.exe: Now preserves head spaces, if any. This allows successfully converting indented blocks of XML.
Major bug fixes
- update.exe: Fixed a serious usability issue when the target was a link to a directory.
- backnum, dirc, dirsize, rd, redo, update, zap: Fixed an issue which sometimes caused failures in Unix.
- which: Fixed wildcards search in Unix.
Please as usual many small improvements and minor bug fixes
Assets
3
New tools
- md2h.bat: Converts a markdown file to HTML, using GitHub APIs, then displays it in your Web browser.
This is useful to verify that the markdown you typed in a README.md file will display as intended after being pushed into GitHub.
Note: I’ve not found how to use GitHub’s actual CSS style sheets. So the generated HTML uses makeshift style sheets, that do not look as good as the real ones. This is not really an issue, since the objective is to verify the structure and contents of the HTML, not the aesthetics of the output. - WhereAmI.bat|WhereAmI.py|WhereAmI.tcl: Three versions of the same script, displaying your location based on your IP address.
I've copied these files over from another project I contributed to: https://github.com/JFLarvoire/today
It works by querying a free Web service, that infers your location based on your IP address.
Note if you're running this on an intranet with a web proxy, you'll get the location of the proxy, not your own location.
Note on note: WhereAmI.bat uses the global proxy configured in Windows. The other two rely on the http-proxy environment variable.
Significant improvements
- cpuid.exe: Now decodes CPUID(7) “Structured Extended Feature Flags”. This lists all processor improvements released in the last 10 years.
(Give it a try withcpuid -vto see what your processor is capable of!)
It also now gets the processor Second Level Address Translation (SLAT) capability from WMI.
Useful to know if the processor is compatible with HyperV 2016 and 2019, and with Docker. - zap.exe: Can now delete multiple directories using wild cards. See the -? help screen for usage information.
- inicomp.exe: Now works with .ini files with sections split in several disjoint parts.
Also added a new -f option to support files with non-standard settings without an =value. - ShadowCopy.ps1: Major performance improvement when searching for the previous versions of a file.
- cascade.tcl: A new -l option to list cascadable applications, or list the cascadable windows for one application.
- regsub.tcl: Added several options for controlling Case [in]dependant matching; Line or global mode; First or all matches.
- AutoRun.cmd:
AutoRun -inow installs the extension scripts fromAutoRun.d\listed inAutoRun.d\default.lst.
This should make it much easier to configure AutoRun on new systems, from the contents of the SysTools.zip distribution.
(Reminder: Among other good things, AutoRun allows having ahistorycommand in Windows, that can be piped togreporfindstr, like thehistorycommand in Linux.)
Plus as usual many other small improvements and bug fixes. See NEWS.md in the project root for the complete list.
Assets
3
JFLarvoire
released this
Main changes
-
The
which/which.exeprogram can now search for programs using wildcards.
This is very useful when you don't quite remember the exact name of a program.
Ex, try:which -a *zip*in Windows, orwhich -a '*zip*'in Unix -
New scripts
paths.batfor Windows, andpathsfor Unix, showing the PATH with one path per line, and making it easy to reorder it.
Optionally manages any other variable structured like PATH.
More intuitive to use than the old AddPaths.bat scripts they replace. -
New scripts
python.batandpip.bat, for managing multiple instances of Python on Windows.
(Useful now that Microsoft installs its own instances, without looking for non-Microsoft ones that may have been there before!) -
The new
Get-Console.batis a front-end toGet-Console.ps1, for capturing the console contents as coloured text into the clipboard. -
All Windows executables now include program properties, with a description, version, product infos, etc.
-
The ARM64 versions of the C programs now build without any warning...
But I've still not been able to test them for lack of a Windows 10/ARM64 test system.
Plus as usual many small improvements and bug fixes.
Assets
3
JFLarvoire
released this
Major novelties:
- The build system now supports Visual Studio 2019, and ARM64 versions of all *.exe programs are provided.
Important: I've not tested these ARM64 programs, as I don't have an ARM64 version of Windows.
If you have one and can help, please tell me in issue #23. - which.exe now implements a -i option for Windows, akin to which -i for Linux.
It efficiently detects DOSKEY aliases, cmd.exe internal commands, and PowerShell aliases and functions.
Runwhich -?for details on how to configure cmd.exe and PowerShell for that.
You can also find a detailed presentation of which.exe there. - A new AutoRun.cmd script, and an AutoRun.cmd.d subdirectory, to easily manage multiple AutoRun scripts.
Several aliases definitions are provided in AutoRun.cmd.d: history, pid, which.
See the Batch/AutoRun.cmd.d/README.md file for details.
You can find a detailed presentation of AutoRun.cmd there.
New scripts:
- DumpLink.bat: Display the contents of Windows Explorer *.lnk shortcuts.
- urldecode.tcl: An old script for decoding URLs encoded with %XX hexadecimal codes.
Other changes:
- All *.exe programs -V/--version now display the minimum Windows version supported.
- which.exe: Added option -- to stop processing switches.
- ag.exe: The silver searcher now expands wildcards in command-line pathnames.
- dirc.exe: Added option -ct to report equal files with different times.
- update.exe: Added option -- to stop processing switches.
Added option -P to show the file copy progress.
Added option -T to reset the destination file time to that of the source file, if they have identical sizes. - chars.exe: Added option -u to display Unicode characters or ranges.
Added option -v to display verbose information.
Improved error reporting when switching code pages.
Bug fixes:
- n.bat: Work around trailing spaces issue in Windows 10 v 2019-03.
- font.exe: Fixed the console font setting, which did not work well with TrueType fonts.
- cfdt.tcl: Fixed the --i2n option, which moved files to the current directory.
- regx.bat: Fixed the dir command when the pathname contains spaces.
- update.exe: Fixed a 2018-12-18 bug causing Error: Not enough arguments
Assets
3
This is a minor release with one important change:
The font.exe tool now has the ability to save and restore the console font.
This is very useful when another program unexpectedly changes the console font as a side effect.
This happens on one of my machines, where PowerShell always changes the font back to a Raster font. (I have no idea why, nor why this happens on this machine and not on the others.)
Anyway, to save and restore the cmd.exe shell font, use the following commands:
:# Save the current font settings in an environment variable
for /f "delims=" %f in ('font -s') do @set OLD_FONT=%f
:# Run the font-breaking command
powershell -c $PSVersionTable
:# Restore the initial font
font %OLD_FONT%
Another minor change in this release: The sml2.exe tool now defaults to ignoring XML parser warnings and errors. They can be re-enabled if desired.
Assets
3
JFLarvoire
released this
- TclSetup.bat is a new script for configuring Windows to run *.tcl files as command-line scripts, and *.tk as windowed scripts.
It's a much improved version of the old setup code that was in thetclsh.bat -sscript, that should also be easier to use.
Runtclsetupto test the current configuration;tclsetup -vfor detailed explanations;tclsetup -sto configure Tcl as proposed. - lower.tcl, upper.tcl, camel.tcl are three trivial scripts for converting text to lower case, UPPER CASE, and Camel Case respectively.
They're useful to convert text case in Windows apps that don't have that built-in capability. Ex: In web pages input fields:
Select the text to convert; Copy it to the clipboard; Run12 lower; Paste the updated clipboard back into the form. - history.bat displays the command history, like the history command in Linux.
Major changes:
- ShadowCopy.ps1 has new options -Mount, -Dismount, -Previous, and -Restore.
This makes it easy to browse the previous files versions, and restore one of them. See the built-in help for details. - 12.bat now temporarily switches to code page 65001, to avoid losing Unicode characters not in the current cmd shell code page.
- Ag.exe has been updated to master version 2.2.0, and now uses the new pthreads4w library version 3.0.0.
- update.exe has a new option -e to erase target files not in the source directory.
- hosts.bat is not dependent on elevate.exe anymore, when run as administrator. Plus several new options.
- regx.bat has a new md command to create a registry key.
- TimeX.bat option -t now displays the start and end times of eXecuted programs.
- tclsh.bat has an improved tclsh.exe search routine, and all the setup code has been removed (as a much better version is available in TclSetup.bat).
Plus lots of other small changes and fixes.
Assets
3
I thought I had it right in release 1.10 already...
Shame on me, there were still bugs with long path > 260 characters in some tools.
- Fixed long path > 260 bytes support in dirc.exe, dirsize.exe, truename.exe.
They now handle and display these paths correctly in any version of Windows.
Tested with many paths up to 450 bytes.
Note that other related tools like update.exe, zap.exe, md.exe, rd.exe worked right already indeed. :-) - Partially fixed it in redo.exe: Due to Windows limitations, it's still generally not
possible to change the current directory to paths > 260 bytes. But the updated redo.exe
can now generate commands with correct long paths, when using the {} argument replacement. - There's still a known problem with backnum.exe. It will be fixed in a future release.
Then one new program
- font.exe lists available fonts and some of their properties.
The -f option is useful to see the fixed-width fonts available in the console:
If available, I recommend using "Lucida Console" for all european and american users,
as it supports all latin, greek, cyrillic, and hebrew characters in any code page.
Finally the sml2.exe program introduced last time has several improvements and bugs fixes.
It's still not as reliable as sml.tcl though.
Assets
3
JFLarvoire
released this
New tools, and original new features in a very old one
-
which.exe: Two significant new features, without equivalent in the original Unix command:
-
A new
-loption for listing files like thels -lcommand.
This is useful in combination with the-aoption to diagnose a common problem:
Why isn't it the new tool that I just built and installed that runs?Example in Windows:
C:\JFL\Temp>which -a -l which 2018-03-22 17:41:50 C:\JFL\Tools\WIN64\which.exe 2018-03-22 17:41:48 C:\JFL\Tools\WIN32\which.exe 2010-05-18 16:38:26 C:\JFL\Tools\WIN32\which.bat 2012-12-17 07:06:47 C:\JFL\Tools\ezWinPorts\Win32\bin\which.exe 1999-11-10 23:00:00 C:\JFL\Tools\UnxUtils\usr\local\wbin\which.exe 2008-08-10 16:23:23 C:\JFL\Tools\GnuWin32\bin\which.exeExample in Linux: (Notice the capital W in Which, to avoid conflicts with the native which.)
larvoire@JFLZB:/JFL/Temp$ Which -a -l which 2016-01-26 19:17:55 /usr/bin/which -> /bin/which 2016-01-26 19:17:55 /bin/whichNotice that the timestamp is that of the final executable file, not that of the link.
This is on purpose, as we want to easily understand which executable is the newest. -
A new
-voption for adding verbose comments about why some files were rejected. Implies the-loption.
Again, this is useful to diagnose common issues, like PowerShell or Bash ignoring the program in the current directory, or Unix shells ignoring files that are not marked executable.Example in Windows:
PS C:\JFL\Temp> which -a -v ag # 2018-03-14 12:52:54 .\ag.exe # PowerShell does not search in "." 2018-03-16 11:55:29 C:\JFL\Tools\WIN64\ag.exe 2018-03-16 11:55:28 C:\JFL\Tools\WIN32\ag.exeExample in Linux: (Notice the capital W in Which, to avoid conflicts with the native which.)
larvoire@JFLZB:/JFL/Temp$ Which -v README # 2018-03-26 12:16:24 /usr/local/bin/README # Not executable
-
-
2note.exe: A new tool to pipe the output of a command directly into a new instance of Notepad.
This is useful when running a command outputing a lot of data, possibly more than the console buffer can hold, and you want to review it before throwing it away.
Ex:my_huge_build_command | 2note -
sml2.exe: A rewrite of the sml.tcl tool in C, based on the libxml2 library.
Advantages: Much faster than sml.tcl; Additional options for using libxml2 filtering and reformatting capabilities.
Drawbacks: Does not preserve non-significant spaces, so files converted twice may not be binary identical to the original file. (although they're logically identical). -
mountw.bat & umountw.bat: New tools for easily mounting and unmounting Windows Images (*.wim files).
Useful when managing a Windows Deployment Server, to customize boot and install images with your favorite tools. -
codepage.exe: Now displays the console font, and outputs a warning if the console font is a raster font that cannot display all characters in the current code page.
Under the hood, there were many improvements in the make system.
This new release was built with Visual Studio 8/2005 and Visual Studio 15/2017.
And it's the first time in 30 years that it entirely builds without any warning. :-)