Suggestion: Add UI to specifiy build file name

[joshbooker]

This tool is great! Thank you!

It'd be nice to have the ability in the UI to changes the build file name - defaults to and updates the value in dbs-properties.json

Use case:
I find it helpful to compare changes between versions of Access files by using git diffs like so:

1. VSC Export Source mdb
2. VCS Export Target mdb
3. git init Target Export Dir
4. git commit 'initital' Target Export dir
5. overwrite target export dir with contents of source export dir
6. Examine/stage git diffs
7. Commit
   (would be interested if folks have similar or other workflows)

In the above case, the task is to examine and apply changes from source mdb to target mdb and then build target mdb. Sometimes the mdb files have different file names. In this case, a build from the updated target export dir builds an mdb having the source mdb name since the file name comes from the dbs-properties.json and vbe-project.json files which were overwritten from the source export (Step5). Additionally, the source mdb is backed up as expected. To build a file of a different name, you can change the file name values in the two aforementioned files before build.

It'd be nice to have the ability in the UI to changes the build file name - defaults to and updates the value in dbs-properties.json

Thanks for listening. Have a great day!

[joyfullservice]

Hi Josh,

A little background on this... Most build tools and technologies center around the source code files. You do your development directly in the source code files, merge and commit changes, and ultimately build your project from those source files. In most of those cases you have options to specify your build file name and path, but the build is an artifact of the source code.

Microsoft Access is different in that the "build" is actually the development environment. You do most of your work directly in Microsoft Access and periodically/incrementally export the database to source files as a practical way to track changes and create a snapshot of source files that can be used to build the database to reflect a specific point in time. Because of this difference, the database maintains a much tighter relationship to the path of the source files.

Version 4 includes a lot more conflict checking to make sure changes in the source files do not inadvertently overwrite modified database objects, and to ensure exports of database changes do not inadvertently overwrite source files that were modified from an external process. The vcs-index.json file is used to manage the state between the source files folder and a specific database build. If you switch things around by changing the build path, the index might get out of sync with the original build.

This does bring up an interesting idea of adding a new option to "Build As..." a different database file. Kind of like a "Save As" operation in Office applications, or an "Export To" operation in an imaging software. This proposed feature would open a "Save As" dialog where you would specify a name and path for the database, and then build the database in the specified location without changing the index file or affecting the normal build. Let me know what you think about that idea...

Workflow Suggestions

Thinking through the workflow you described above, here are a couple suggestions/ideas I have found helpful:

External Diff Tool

I will often use WinMerge to compare two folders of source files. This lets me drill down into subfolders and easily compare the source differences between two different versions. Source files can be dragged and dropped between the folders, or even updated directly before building the database from source. Sometimes I will run an export, then rename the export folder and run an export again to generate a fresh copy of the export folder so I can review source changes between different versions of the database.

You can also use two copies of the database file with different names, and compare the source folders for each version. Of course you will have some differences from the actual database file name, but just about everything else should match up pretty well.

Git Branch Merging

Another approach is to export different versions of a database into different branches of a git repository. You can then use some of the built-in tools to review and resolve conflicts, and merge changes from one branch to another. I do something like this to manage this add-in project. I have a master branch with the official releases of the add-in, and a dev branch where I do most of the development work.

Hope that helps!

[joshbooker]

Thanks for the prompt and in depth response.
Save As sounds good.

[joyfullservice]

I haven't done extensive testing on this, but the new feature seems to work. I can build a database from source files to a new location and new file name without impacting the index or options of the selected source files.

This should roll out with the next beta release, and you can try it out and let me know what you think. 😄

[joshbooker]

Built and installed the dev branch and tried Build As. I like it. The toolbar UI is great as well BTW. Thanks.

Still trying to get my head wrapped around tight coupling of source and built file. I mean I understand of course that dev work is done in Access so coupling is important but in context of my 'compare two files' use case, it seems the coupling is more restrictive.

For example, I find myself wanting to both select the source dir and also Build As another name. I haven't played with merge yet but it seems I may be able to do a similar compare workflow IF I were able to select a different source dir. Perhaps it will become more clear once I get used to it but it just seems selecting both source dir and build target would open all possibilities (with appropriate caveats and warnings of course regarding overwriting changes etc etc)

Thanks for listening. PS - this is not criticism. I love your work. Just giving my $0.02 as a newbie to this great tool.

[bclothier]

FWIW, and this is just my opinion/suggestion. Up to now, the traditional approach has to been to make a separate copy of Access file every time we might make a major change or need to do experimentation. Carrying this habit over into git repository actually is counterproductive and defeat the point of having a git repository. To really leverage it, you should not have to make a separate source directory but rather have only single directory and maintain multiple git branches, which will then allow you to 1) seamlessly change from one branch to another, 2) do a diff between branches, 3) control when you are ready to merge one branch into another, 4) have a history of multiple commits available to help you analyze how one branch diverges from another branch.

In other words, a different paradigm is needed when you want to compare sources. I understand the need for build as since it's simpler to have multiple built .accdb files to do some interactive testing but in the terms of comparing sources, you can't beat branching. So you should never have multiple source directories but rather branches.

Just my $0.02ZWD.

[joshbooker]

Good feedback thanks for taking the time. I don’t often use branches but should try more.
If I understand correctly your approach would be

1. Export target
2. Git init, commit 'main'
3. Create, checkout branch 'source'
4. Export source to same dir
5. Compare branches

correct?

thanks again.

[bclothier]

Yes, you get the idea.

I'll add that when you are ready to merge, you would switch back to 'main', then pull from the 'source', which will then bring the 'main' up to the date with the 'source'. You can then delete the branch and then create a new branch.

Furthermore, you would be able to selectively commit only what you want to ultimately merge into the 'main' branch. The exact process depends on how you do it/what git UI client tool you may but generally, you'd select only files or hunks (e.g. the changes within the file as shown in a diff tool) and commit only those changes. That way you can control what goes into the commit and ultimately is then pulled into another branch.

HTH.

[joyfullservice]

Great suggestions! Just a couple more thoughts to add on the practical side...

The default setup uses the .gitignore file to exclude the Access binary files from version control. This means that when you switch branches, the existing database file stays unchanged. You can even keep Access running with your database file open while switching branches. After switching branches, just click Build From Source on the toolbar, and in a few seconds you will have a fully operational database built from the updated set of source files.

Another tip to be aware of is that if you click Build From Source or Build As... with no database currently open, you will be initially prompted to select a source files folder. This is a fairly new functionality added within the past month allowing the add-in to load into Access without a database file being open. (See #405)

I definitely second the recommendation to take advantage of the branching functionality in git. It really opens up the power of version control and source-based development workflows. The source files is really the only place you can practically and comprehensively compare different versions of a database.

As you get more familiar with merging changes between various sets of source files, just keep in mind that a few areas like form and report layouts have sequential identifiers and won't play as well with complex merges. Changing control properties is fine, but if you are adding and removing controls, you want to keep those changes on the same branch.

It is definitely a mental paradigm shift to think of the source files as the ultimate definition of a database project, but I have found this to be a rewarding transition. Instead of the "black box" of a binary database file where I often forget about little changes I made here and there, the source files in git provide a transparent visibility into every granular change I have made over time. I can also seamlessly collaborate with other developers. Integrating a change is as simple as accepting a PR and building (or merging) from source.

[bclothier]

This makes me think we could do a bit better in this area. The issue here is that when we checkout a branch, the .accdb file won't be in synchronization and it would be not fun if the developer forgot to build/merge.

At minimum, I think the add-in could at least track what branch the .accdb was built from and validate whether the current git branch is the same and if not, to warn the developer with a prompt to merge the changes into the .accdb file. That may help raise awareness and avoid polluting the branches with changes from .accdb file that didn't originate with the same branch.

[joshbooker]

This makes me think we could do a bit better in this area. The issue here is that when we checkout a branch, the .accdb file won't be in synchronization and it would be not fun if the developer forgot to build/merge.

At minimum, I think the add-in could at least track what branch the .accdb was built from and validate whether the current git branch is the same and if not, to warn the developer with a prompt to merge the changes into the .accdb file. That may help raise awareness and avoid polluting the branches with changes from .accdb file that didn't originate with the same branch.

Was thinking the same thing but without much branching experience couldn't verbalize yet. Basically branching is good way to compare versions but the tight coupling problem still exists and in some ways is more apt to mistake without branch-binary relationship.

[joshbooker]

off topic - should I open a discussion instead?

• Have you any experience with a 32-bit and 64-bit branch in the same project? I mean it's probably not ideal and I intend to get the servers on same bitness eventually but are there pitfalls using VCS? Besides handling the 'PtrSafe' declarations are there other pitfalls like code reference guids or something? If so, thanks in advance for saving me the hassle as I won't even try it.
• any idea why the toolbar doesn't show on one system (64-bit) where I've installed 4.0.15 while it does on the other (32-bit)

[bclothier]

Let's keep the issues focused so for those questions, please start a new discussion, thanks!

[joyfullservice]

This makes me think we could do a bit better in this area. The issue here is that when we checkout a branch, the .accdb file won't be in synchronization and it would be not fun if the developer forgot to build/merge.

At minimum, I think the add-in could at least track what branch the .accdb was built from and validate whether the current git branch is the same and if not, to warn the developer with a prompt to merge the changes into the .accdb file. That may help raise awareness and avoid polluting the branches with changes from .accdb file that didn't originate with the same branch.

That is one of the key roles of the vcs-index.json file. The reality is that we can't assume that the binary is in sync with the source, even if we know which branch it was last sync'd with. For that matter, we don't even know with confidence that the source files themselves are in sync with the last commit. (You could have unversioned files, uncommitted changes, stashed changes, etc...)

To me, it seemed that the most robust approach was to actually compare the database objects with the source files themselves, and use the index as a third element of comparison to determine whether it was the source files or database objects that changed. That's why most objects have a "hash" that would change when the object or source file is modified. This allows us to detect changes and display a conflict resolution dialog so the user can decide how they want to proceed.

Since the index file is committed to version control alongside the source files, it stays in the relevant branch. Before switching to a new branch, I first export from the database to make sure all my changes are saved, then after switching to the new branch, I build from source to make sure my database reflects the current set of source files. The automatic backups and conflict resolution screens give me a bit of a safety net if I forget a step in this process.

[joshbooker]

The struggle in my mind is going from Access where path/fileName is everything for version control, to source while two Access versions need the same name and same source dir.

A lightbulb just went on for me.

1. Always know what branch is checked out
2. Always Export before switching branches
3. Always Build after switching branches

I should just trust the tool but do agree with @bclothier it would help ones leap of faith to include branch name in the index file, and in the built Access file and a warning on Export if branch not matching the local repo.

Thanks for listening.

[joyfullservice]

Thanks, Josh! This is helpful feedback.

[hecon5]

I know I'm late to the party (also this feature seems to work), but while I'd normally say "yah, branch diffing is the way to go", in the case of Access development, we've found Worktrees are really really handy.

This allows us to have a New Feature, Beta/PreRelease, and Stable versions deployed and cross check features, and have each branch checked out all the time. This has avoided us releasing a beta/pre-release to the bulk of our users, which some get all worked up about.

[mwolfe02]

Recently, I've been coming around to the idea of using feature flags in place of branching. Branching inevitably requires merging, which is often painless but can occasionally be agonizing. With feature flags, you hide your incomplete work behind some sort of boolean flag or--in the case of a new form or report--simply don't expose it in the user interface. Certain situations--like introducing a third-party dependency that requires an updated deployment process--still benefit from branching, but I'd say 95%+ of new development can be handled via feature flags.

[hecon5]

We also use feature flags, and combine them with branches as the "stable" and "beta" flags are set in the branch; we still require branches, especially for things that affect the underlying operation (not just new forms), which can affect more than just the UI, we've not found a "good" way to handle this, but I'm all ears for hearing about it.

I keep meaning to share some of our build out process, but so many other things sit between that and me right now.