HAST-295: Open-source Hast.Core

.github/workflows/build-and-test.yml

@@ -9,46 +9,25 @@ on:
 
 jobs:
   build-and-test-larger-runners:
-    strategy:
-      matrix:
-        # This is an undocumented way of creating a matrix of multiple values, see:
-        # https://stackoverflow.com/questions/66025220/paired-values-in-github-actions-matrix.
-        build: [
-            {solution: Hastlayer.SDK.Client.sln, machine-type: "['ubuntu-22.04']", test-disable: true},
-            {solution: Hastlayer.SDK.Linux.sln, machine-type: "['ubuntu-22.04']", test-disable: false},
-            {solution: Hastlayer.SDK.sln, machine-type: "['windows-2022-8core']", test-disable: false}
-          ]
-    if: github.ref_name != 'dev'
+    if: github.ref_name != github.event.repository.default_branch
     name: Build and Test (larger runners)
     uses: Lombiq/GitHub-Actions/.github/workflows/build-and-test-dotnet.yml@dev
-    secrets:
-      CHECKOUT_TOKEN: ${{ secrets.LOMBIQBOT_GITHUB_PERSONAL_ACCESS_TOKEN }}
     with:
-      machine-types: ${{ matrix.build.machine-type }} 
+        # If we'll have another concurrent job in this workflow then the next line should be removed.
+      cancel-workflow-on-failure: "false"
+      machine-types: "['ubuntu-22.04-4core']"
       timeout-minutes: 15
       build-configuration: Debug
-      build-solution-path: ${{ matrix.build.solution }}
-      test-disable: ${{ matrix.build.test-disable }}
 
   build-and-test-standard-runners:
-    strategy:
-      matrix:
-        build: [
-            {solution: Hastlayer.SDK.Client.sln, machine-type: "['ubuntu-22.04']", test-disable: true},
-            {solution: Hastlayer.SDK.Linux.sln, machine-type: "['ubuntu-22.04']", test-disable: false},
-            {solution: Hastlayer.SDK.sln, machine-type: "['windows-2022']", test-disable: false}
-          ]
-    if: github.ref_name == 'dev'
+    if: github.ref_name == github.event.repository.default_branch
     name: Build and Test (standard runners)
     uses: Lombiq/GitHub-Actions/.github/workflows/build-and-test-dotnet.yml@dev
-    secrets:
-      CHECKOUT_TOKEN: ${{ secrets.LOMBIQBOT_GITHUB_PERSONAL_ACCESS_TOKEN }}
     with:
-      machine-types: ${{ matrix.build.machine-type }} 
+      # If we'll have another concurrent job in this workflow then the next line should be removed.
+      cancel-workflow-on-failure: "false"
       timeout-minutes: 30
       build-configuration: Debug
-      build-solution-path: ${{ matrix.build.solution }}
-      test-disable: ${{ matrix.build.test-disable }}
 
   post-pull-request-checks-automation:
     name: Post Pull Request Checks Automation

.github/workflows/create-jira-issues-for-community-activities.yml

@@ -9,7 +9,7 @@ on:
     types: [opened]
 
 jobs:
-  call-create-jira-issues-for-community-activities-workflow:
+  create-jira-issues-for-community-activities:
     uses: Lombiq/GitHub-Actions/.github/workflows/create-jira-issues-for-community-activities.yml@dev
     secrets:
       JIRA_BASE_URL: ${{ secrets.DEFAULT_JIRA_BASE_URL }}

.gitmodules

@@ -1,11 +1,3 @@
-[submodule "Hast.Remote.Bridge"]
-	path = Hast.Remote.Bridge
-	url = https://github.com/Lombiq/Hastlayer-Remote-Bridge.git
-	branch = dev
-[submodule "Hast.Core"]
-	path = Hast.Core
-	url = https://github.com/Lombiq/Hastlayer-Core.git
-	branch = dev
 [submodule "Lombiq.Arithmetics"]
 	path = Lombiq.Arithmetics
 	url = https://github.com/Lombiq/Arithmetics.git

Docs/DevelopingHastlayer.md

@@ -8,28 +8,10 @@ How to work on Hastlayer itself?
 - Software that was written previously, without knowing about Hastlayer should be usable if it can live within the constraints of transformable code. E.g. users should never be forced to use custom attributes or other Hastlayer-specific elements in their code if the same effect can be achieved with runtime configuration (think about how members to be processed are configured: when running Hastlayer, not with attributes).
 - If some code uses unsupported constructs it should be made apparent with exceptions. The hardware implementation silently failing (or working unexpectedly) should be avoided. Exceptions should include contextual information (e.g. the full expression or method of the problematic source, link to the GitHub issue on adding support for it; see the `AddParentEntityName()` extension method) and offer hints on possible solutions.
 
-## Maintaining the flavors of the Hastlayer solution
-
-Generally the *client* branch should be only merged to, but never from, the *dev* branch.
-
-There are separate solution files for the two flavors that only differ in whether they include *Hast.Core*. Should the solution change then make the changes in the dev solution file, copy it over the client one and remove the *Hast.Core* solution folder. If you switch between the two solutions while on the *dev* branch then temporarily configure the Hastlayer shell to use the Client flavor (see `IHastlayerConfiguration`).
-
-To allow the same code in the samples and elsewhere to support both scenarios dynamic library loading is utilized. For this to work *Hast.Core* projects should adhere to the following:
-
-- The built DLL name to start with "Hast."
-- Their projects need to be located in a subfolder of *Hast.Core*.
-- Both the Debug and Release build output directories should be set just to *bin\\*.
-
-If a *Hast.Core* projects needs to have types accessible in the Client flavor then create a corresponding `Abstractions` project. Such projects should follow the same rules listed above as *Hast.Core* projects with the only difference being that they should be located in a subfolder of *Hast.Abstractions*. Exceptions are projects that are directly added as imported extensions in `Hast.Layer`: Those can be just normal projects (like `Hast.Transformer.Abstractions`).
-
-Note that if either kinds of projects reference another project that should be treated in the same way, with a manifest file and build output directories set (see e.g. `Hast.VhdlBuilder`).
-
-When merging from the `dev` to the `client` branch make sure to not merge entries for newly added *Hast.Core* subrepos (or remove them after the merge).
-
 ## When creating a new project
 
 If you add a new project to the solution make sure link the *SharedAssemblyInfo.cs* file from the root to its Properties node. See the comments in the file on what that is.
 
 ## Notes on dynamically linked projects
 
-Be aware that Hastlayer is a modular application using dynamic linking to scan for and attach non-essential components; for example the Core projects. Because of this if you change something in those (i.e. in projects that are not directly or indirectly statically referenced from the currently executing assembly), then you need to explicitly build the projects in question so msbuild can deploy them to the dependencies folder.
+Be aware that Hastlayer is a modular application using dynamic linking to scan for and attach non-essential components; for example the Core projects. Because of this if you change something in those (i.e. in projects that are not directly or indirectly statically referenced from the currently executing assembly), then you need to explicitly build the projects in question so MSBuild can deploy them to the dependencies folder.

Docs/GettingStarted.md

@@ -5,7 +5,6 @@
 To begin working with Hastlayer you'll need the following:
 
 - The SDK and Hardware Framework repositories cloned.
-- Access to Hastlayer Remote Services, which does the actual .NET to hardware transformation. Evaluation, academic and personal access is currently free and you can sign up on [hastlayer.com](https://hastlayer.com).
 - A compatible FPGA board. You have the following options here:
   - For simpler workloads and testing: The [Nexys A7 (formerly known as Nexys 4 DDR)](https://store.digilentinc.com/nexys-a7-fpga-trainer-board-recommended-for-ece-curriculum/) board (which is **NOT** the same as the non-DDR Nexys 4, be sure to purchase the linked board!) is suitable. The **Nexys A7-100T** version is required. Note that this is a relatively low-end development board that can't fit huge algorithms and it only supports slow communication channels. So with this board Hastlayer is only suitable for simpler algorithms that only need to exchange small amount of data.
   - For academic workloads: Microsoft's FPGA platform, [Project Catapult](https://www.microsoft.com/en-us/research/project/project-catapult/) is supported too, which offers high-end hardware. You'll need to apply for a cloud Catapult node via the [Project Catapult Academic Program](https://www.microsoft.com/en-us/research/academic-program/project-catapult-academic-program/). Be sure to [let us know](https://hastlayer.com/contact) if you'd like to use Catapult and we'll help you get going.
@@ -18,32 +17,19 @@ To begin working with Hastlayer you'll need the following:
 
 Catapult acknowledgment: The authors acknowledge the [Texas Advanced Computing Center (TACC)](http://www.tacc.utexas.edu) at The University of Texas at Austin for providing HPC resources that have contributed to the development of this project. This material is based on work supported by access to [Project Catapult](https://www.microsoft.com/en-us/research/project/project-catapult/) hardware and technology donated by Microsoft.
 
-If you have a compatible FPGA board you can run the default sample even without having access to Hastlayer Remote Services.
-
-## Flavors of Hastlayer
-
-The Hastlayer components come in two "flavors" with corresponding branches in their repositories:
-
-- Developer (*dev* branch): This is used by developers of Hastlayer itself. It includes the full source code. Most possibly you don't need this one.
-- Client (*client* branch): Used by end-users of Hastlayer who run Hastlayer in a client mode, accessing *Hast.Core* as a remote service, i.e. Hastlayer Remote Services. *Hast.Core* encompasses those components of Hastlayer that do the heavy lifting of software to hardware transformation. Most possibly you need this one.
-
-You'll see this terminology be used further on.
-
 ## First steps
 
 These would be your first steps on starting to work with Hastlayer by getting the samples working:
 
-1. Clone the necessary repositories with git. Always checkout the `client` or `dev` branch corresponding to your flavor. Make sure to allow Git to initialize submodules!
+1. Clone the necessary repositories with git. Make sure to allow Git to initialize submodules!
    1. Clone the [Hastlayer SDK repo](https://github.com/Lombiq/Hastlayer-SDK).
    2. Clone of the the Hardware Framework repos corresponding to your choice of hardware platform:
       - The [Hastlayer Hardware Framework - Xilinx repo](https://github.com/Lombiq/Hastlayer-Hardware-Framework---Xilinx) for the Nexys A7.
       - The [Hastlayer Hardware Framework - Xilinx Vitis repo](https://github.com/Lombiq/Hastlayer-Hardware-Framework---Vitis) for Alveo Data Center Accelerator Cards is already included as a git submoule so you don't have to clone it separately.
       - The [Hastlayer Hardware Framework - Catapult repo](https://github.com/Lombiq/Hastlayer-Hardware-Framework---Catapult) for Microsoft Catapult.
 2. Set up the hardware project as explained in the Hardware Framework's documentation and program the FPGA for the first time.
-3. Open the Visual Studio solution of the SDK corresponding to your flavor of Hastlayer.
-4. Set the `Hast.Samples.Consumer` project (under the *Samples* folder) as the startup project here. If you're working in the *client* flavor then you'll need to configure your credentials, see that project's documentation.
-5. Start the sample project. That will by default run the sample that is also added by default to the Hardware project.
-6. You should be able to see the results of the sample in its console window.
+3. Open the Visual Studio solution and start the `Hast.Samples.Consumer`. That will by default run the sample that is also added by default to the Hardware project.
+4. You should be able to see the results of the sample in its console window.
 
 If everything is alright follow up with the rest of this documentation to write your first own Hastlayer-using algorithm. You can also check out the many documented samples under the *Samples* solution folder.
 

Docs/ReleaseNotes.md

@@ -5,6 +5,7 @@ Note that the Hardware Framework projects have their own release cycle and relea
 ## vNext
 
 - The consumer sample app now uses [Terminal.Gui](https://github.com/migueldeicaza/gui.cs) to provide an easy to use GUI within a terminal window. Command-line configurability remains and is even extended.
+- Hastlayer is now fully open-source! Previously `Hast.Core` (the core logic that did the .NET-to-hardware transformation) was private. But now the whole of Hastlayer is available for you as fully open source software!
 
 ## v1.2.1, 27.10.2021
 

Docs/WorkingWithHastlayer.md

@@ -5,7 +5,7 @@
 The Hastlayer developer story is not ideal yet - we're working on improving it by [making the SDK available from NuGet](https://github.com/Lombiq/Hastlayer-SDK/issues/35). For now the below one is the easiest approach to add Hastlayer to your application:
 
 1. Clone the Hastlayer repository into a subfolder of your application.
-2. Copy the Hastlayer solution file corresponding to your Hastlayer flavor and use that to add your own projects to (you'll need to change project paths there to point to the Hastlayer subdirectory; [this example](Attachments/Hastlayer.SDK.Client.sln) shows how a Client solution file looks if Hastlayer is cloned to a folder named "Hastlayer", but this is just a static sample, do copy the latest one!). This way you'll have all the necessary projects added. Alternatively you can also add the Hastlayer projects to your existing solution, just make sure to add all of them.
+2. Copy the Hastlayer solution file and add your own projects to it. This way you'll have all the necessary Hastlayer projects available. Alternatively you can also add the Hastlayer projects to your existing solution, just make sure to add all of them.
 3. In the project where you want to use Hastlayer add the necessary initialization code (as shown in the samples) and the necessary project references (Visual Studio will suggest adding the right projects most of the time, otherwise also take a look at the samples).
 
 When Hastlayer is updated you can just pull in changes from the official Hastlayer repository, but you'll need to keep your solution file up to date by hand.
@@ -106,7 +106,7 @@ Think of these as breakpoints where you read out variable values with the debugg
 Even if the algorithm doesn't properly terminate you can use this technique, but you'll need to inspect the content of the memory on the FPGA; for the Nexys A7 you can do this in the Xilinx SDK's Memory window (everything written with `SimpleMemory` starts at the address `0x48fffff0`).
 
 ### Checking the decompiled source
-When you're working with the Developer flavor of Hastlayer it can also help to see what the decompiled C# source code looks like. You can save that to files, see `Hast.Transformer.DefaultTransformer` and look for `SaveSyntaxTree` (this is enabled in Debug mode by default).
+When you're working with the full source of Hastlayer it can also help to see what the decompiled C# source code looks like. You can save that to files, see `Hast.Transformer.DefaultTransformer` and look for `SaveSyntaxTree` (this is enabled in Debug mode by default).
 
 ### Dumping (and loading) SimpleMemory content
 You can store the contents of a `SimpleMemory` instance in a binary format, also as a file. Similarly you can load them into a `SimpleMemory` too.

Hast.Abstractions/Hast.Transformer.Abstractions/Configuration/TransformerConfiguration.cs

@@ -17,26 +17,15 @@ public class TransformerConfiguration
     /// Gets the list of the member invocation instance counts, i.e. how many times a member can be invoked at a given
     /// time.
     /// </summary>
-    public IEnumerable<MemberInvocationInstanceCountConfiguration> MemberInvocationInstanceCountConfigurations
-    {
-        // Since _memberInvocationInstanceCountConfigurations is a ConcurrentDictionary the order of its items is not
-        // necessarily the same on all machines or during all executions. Thus we need sorting so the transformation ID
-        // is deterministic (see DefaultTransformer in Hast.Transformer). Also, ToArray() and the setter are needed for
-        // JSON de/serialization when doing remote transformation.
-#pragma warning disable S2365 // Properties should not make collection or array copies
-        get => _memberInvocationInstanceCountConfigurations.Values.OrderBy(config => config.MemberNamePrefix).ToArray();
-#pragma warning restore S2365 // Properties should not make collection or array copies
-
-        private set
-        {
-            _memberInvocationInstanceCountConfigurations.Clear();
-
-            foreach (var configuration in value)
-            {
-                AddMemberInvocationInstanceCountConfiguration(configuration);
-            }
-        }
-    }
+    /// <remarks>
+    /// <para>
+    /// Since _memberInvocationInstanceCountConfigurations is a ConcurrentDictionary the order of its items is not
+    /// necessarily the same on all machines or during all executions. Thus we need sorting so the transformation ID is
+    /// deterministic (see DefaultTransformer in Hast.Transformer).
+    /// </para>
+    /// </remarks>
+    public IEnumerable<MemberInvocationInstanceCountConfiguration> MemberInvocationInstanceCountConfigurations =>
+        _memberInvocationInstanceCountConfigurations.Values.OrderBy(config => config.MemberNamePrefix);
 
     /// <summary>
     /// Gets or sets a value indicating whether to use the SimpleMemory memory model that maps a runtime-defined memory

Hast.Abstractions/Hast.Vitis.Abstractions/Docs/Nimbix.md

@@ -36,7 +36,7 @@ You can have multiple such compilations running at the same time, as there are e
 
 This is assuming that you are going to run `Hast.Samples.Consumer` with the image processing sample but other apps will behave similarly.
 
-1. Run the `dotnet Hast.Samples.Consumer.dll -device "Alveo U280" -sample ImageProcessingAlgorithms` command. Add the `-verify` switch if you want to verify whether the hardware output is the same as the software one, i.e. the device works properly. Also add the `-appname` and `-appsecret` switches if you don't have these hard-coded in the app.
+1. Run the `dotnet Hast.Samples.Consumer.dll -device "Alveo U280" -sample ImageProcessingAlgorithms` command. Add the `-verify` switch if you want to verify whether the hardware output is the same as the software one, i.e. the device works properly.
 3. If it ran successfully, check the output picture by typing `thunar contrast.bmp`.
 
 

Hast.Abstractions/Hast.Vitis.Abstractions/Models/VitisBuildConfiguration.cs

@@ -1,6 +1,3 @@
-using Hast.Common.Enums;
-using Hast.Common.Interfaces;
-
 namespace Hast.Vitis.Abstractions.Models;
 
 public class VitisBuildConfiguration
@@ -23,10 +20,6 @@ public class VitisBuildConfiguration
     /// </summary>
     /// <remarks>
     /// <para>It prompts via the command line so this is not suitable for GUI applications.</para>
-    /// <para>
-    /// This setting is ignored in the <see cref="IHastlayerFlavorProvider.Flavor"/> is <see
-    /// cref="HastlayerFlavor.Client"/>.
-    /// </para>
     /// </remarks>
     public bool PromptBeforeBuild { get; set; }
 }