Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quartus: init at 19.1.0.670 #75561

Merged
merged 3 commits into from Jan 17, 2020
Merged

quartus: init at 19.1.0.670 #75561

merged 3 commits into from Jan 17, 2020

Conversation

@kwohlfahrt
Copy link
Contributor

@kwohlfahrt kwohlfahrt commented Dec 12, 2019

Motivation for this change

Add Quartus Prime software for programming Intel FPGAs. There are unfortunately no open-source alternatives.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Notify maintainers

This is a new package - should i add myself as a maintainer, even though I don't have commit access to nixpkgs?

@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch from 0d4ee56 to 1dadee5 Dec 13, 2019
@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Dec 13, 2019

This also adds a utility that silently eats chmod syscalls that set SUID/SGID bits. The installer insists on setting this on directories, which nix then causes to fail.

Nix could in theory allow this, I think the action is safe on directories but I'm not sure it is possible to intercept the syscall, check arguments and then proceed in a way that does not produce a race condition (where a directory is replaced with a file after the check but before the operation takes place).

@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Dec 13, 2019

This is a new package - should i add myself as a maintainer, even though I don't have commit access to nixpkgs?

Yes, it's very common to maintain while not having commit access. Please add yourself as a maintainer to the maintainers file and put that addition in a separate commit.

Other then that and my 1 comment, LGTM. Just out of curiosity, is this your 1st contribution to Nixpkgs? It seems you have used advanced mechanisms to package this, similar to WolframAlpha's package.. Well done!

@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch 2 times, most recently from ef5ac82 to 0b115a3 Dec 14, 2019
@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Dec 14, 2019

Thanks for the review! This isn't my first contribution to nixpkgs, but it's definitely my trickiest so far, open-source is generally much easier.

I've added myself to the maintainers.

@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch from 0b115a3 to 1e031cd Dec 14, 2019
@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Dec 14, 2019

Is it possible to have more than one executable in a FHS user environment? The quartus program really comes with several binaries that should be runnable from the command line.

@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Dec 14, 2019

Good question. I'm a fan myself of the command line tools that come with Quartus and I used them back when I was on Arch and I developed stuff for FPGA.

It's a good question to ask on the forum. The main issue is that there's the runScript argument which should be mandatory for this.

1 workaround I have in mind while thinking about it, is this:

Create a wrapper script that will replace the current runScript. Make this wrapper check it's $0 and accordingly launch the right executable in the FHS env. Afterwards, create a wrapper package that will create symlinks to that FHS' wrapper script and use different names in $out/bin/. This way, the symlinks will emulate a different $0 and thus operate a different executables in the FHS environment.

I hope you can imagine what I'm describing. From AFAIK this is a common practice for example with rustup - there's only 1 executable and it checks it's $0 in order to operate as rustc or cargo or whatever. The same is done also with bash and sh - if it's $0 is sh it'll operate in a sh compatible mode.

@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Dec 15, 2019

OK, I'll try and put something together. I'd like it to work with .desktop files so I can use it nicely from a menu.

Secondly, if you have much experience with Quartus - the "Platform Designer" tool does not work at least when launched from the main UI. Unfortunately, there are no GUI or console error logs, I just get the Platform Designer splash screen and then nothing. Do you know either which binary this is, so I can try running it directly, or if there are any more detailed logs kept anywhere? Found it, missing some X libraries.

I wouldn't consider this a blocker but it would be nice to fix.

@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch from 1e031cd to 516095b Dec 15, 2019
@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Dec 15, 2019

I'm glad you worked this out @kwohlfahrt! as for the other command line interface executables, could you perhaps just verify, as a start, that the command line tools listed here are runnable from inside the FHS env? You can test it by temporarily replacing the runScript with bash and then see if you can launch them from that shell..

If they all work fine, I think we can work out together a way to make all of these executables available in the user's $PATH with a wrapper package and a wrapper script, as I tried to explain in my previous comment.

@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Dec 16, 2019

quartus/bin/jtagconfig works, quartus/bin/quartus obviously does, quartus/sopc_builder/bin/qsys-edit is OK. Any others I should check?

modelsim_ase/bin/vsim exits with Read failure in vlm process. I assume I also have to set-up some licensing things to run this directly, since it works when launched from the main UI.

@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Dec 16, 2019

Good job @kwohlfahrt. There are many more executables it would be nice to check (e.g quartus_asm, quartus_cdb...) - please see the list in this PDF's TOC.

As for the modelsim error, see this suggested solution:

https://bbs.archlinux.org/viewtopic.php?pid=1708875#p1708875

@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch from 516095b to d9257a4 Jan 9, 2020
@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Jan 9, 2020

Is this ready for review?

@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Jan 9, 2020

@Mic92 I have been using this package via the main GUI for a few months, it seems to be working OK. However, I have not tested many of the individual executables suggested above. Hoping to do so soon now I'm back from the holiday, so I don't mind deferring this until I have, but I also wouldn't object to merging this now and fixing any issues found later.

@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Jan 9, 2020

@kwohlfahrt From my experience with the PRs culture in Nixpkgs, I think it'll be worth marking this as WIP untill it'll be fully ready.

@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Jan 9, 2020

The following work: ../sopc_builder/bin/qsys-generate, ./quartus_map, quartus_fit, quartus_asm, quartus_sta, quartus_eda. Some others give me errors that they are not available with my license, or that I am missing command-line arguments but they do run.

vsim still does not work - however Ubuntu 18.04 is supported which has freetype-2.8.1, so it is not exactly the same issue. I get the same error on Ubuntu, the missing file seems to be /usr/local/bin/InitializeINIFile on Ubuntu and /run/current-system/sw/bin/InitializeINIFile on Nix. I read the documentation wrong, ModelSim is only supported on RHEL6-7 and SUSE SLE 11-12 - I don't have any subscription to these so I cannot test it in the supported case. Some sites suggest I am missing libpng-12

@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Jan 10, 2020

I don't have an Intel account to download/test this but would merge this pull request if somebody else was able to launch the GUI i.e. @doronbehar

pkgs/os-specific/linux/unstick/default.nix Outdated Show resolved Hide resolved
This nix expression requires that ${name} is already part of the store.
Download it from ${meta.homepage} and add it to the nix store with:
nix-prefetch-url <URL>

This comment has been minimized.

@doronbehar

doronbehar Jan 10, 2020
Contributor

BTW I'd suggest adding here that it can also be done via nix-store --add <file>

This comment has been minimized.

@kwohlfahrt

kwohlfahrt Jan 12, 2020
Author Contributor

Sure, is there any reason for preferring one over the other? Going to hold off on making any changes until we get the hash issue sorted out though.

This comment has been minimized.

@kwohlfahrt

kwohlfahrt Jan 13, 2020
Author Contributor

I've removed the custom message. The default message now suggests nix-store --add-fixed sha256 or nix-prefetch-url --type sha256 file://path/to.

@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Jan 10, 2020

I thought up until now I'll need a paid license in order to test this but it seems it's supposed to be possible to download the what's called "standard" edition of the QuartusLiteSetup-${version}-linux.run files and that's what the derivation is targeted at - the only requirement is to sign up for a free account. Here are my findings:

I downloaded these .run files and added them with nix-store --add (see my review about this) and I think because my sha256sum were different I still got the message that I need to put the ModelSim .run file in my /nix/store/. The shas seem to be different for the QuartusLiteSetup .run file as well:

$ ls -l /nix/store/4yihv30am9640ixijz0ip7x4msnlwshx-QuartusSetup-19.1.0.670-linux.run /nix/store/gy4maz398wv3d8hrf3j95zrcddrrmlig-QuartusLiteSetup-19.1.0.670-linux.run.drv
-r--r--r-- 1 root root 2701700693 Jan  1  1970 /nix/store/4yihv30am9640ixijz0ip7x4msnlwshx-QuartusSetup-19.1.0.670-linux.run
-r--r--r-- 1 root root       1486 Jan  1  1970 /nix/store/gy4maz398wv3d8hrf3j95zrcddrrmlig-QuartusLiteSetup-19.1.0.670-linux.run.drv
$ ls -l /nix/store/y75qnagk59qfhpc5sla83iv7g4jnxdb9-ModelSimSetup-19.1.0.670-linux.run /nix/store/hj9gvpmzjszrcq5yndq6jjs8n7jrcdkl-ModelSimSetup-19.1.0.670-linux.run.drv /nix/store/0bi68pph1sn7c1dgfg4d4dyvfglds1lv-ModelSimSetup-19.1.0.670-linux.run.lock
-rw------- 1 root root          0 Jan 10 17:47 /nix/store/0bi68pph1sn7c1dgfg4d4dyvfglds1lv-ModelSimSetup-19.1.0.670-linux.run.lock
-r--r--r-- 1 root root       1477 Jan  1  1970 /nix/store/hj9gvpmzjszrcq5yndq6jjs8n7jrcdkl-ModelSimSetup-19.1.0.670-linux.run.drv
-r--r--r-- 1 root root 1047232539 Jan  1  1970 /nix/store/y75qnagk59qfhpc5sla83iv7g4jnxdb9-ModelSimSetup-19.1.0.670-linux.run

@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Jan 12, 2020

@kwohlfahrt can you confirm the different checksums?

@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Jan 12, 2020

Sorry, missed the previous comment. I get the following checksums for the ModelSimSetup file. The md5sum matches that published on the download website:

> md5sum ModelSimSetup-19.1.0.670-linux.run 
142526799b9f80e1fe1d11743724a668  ModelSimSetup-19.1.0.670-linux.run
> sha256sum ModelSimSetup-19.1.0.670-linux.run
d9f841c12a0d7360222068610dca23bff6ac90e9f48aaa2c429b321952763b48  ModelSimSetup-19.1.0.670-linux.run

nix-prefetch-url produces the checksum added in the PR. nix-store --add however comes up with a different store path, I'm not sure what's up with that.

> nix-prefetch-url file://(pwd)/ModelSimSetup-19.1.0.670-linux.run
warning: dumping very large path (> 256 MiB); this may run out of memory
[998.7 MiB DL]
path is '/nix/store/0bi68pph1sn7c1dgfg4d4dyvfglds1lv-ModelSimSetup-19.1.0.670-linux.run'
0j1vfr91jclv88nam2plx68arxmz4g50sqb840i60wqd5b0l3y6r
> nix-store --add ./ModelSimSetup-19.1.0.670-linux.run 
warning: dumping very large path (> 256 MiB); this may run out of memory
/nix/store/b6cc9qpmiq8h441984p555601s3w4p2v-ModelSimSetup-19.1.0.670-linux.run

I notice the hash part of the path from nix-prefetch-url matches the .lock file from @doronbehar above.

Additionally, vsim works if I apply the following overlay: self: super: { libpng = super.libpng12; }. Is there a way to apply this to the multiPkgs of buildFHSUserEnv?

@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch from d9257a4 to 4e1d0d2 Jan 12, 2020
@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Jan 13, 2020

@doronbehar can you re-check?

@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Jan 13, 2020

I got rid of the custom message. Note that nix-store --add produces a different store path compared to nix-store --add-fixed sha256 and nix-prefetch-url. Could you try again using one of the latter two methods @doronbehar?

Regarding vsim/libpng12: Adding libpng12 to multipkgs is insufficient, I still get the error below.

Initialization problem, exiting.

    while executing
"InitializeINIFile quietly"
    invoked from within
"ncFyP12 -+"
    (file "/mtitcl/vsim/vsim" line 1)
** Fatal: Read failure in vlm process (0,0)

I assume this occurs if libpng is linked anywhere instead of libpng12 - I'm not sure how these versions are resolved at runtime. So far, the only fix I've found is the overlay approach, so all libraries that depend on libpng are rebuilt with libpng12. For example, freetype depends on libpng, so that must be overridden. But then fontconfig depends on freetype, so I need to override fontconfig's freetype dependency with the one we just built. I imagine there are a fair few libpng dependencies buried somewhere in the X11 requirements as well, so I don't think it is practical to do this one package at a time.

This is also my reasoning for splitting out the ModelSim package - these overridden won't be available from the Nix cache, and I would prefer to use the up-to-date libraries where they don't cause issues.

PS - I didn't summon the extra reviewers, that happened automatically.

@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch from 124b5dd to c6622eb Jan 13, 2020
@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Jan 13, 2020

I think I got vsim working! The overrides are a bit ugly the way they are now though, can this be applied to all of pkgs cleanly somehow?

@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Jan 13, 2020

@kwohlfahrt Well done for finding out about nix-store --add-fixed and changing the message. I've readded the files with:

~/downloads $ nix-store --add-fixed sha256 ./QuartusSetup-19.1.0.670-linux.run ./ModelSimSetup-19.1.0.670-linux.run

When building the derivation, I got no error as for the ModelSim setup .run file but only for Quartus. It's seems as if it's because the file names are different - mine is ./QuartusSetup-19.1.0.670-linux.run and the expected filename is QuartusLiteSetup-19.1.0.670-linux.run. After renaming the file and running nix-store --add-fixed again, I tried building but the message came up again. Could you recheck please with Quartus' download on your end what is the filename that's used when you download the file?

Here are the files I have in my /nix/store right now:

$ ls -l /nix/store/*Quartus*
-r--r--r-- 1 root root 2701700693 Jan  1  1970 /nix/store/4yihv30am9640ixijz0ip7x4msnlwshx-QuartusSetup-19.1.0.670-linux.run
-rw------- 1 root root          0 Jan 13 19:06 /nix/store/9m7nc46lcrjhr74lag3h2j8wlxbibk6g-QuartusLiteSetup-19.1.0.670-linux.run.lock
-r--r--r-- 1 root root       1486 Jan  1  1970 /nix/store/gy4maz398wv3d8hrf3j95zrcddrrmlig-QuartusLiteSetup-19.1.0.670-linux.run.drv
-r--r--r-- 1 root root 2701700693 Jan  1  1970 /nix/store/pnn5bvcz7q6cp5zqf77b66csw0h4qn48-QuartusSetup-19.1.0.670-linux.run
-r--r--r-- 1 root root 2701700693 Jan  1  1970 /nix/store/svq7aag9lc5mmbp0rr00qfqwk38hmf3k-QuartusLiteSetup-19.1.0.670-linux.run
-r--r--r-- 1 root root       1486 Jan  1  1970 /nix/store/waznhb972qp8nfj2h70zqg61b7z9ps8a-QuartusLiteSetup-19.1.0.670-linux.run.drv
$ ls -l /nix/store/*ModelSim*
-r--r--r-- 1 root root 1047232539 Jan  1  1970 /nix/store/0bi68pph1sn7c1dgfg4d4dyvfglds1lv-ModelSimSetup-19.1.0.670-linux.run
-r--r--r-- 1 root root       1477 Jan  1  1970 /nix/store/ga6v7kmprbscabh98zwdkl6mvilh8qjw-ModelSimSetup-19.1.0.670-linux.run.drv
-r--r--r-- 1 root root       1477 Jan  1  1970 /nix/store/hj9gvpmzjszrcq5yndq6jjs8n7jrcdkl-ModelSimSetup-19.1.0.670-linux.run.drv
-r--r--r-- 1 root root 1047232539 Jan  1  1970 /nix/store/y75qnagk59qfhpc5sla83iv7g4jnxdb9-ModelSimSetup-19.1.0.670-linux.run
@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Jan 13, 2020

I think I got vsim working! The overrides are a bit ugly the way they are now though, can this be applied to all of pkgs cleanly somehow?

I don't think it's that bad, but other reviewers may have a stronger say on this then I :)

# This seems ugly - can we override `libpng = libpng12` for all `pkgs`?
freetype = pkgs.freetype.override { libpng = libpng12; };
fontconfig = pkgs.fontconfig.override { inherit freetype; };
libXft = pkgs.xorg.libXft.override { inherit freetype fontconfig; };

This comment has been minimized.

@Mic92

Mic92 Jan 13, 2020
Contributor

I think it is possible to apply overlays with overrideScope. I do not have concrete code to do that though.

@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Jan 13, 2020

I think I got vsim working! The overrides are a bit ugly the way they are now though, can this be applied to all of pkgs cleanly somehow?

I don't think it's that bad, but other reviewers may have a stronger say on this then I :)

It is acceptable overlays might add more complexity than this.

@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Jan 14, 2020

It's seems as if it's because the file names are different - mine is ./QuartusSetup-19.1.0.670-linux.run and the expected filename is QuartusLiteSetup-19.1.0.670-linux.run.

There is a drop-down menu at the top of the download page, where you can select between "Pro", "Standard" and "Lite" downloads, above the release option. This should be set to "Lite", the "Standard" option results in the "QuartusLiteSetup-${version}.run". This URL should set it appropriately: https://fpgasoftware.intel.com/19.1/?edition=lite&platform=linux

@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Jan 14, 2020

@kwohlfahrt Could you add this as a comment or in the error message?

@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch from c6622eb to 4d06953 Jan 15, 2020
kwohlfahrt added 3 commits Dec 14, 2019
This requires a FHSUserEnv because the licensing executable checksums
itself (I think).

The override of libpng is a bit ugly, can this be cleaned up?
@kwohlfahrt kwohlfahrt force-pushed the kwohlfahrt:quartus branch from 4d06953 to 38e6c84 Jan 15, 2020
@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Jan 15, 2020

Done - I modified the URL to include the edition and version. I also changed the name of the package to quartus-prime-lite, to better reflect that this is not the standard/pro edition.

@ttuegel ttuegel removed their request for review Jan 16, 2020
@Mic92 Mic92 merged commit d3a0bef into NixOS:master Jan 17, 2020
15 checks passed
15 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
unstick on aarch64-linux Success
Details
unstick on x86_64-linux Success
Details
@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Jan 17, 2020

@expipiplus1 I guess Intel has cleaned it up :)

@jonringer jonringer mentioned this pull request Jan 24, 2020
2 of 10 tasks complete
@expipiplus1
Copy link
Contributor

@expipiplus1 expipiplus1 commented Feb 11, 2020

\o/ Thanks, can't wait to try this out.

Anyone reading might also be interested in @bjornfor's quartus packaging: https://github.com/bjornfor/nixos-config/blob/master/pkgs/altera-quartus/default.nix

@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Feb 11, 2020

@expipiplus1 you can try right away by running nix-env -f. -iA quartus-prime-lite inside nixpkgs/ repo.

@bjornfor
Copy link
Contributor

@bjornfor bjornfor commented Feb 11, 2020

Quartus is in nixpkgs master? Nice!

@kwohlfahrt kwohlfahrt deleted the kwohlfahrt:quartus branch Mar 13, 2020
@meck
Copy link

@meck meck commented Jul 22, 2020

@kwohlfahrt would it be possible to add the nios2 eclipse stuff as an optional component? its a manual install. I tried my hand at it but I didn't get far...

@kwohlfahrt
Copy link
Contributor Author

@kwohlfahrt kwohlfahrt commented Jul 27, 2020

@meck I'm not currently planning to work on the eclipse integration. I haven't used eclispe before, and the instructions linked call for a very old version of eclipse that might be a lot of effort to get working.

@meck
Copy link

@meck meck commented Jul 28, 2020

Thats resonable, thanks anyway!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

7 participants
You can’t perform that action at this time.