New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

descrypt-ztex #2307

Merged
merged 1 commit into from Oct 18, 2016

Conversation

Projects
None yet
6 participants
@Apingis
Collaborator

Apingis commented Oct 12, 2016

Let's discuss possible merge into bleeding-jumbo.
There might be points of concern.

  • Size of the add-on. It includes "bitstream" (FPGA configuration file) ~4MB. Every ztex format will have its own bitstream.
  • Requires libusb-1.0 to build.
  • Format's init() function performs scan for USB devices. If ztex board is found, an expensive initialization follows (~2s + 1s per each board). After more ztex formats added, switching from one ztex format to another would take the same time. Some command-line options such as --list=format-details call init() for every format.
  • For now, format works as it is; however there are numerous issues that I'm going to raise at john-dev.
@claudioandre-br

This comment has been minimized.

Show comment
Hide comment
@claudioandre-br

claudioandre-br Oct 13, 2016

Collaborator

Have you seen the build failed inside CI?

Collaborator

claudioandre-br commented Oct 13, 2016

Have you seen the build failed inside CI?

@jfoug

This comment has been minimized.

Show comment
Hide comment
@jfoug

jfoug Oct 13, 2016

Collaborator
Configure finished.  Now 'make clean && make -s' to compile.

make[2]: *** No rule to make target 'all'.  Stop.

Makefile:1391: recipe for target 'ztex' failed

make[1]: *** [ztex] Error 2

make[1]: *** Waiting for unfinished jobs....

x86_64-w64-mingw32-ar: creating aes.a

Makefile:186: recipe for target 'default' failed

make: *** [default] Error 2

Something in the Makefile.in (likely) is wrong. Also, this like will not build until new libs are installed (correct?)

Collaborator

jfoug commented Oct 13, 2016

Configure finished.  Now 'make clean && make -s' to compile.

make[2]: *** No rule to make target 'all'.  Stop.

Makefile:1391: recipe for target 'ztex' failed

make[1]: *** [ztex] Error 2

make[1]: *** Waiting for unfinished jobs....

x86_64-w64-mingw32-ar: creating aes.a

Makefile:186: recipe for target 'default' failed

make: *** [default] Error 2

Something in the Makefile.in (likely) is wrong. Also, this like will not build until new libs are installed (correct?)

@magnumripper

This comment has been minimized.

Show comment
Hide comment
@magnumripper

magnumripper Oct 13, 2016

Owner

Perhaps we should have a permanent "ztex" branch for these things? Someone would need to maintain it though (not me).

Owner

magnumripper commented Oct 13, 2016

Perhaps we should have a permanent "ztex" branch for these things? Someone would need to maintain it though (not me).

@solardiz

This comment has been minimized.

Show comment
Hide comment
@solardiz

solardiz Oct 13, 2016

Collaborator

On Wed, Oct 12, 2016 at 11:48:49PM -0700, magnum wrote:

Perhaps we should have a permanent "ztex" branch for these things? Someone would need to maintain it though (not me).

What's the problem with having it in the main/only jumbo branch? Just need a configure option to make the extra dependencies (just libusb?) optional. No? For example, cgminer had "--enable-ztex" (IIRC, the option was called that) right in the main branch while it made sense (until Bitcoin went ASIC only), along with OpenCL and other stuff.

Collaborator

solardiz commented Oct 13, 2016

On Wed, Oct 12, 2016 at 11:48:49PM -0700, magnum wrote:

Perhaps we should have a permanent "ztex" branch for these things? Someone would need to maintain it though (not me).

What's the problem with having it in the main/only jumbo branch? Just need a configure option to make the extra dependencies (just libusb?) optional. No? For example, cgminer had "--enable-ztex" (IIRC, the option was called that) right in the main branch while it made sense (until Bitcoin went ASIC only), along with OpenCL and other stuff.

@jfoug

This comment has been minimized.

Show comment
Hide comment
@jfoug

jfoug Oct 13, 2016

Collaborator

It should not be hard to get it in there working. Easy to add into configure, and #ifdef the thing like we do other non-pluggable formats. @magnumripper just does not want to be the person to support it (nor do I), since we have no hardware, and no real ability to maintain. If there is no maintainer, then it will end up like cuda, which really is dead and should be deprecated without a proper maintainer.

Collaborator

jfoug commented Oct 13, 2016

It should not be hard to get it in there working. Easy to add into configure, and #ifdef the thing like we do other non-pluggable formats. @magnumripper just does not want to be the person to support it (nor do I), since we have no hardware, and no real ability to maintain. If there is no maintainer, then it will end up like cuda, which really is dead and should be deprecated without a proper maintainer.

@Apingis

This comment has been minimized.

Show comment
Hide comment
@Apingis

Apingis Oct 13, 2016

Collaborator

It didn't allow to commit src/ztex/Makefile because .gitignore prohibits Makefile's, that results in fail inside CI. I think I have to let configure create src/ztex/Makefile. Also it would require libusb-1.0 (going to locate exact name for apt-get, that's probably libusb-1.0-0-dev). The format itself is "pluggable" (ztex_descrypt_fmt_plug.c), existing configure adds it properly but it needs extra objects to be built in src/ztex and extra library (-lusb-1.0).

Collaborator

Apingis commented Oct 13, 2016

It didn't allow to commit src/ztex/Makefile because .gitignore prohibits Makefile's, that results in fail inside CI. I think I have to let configure create src/ztex/Makefile. Also it would require libusb-1.0 (going to locate exact name for apt-get, that's probably libusb-1.0-0-dev). The format itself is "pluggable" (ztex_descrypt_fmt_plug.c), existing configure adds it properly but it needs extra objects to be built in src/ztex and extra library (-lusb-1.0).

@magnumripper

This comment has been minimized.

Show comment
Hide comment
@magnumripper

magnumripper Oct 13, 2016

Owner

Perhaps we should have a permanent "ztex" branch for these things? Someone would need to maintain it though (not me).

What's the problem with having it in the main/only jumbo branch?

I was just reflecting over the statement about 4 MB bitstream per ztex format, in case that is an actual problem. Perhaps it isn't. Including this in the main branch is better in many ways and we wouldn't need another maintainer then.

Owner

magnumripper commented Oct 13, 2016

Perhaps we should have a permanent "ztex" branch for these things? Someone would need to maintain it though (not me).

What's the problem with having it in the main/only jumbo branch?

I was just reflecting over the statement about 4 MB bitstream per ztex format, in case that is an actual problem. Perhaps it isn't. Including this in the main branch is better in many ways and we wouldn't need another maintainer then.

@magnumripper

This comment has been minimized.

Show comment
Hide comment
@magnumripper

magnumripper Oct 13, 2016

Owner

It didn't allow to commit src/ztex/Makefile because .gitignore prohibits Makefile's, that results in fail inside CI. I think I have to let configure create src/ztex/Makefile

Yes, you should call it src/ztex/Makefile.in and have autoconf produce the Makefile.

Owner

magnumripper commented Oct 13, 2016

It didn't allow to commit src/ztex/Makefile because .gitignore prohibits Makefile's, that results in fail inside CI. I think I have to let configure create src/ztex/Makefile

Yes, you should call it src/ztex/Makefile.in and have autoconf produce the Makefile.

descrypt-ztex format; includes library for operating ZTEX hardware and
bitstream (4MB). Requires libusb-1.0. Autoconf support.
@Apingis

This comment has been minimized.

Show comment
Hide comment
@Apingis

Apingis Oct 18, 2016

Collaborator

Updated pull request (squashed commit)

  • Added autoconf support. Created m4/ax_ztex.m4. Introduced --enable-ztex command-line switch to configure. Without the switch, it completely excludes ztex format from the build.
  • Renamed to ztex_descrypt.c, removed from "plugformat" objects.
  • Some corrections after tests on Ubuntu 64-bit, with other version of gcc (4.6)

There're still issues that I've raised at john-dev, that's not format specific ones.

One thing I'm not happy with is quality estimation on John's home page. It says "its overall quality is lower" about bleeding-jumbo. I would disagree with such statement about my code, unless there would be a substantiation.

Collaborator

Apingis commented Oct 18, 2016

Updated pull request (squashed commit)

  • Added autoconf support. Created m4/ax_ztex.m4. Introduced --enable-ztex command-line switch to configure. Without the switch, it completely excludes ztex format from the build.
  • Renamed to ztex_descrypt.c, removed from "plugformat" objects.
  • Some corrections after tests on Ubuntu 64-bit, with other version of gcc (4.6)

There're still issues that I've raised at john-dev, that's not format specific ones.

One thing I'm not happy with is quality estimation on John's home page. It says "its overall quality is lower" about bleeding-jumbo. I would disagree with such statement about my code, unless there would be a substantiation.

@magnumripper

This comment has been minimized.

Show comment
Hide comment
@magnumripper

magnumripper Oct 18, 2016

Owner

OK this looks good now but I can't test it at all. So @solardiz should I just merge it?

Owner

magnumripper commented Oct 18, 2016

OK this looks good now but I can't test it at all. So @solardiz should I just merge it?

@solardiz

This comment has been minimized.

Show comment
Hide comment
@solardiz

solardiz Oct 18, 2016

Collaborator

On Tue, Oct 18, 2016 at 07:41:58AM -0700, magnum wrote:

OK this looks good now but I can't test it at all. So @solardiz should I just merge it?

Yes, please merge it. I'm in contact with Denis regarding further testing, and both of us got the hardware. If you like, you can test it remotely (we have a box with a ztex board connected to it currently online, albeit temporarily).

Collaborator

solardiz commented Oct 18, 2016

On Tue, Oct 18, 2016 at 07:41:58AM -0700, magnum wrote:

OK this looks good now but I can't test it at all. So @solardiz should I just merge it?

Yes, please merge it. I'm in contact with Denis regarding further testing, and both of us got the hardware. If you like, you can test it remotely (we have a box with a ztex board connected to it currently online, albeit temporarily).

@magnumripper magnumripper merged commit 0633422 into magnumripper:bleeding-jumbo Oct 18, 2016

2 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@sergiy213

This comment has been minimized.

Show comment
Hide comment
@sergiy213

sergiy213 Sep 1, 2018

Xilinx Zynq 7020 has same software as ZTEX 1.15y boards?it is compatible with John the ripper?

sergiy213 commented Sep 1, 2018

Xilinx Zynq 7020 has same software as ZTEX 1.15y boards?it is compatible with John the ripper?

@solardiz

This comment has been minimized.

Show comment
Hide comment
@solardiz

solardiz Sep 1, 2018

Collaborator

@sergiy213 This is not a user support forum. Please use the john-users mailing list for questions such as yours.

That said, we supported bcrypt (only) on ZedBoard and Parallella board (only these two) with Xilinx Zynq 7020 in a temporary unofficial branch made for research purposes in 2013-2014. This is not currently supported, and the performance was much worse than what we currently have on the supported ZTEX 1.15y boards.

Collaborator

solardiz commented Sep 1, 2018

@sergiy213 This is not a user support forum. Please use the john-users mailing list for questions such as yours.

That said, we supported bcrypt (only) on ZedBoard and Parallella board (only these two) with Xilinx Zynq 7020 in a temporary unofficial branch made for research purposes in 2013-2014. This is not currently supported, and the performance was much worse than what we currently have on the supported ZTEX 1.15y boards.

@sergiy213

This comment has been minimized.

Show comment
Hide comment
@sergiy213

sergiy213 Sep 1, 2018

thank you very much for clear advice .as i see is very hardware specific and heavy managed task.

sergiy213 commented Sep 1, 2018

thank you very much for clear advice .as i see is very hardware specific and heavy managed task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment