Integrate with I2P-Java via libsam3 #5070
Comments
|
If we're convinced that i2p-java is going to continue then why not. We have a responsibility to maximise the protection that users have available to them. |
|
Here are some first thoughts at how this would work, as a starting point for comment by the Monero and I2P communities: I2P provides a C library called libsam3, which makes it easy for the Monero daemon to communicate with a SAMBridge instance via tcp sockets. SAM stands for Simple Anonymous Messaging. SAMBridge is Java code that will translate between the high level SAM protocol and the lower level I2CP "I2P Client Protocol". Therefore the Monero daemon will open tcp sockets to SAMBridge with libsam3 and communicate using the SAM protocol, and SAMBridge will open a single tcp socket to the I2P router and communicate using the I2CP protocol. Advanced Monero daemon users should be given the option to connect via SAM to their own full I2P router installation. This will be a useful feature during development, because it means development can begin prior to the existence of an embedded I2P-Java router. Embedded I2P-Java/SAMBridge instances The Monero daemon will need to launch a JVM that will run the I2P Java router and a SAMBridge instance. A tiny Java project will exist that will bundle the following I2P Java libraries: i2p.jar, router.jar, streaming.jar, mstreaming.jar and jbigi.jar. This Java project will then have a main Java class that will do something very close to the following code, which was provided at https://geti2p.net/en/docs/applications/embedding In addition, the code will instantiate and run an instance of the Java SAMBridge class. Both the I2P-Java router and the SAMBridge instance will be launched by the same Java code running inside the same JVM process. The Java project will be built into per-OS executables using the jlink tool. This tool will, for each OS, produce a directory tree that will contain both the compiled Java code and specific JVM executables for each OS. From one build environment OS, executables can be generated for all available OSs by the jlink tool. The jlink tool will use a module-info.java file to build these executables such that only the necessary parts of the JVM are included. This will ensure the size of the executables is kept to a minimum. These per-OS joint Embedded I2P-Java/SAMBridge executables will be bundled with Monero builds, and used to send traffic via I2P. End users will have no idea they are running Java code or using a JVM. The Monero daemon will handle the startup and shutdown of this Embedded I2P-Java/SAMBridge process. |
knaccc
changed the title
|
Hi, zab here. You pretty much have everything figured out as far as integrating goes. The only issues I see are that using modules and jlink ties us to Java 9+, and while Java 11 is already out and is LTS, the default download for Windows on www.java.com is still 8, which means Oracle doesn't feel confident enough yet to bump it to 11. Currently the I2P code is Java 7 compliant. The nasty bit about modules and jlink is that they are defined in .java files, which means we either force 9+ code compatibility or employ some very creative exclusion filters to make sure our code still builds on 7. This is based on very preliminary research because I learned about jlink same time as you did :) So it's more of a strategic decision whether and how to modularise; Monero certainly doesn't need a full I2P distribution (that includes email, torrent client and a kitchen sink). To alleviate @fluffypony 's concern Java I2P is healthier than ever and definitely going to continue. We even have a paid program to support full-time developers and are currently employing 3 core developers full-time and 2 part-time, in addition to some PR and design folk. |
@knaccc s/developers/developer/. And they're still technically in beta; have been for ~15 years.
@fluffypony because your whole platform for years, even before Kovri, was to not use Java I2P. In fact, you were the first and only person to shoot down a socket based API; making everyone wait for years as a result. In fact, you were the deciding factor at that time but flaked out; leaving me to clean up your mess. And here you are now, as you conveniently appear to sway opinion on yet another decision that is beyond your technical grasping. But, aside from that, the real reason is because there is already a better solution than hooking into java I2P directly from monero. With that said, Monero is a money project, not an anonymity project, so I don't expect most here to understand what's happening aside from the few devs who have, fortunately, reached-out and made contact. Besides, you'll certainly want "something" "now", so; have at it.
@fluffypony "We"? Who do you think you are? You're not an anonymity developer; you're barely technically literate. If you were truly responsible, then this wouldn't have happened. If you were truly responsible, then you would've already engaged our development in Besides; this entire issue has already been deprecated by Sekreta. Single-system solutions are a thing of the past. While Monero enjoys the past, other privacy coins will lead the future. I've already pointed out that danger in my post. My only advice now is to not get too attached to this hack of a request (#5070). Also, throw salt on blockchain developers who moonlight as network anonymity developers since the technology involved is not the same.
@zlatinb So, there's reality, which is exposed via monotone's/git's log, and there is what you say, which is more of an illusive reality. Java I2P logs look no different than how they've looked over the years with the exception of Meeh doing some OS X work in 2018 (good to see he's back though, he MIA'd for a long time). Aside from him, there are almost no other authors besides zzz for 2018/2019 (and those other authors have very few commits); and zzz's always been the lead/full-time dev since jrandom left. |
Yes, because it would require installing Java, which adds a Java dependency to Monero. See, for instance, this post of mine from July, 2015. @knaccc is suggesting that the modularisation in Java 9 makes it self-contained, and thus doesn't install Java. |
It's important to remember that currently Monero runs without Kovri/i2p/Sekreta/Tor - it simply runs over clearnet - and this proposal would at least improve the status quo.
@anonimal No need to resort to ad hominem attacks here. The beauty of a community effort is that we all have strengths and weaknesses. Let your work speak for itself, and rational people will adopt it. Attacking community members will only distract people from your hard work. |
|
Why embed anything at all at any point instead of keep communicating over an ipc such as sockets? Maybe a better investment would be a monero binary which is ipc-agnostic to kovri, i2p-java, tor or sekreta. Then everyone can run whay they choose without dead dependencies. Embedding seems like a too tight coupling for unclear benefits compared to ipc. |
|
@zlatinb Thanks for taking the time to comment!
I've figured out how to automatically convert your existing JARs into modules. No code modifications will be necessary to I2P's repository.
I know a lot of people that rely heavily on Java 11 in production. so I have a good feeling about it.
As above, you won't have to worry about this. Your Java 7 JARs are easily turned into modules and run perfectly on Java 11.
That's wonderful to hear! |
|
I've just released an embedded I2P Java router with SAM, that would be suitable for embedding into Monero. The GitHub repo is here: https://github.com/knaccc/embedded-i2p-java-router-with-sam It works by taking existing I2P JARs and turning them into modules. These are then used with jlink to produce an executable with a minimal JVM. @zlatinb I'd be very interested in your thoughts, in particular on two things:
Thanks! |
Just to be clear, when we say embedding we actually mean bundling, but we've gotten so used to saying "embedding". Comms would absolutely go over RPC via TCP, or over IPC. |
|
@knaccc I'm impressed, well done! Your approach to converting jars into modules will come in handy because honestly we can't wait to get rid of our dependency on Java as well. Now to your questions:
Usually the SAM bridge is launched via a "clients.config" file, but it might work the way you've done it too. What I recommend is you wait until the Router state machine reaches RUNNING state before starting the SAM bridge. Poke around the code to see exactly how to get a reference to it.
Trial and Error :) more specifically write a small Hello World application that uses the SAM library, enable full logging in the router and hammer it until it works. I think you don't really need to bundle anything to get it to bootstrap and join the I2P network. If you're comfortable with Python here is a nice guide - https://geti2p.net/en/blog/post/2018/10/23/application-development-basics |
|
@zlatinb Thanks, I'll make that update to the code. Btw, I've tested the executable built with this project by setting up a Node.js client and server to talk to each other via it (using https://github.com/redhog/node-i2p). Everything seems to work great! |
|
Btw the total executable ended up as 36MB (37281943 bytes). It was reduced to 19MB (19764972 bytes) when compressed with |
I think that has been the idea from the start, primarily because the latency of the i2p network and the size of the monero blocks. Though who knows - if our fluffyblocks is smart enough, block relay has effectively the same cost as tx relay. Though knaccc may have a different idea. In general, does this bundling and modularization and lack of dependencies change the amount of resources used by java i2p? I just checked on my node server and currently java is using 50% cpu resources ... and now 64%... back to 50%... I'm just worried that bundling this with monero, and assuming its an always on situation in order to provide the most privacy, is going to create another case of people not using the monero core software because its resource intensive. "What the heck is using 50% of my CPU? My superbloated javascript webpages don't load nearly as fast as they used to! Oh, its the monero software. DELETED." Because of this resource use (and perhaps independent of it), users will probably start and stop The Monero Software (which now has i2p), and from what I've gathered its best to keep the router running so your tunnels get all interconnected and integrated. Of course the flip side of this is that bundling java-i2p is better than nothing, and ultimately if the software is designed for some extreme user apathy or we wait for perfect optimizations, then we'll have a .... i dunno. Best getting in the way of good situation. |
Not really, you are still running a JVM with it's bells and whistles (JIT, GC, etc.). The main source of CPU usage are the crypto operations, which reminds me - @knaccc it is important to add jbigi.jar and the NativeBigInteger libraries to the bundle, otherwise performance will suffer.. a lot! The number of crypto operations is proportional to the amount of I2P traffic that is routed and that can be limited through configuration, so it's a matter of choosing sane defaults. |
|
Since no one in this thread has developed I2P technology longer than I, allow me to drop some important information: Note: for those who don't know; NTCP is the encrypted TCP transport layer for I2P. This discussion was regarding the new 2nd version. This is a mission-critical protocol, a base-layer protocol that ferries your layered messages. Let me repeat: Firstly, that's false: he has complained - as have many people over the years about how zzz handles (or doesn't handle) the project. From everything to the above to refusing to use git - and more. There are lots of IRC dumps to prove this. Project decisions affect code decisions: don't forget that. Secondly, and more importantly; someone who is opposed to collaborating with the vast majority of scientists and researchers among the global human population (people that only live on clearnet) thinks that this approach is "working fine so far". ...This is Java I2P, folks. This is zzz, the lead and only developer of java I2P, and java I2P contributor for at least 13 years. Things have not changed in this area. This attitude has not changed over the years. Don't believe me? Feel free to ask other developers such as psi about I2P development. You can find him working on LokiNet. psi had been involved with the java I2P project even longer than I (yes, I also contributed to java I2P), so I'm sure he can provide another perspective. Or even ask str4d for that matter, as he is working on Ire among other things. Sidenote: Sekreta will support LokiNet and Ire (see list of all supported networks). But, how is this an issue now? How does this affect Monero? Now, in January 2019, it's been almost a year later and there's still no audit of NTCP2. No official 3rd party review of anything related to NTCP2 outside of Noise (which is a completely separate component, and which received its own audit long ago). In March of 2018, java I2P's budget was around ~4 million USD in cryptocurrency and not a single penny was spent for a formal audit. Wrap that around your head. Either java I2P is still a lost cause of a project or their intentions behind such a lack of responsible development should be considered questionable at best. This is not an isolated incident. This type of history has only been repeating throughout every aspect of java I2P development - year after year after year after year. And now, a few Monero contributors want to integrate technology that they know very little about - and that's a big problem for Monero. So, when flakeypony pretends to not know this, despite using the exact reasons above (shall I provide logs?) as a catalyst to not use java I2P (specifically, zzz), you must question his legitimacy not only as an honest person but as any sort of anonymity consultant; as he has repeatedly proven himself to be lackluster in both of these areas. In other words, it's time to get with the program. Archaic tools are for archaic problems. It's laughable that #5070 is even a proposal but I can smell the desperation in the air because of the 2018 bear. Monero is better-off using a SOCKS proxy at this point, at least until Sekreta is integrated. Otherwise, deciding parties will do Monero users a massive disservice by enforcing a single implementation / single-network per-channel solution. At least with a temporary "only SOCKS proxy" solution, you'll have less blood on your hands (everyone can still use any existing I2P implementation or network of their choice so long as the implementation of choice support a SOCKS proxy). Lastly, allowing anything Java to infiltrate the Monero implementation is literally the writing on the wall for Monero innovation. I'm so sad to see it end this way; especially since I'm one of the few who have dedicated their lives to building Monero Project since 2015. |
|
Some fact-checking here:
Pull the monotone source and search for "zab" and "zab2". If you started in 2015 I got you beat by 3 years.
Psi aka Jeff aka https://github.com/majestrate has already made his opinion known here https://i2p.rocks/blog/kovri-and-the-curious-case-of-code-rot-part-2.html and here https://i2p.rocks/blog/kovri-and-the-curious-case-of-code-rot-part-3.html but do ask him.
If Monero hates Java that much, i2pd supports the SAM protocol as well. I don't know C++ so I can't vouch for their code, but orignal had no problem collaborating with us on NTCP2. |
I have no opinion on this. I am also not a C/C++ person, so it'd be completely up to whoever takes on the baton from here to decide how this might work. It'd be nice to be able to give people the option to hide all of their Monero related traffic.
I did not include the jbigi.jar for this first iteration, for simplicity. Some of the other CPU drain could be because the node is starting for the first time. As @zlatinb points out, calling the native crypto libraries by including jbigi.jar in the future will result in a big crypto speedup when integrated. I've read that the RSA/ElGamal crypto is also much slower than the newer EC crypto in NTCP2. Perhaps someone can comment on this.
I'm a big proponent of ending up with a well written, small footprint, security audited C/C++ I2P router with multiple collaborators from both within and outside the Monero community. The only reason I suggested Java-I2P in the meanwhile is because that dream scenario seems to be at least 1-2 years away. |
@zlatinb I wasn't exactly sure what files to include to make this work, other than the jar file. Am I correct in thinking that the I2P-Java project is currently only providing native builds of the bigint libraries for Linux, and not for Win/MacOS? On the topic of the I2P base directory, it looks like it's possible to leave it completely empty. However, there are certificate checking errors, and so it seems like it would be best to include the certificates, hosts, blocklist and geoip content. Do you agree? How necessary do you think each of these are? I was also struggling to figure out where this content was from, since it's not in your Github repository (this data dir content is present in the Ubuntu distribution of I2P-Java). |
There are libraries for Linux, Windows, MacOS and FreeBSD. The full list is here https://github.com/i2p/i2p.i2p/tree/master/installer/lib/jbigi . The libraries that start with "jbigi" are the actual implementations for platform*CPU architecture. Those that start with "jcpuid" are used to identify the CPU architecture so that the optimal implementation library can be loaded. Looking at router and wrapper logs during startup is your best bet for identifying the correct bundling.
|
|
@Gingeropolous Please could you check your ~/.i2p/wrapper.log file, and see if it contains the lines: On my laptop, the I2P Java process is rarely using more than 1-2% CPU, so I'm not sure why this is happening for you. Are you running on the same machine where you installed the Ubuntu I2P package? It looks like when you run this embedded I2P project, if you run it on the same machine where you installed the Ubuntu I2P package, then it will be using native BigInteger operations. This is because the NativeBigInteger class that calls the native code is already contained in the i2p.jar file that this project uses, and you'll already have the libjbigi.so native library installed in your /usr/lib/jni directory. |
|
I've just released a new version that runs on Linux, and from Linux will build the Linux, MacOS and Windows zero-dependency distributions. Beware that the scripts will need to download all 3 JDKs, totalling hundreds of megabytes. Unlike the previous version of this project, you will not need to pre-install Linux packages for Java, the Ant build tool or I2P in order to build the zero-dependency router distributions. The native BigInteger libraries are now included in distributions for all platforms. You can check if the native libraries are being used by launching the router and checking the contents of the i2p.config/wrapper.log log file. The final zero-dependency distribution sizes are as follows:
|
|
yeah, im running the ubuntu i2p package @knaccc , those were the numbers I was talking about. Does the version you are working with end up using less resources? I might try out your version... but then i gotta migrate my tunnel configurations etc... |
|
This project has been renamed I2P-zero, because it means less typing for me and less confusion when discussing it. New github repo URL: https://github.com/knaccc/i2p-zero |
|
With a couple of build script tweaks I got this building and running on macos. Uses hardly any resources! Nice work @knaccc. |
|
Few notes
@jtgrassie It takes quite some time for a new router to start contributing to the network, until that happens it won't be using pretty much any resources. @Gingeropolous this distribution does not include i2ptunnel.jar, so it will not open any of your tunnels. |
But what if I don't want and need bundling because I'm already running my own i2p router or tor? I'd just want monero to use what I already have setup and don't bundle stuff I'll never use. |
Then there'd probably be a startup flag to tell the daemon to not run the bundled i2p , and compile flags to not bring it in... if i had to guess, and if i have an actual understanding of what bundling means. |
|
@orignal Dude, stop. You're embarrassing yourself. Please stop. You literally disappeared and, as far as anyone could tell, abandoned i2pd. After AN ENTIRE MONTH a few people decided to fork the project. I offered them a home at the Monero project. It doesn't matter if it was working with Anoncoin or not, the project was forked and it was up to the new maintainer (@anonimal) to figure out what to do. The fact that you came back later is tangential, because you came back and were immediately hostile to everyone. This fact is evidenced by the terms you use today like "theft" instead of "forked my abandoned code". I get that there's a language barrier, but even the IRC logs you posted on Reddit last year make you look like you misinterpreted the situation, not like @anonimal and I are evil, conniving tricksters who stole your livelihood. It's better if you just walk away from this with your dignity intact. |
|
@fluffypony you didn't fork, you decided to hijack entire i2pd. Without authorization from anybody. Have you shared info about anci2pd project among other Monero people? I doubt. |
|
@orignal Forking is not hijacking, stop being ridiculous. Also, that commit was EinMByte, who was/is a prominent i2p contributor. I have no control over what he did on a repo that I don't have collaborator privileges on. Failed to pay whom or for what? I literally have no idea what you're talking about. What repository did I not have access to? The i2pd repo was open, was it not? I don't recall being given special privileges to access anything. Again, the fact that you came back and were working on i2pd was no secret. anonimal chose to continue with Kovri regardless of whatever i2pd and Anoncoin were doing, and that was his call. There is no conspiracy here, nobody was hiding information that was public in any event. The excess paranoia is not helpful. This argument isn't pushing the conversation further. I'm going to add you to my ignore list, feel free to continue with your paranoid rants as you wish. |
|
@orignal are you the guy who erased git history https://github.com/monero-project/kovri-docs/blob/master/i18n/en/faq.md#what-were-the-turning-points-that-lead-to-forking-from-i2pd-and-why-are-there-two-i2pd-repositories-one-on-bitbucket-and-one-on-github, moved project to https://bitbucket.org/orignal/i2pd/src/master/ and then came back? |
He has never been an i2pd contributor before. Furthermore he has been declined by me explicitly.
Let me refresh your memory. When i2pcpp guy has stopped the project, you start buzzing me and promised to pay on behalf of the Monero. Or you didn't and just wanted to say "hi"?
Blaming 1M this time?
Answer yes or no if you have shared this info among Monero people. |
They have erased the history not me. I continued on the bitbucket (and I keep maintaining that mirror now) because I was not able to fight with this gang by that time due personal circumstances. When i2pd got back to github I continued from the original tree to retain the history and moved their stuff to the branch "cryptopp" as evidence of the theft. |
|
In a probably futile attempt to get this thread back on topic I'd like to point out that it is a good idea to as a lawyer or Oracle themselves whether the BCL applies to the bundles produced by jlink. The BCL says the full JRE is subject to export restrictions (Iran, North Korea, etc) but says nothing about jlink. |
|
Here's a starting point for figuring out license concepts: Oracle open sourced the JDK under GPLv2 with Classpath Exception. See https://openjdk.java.net/legal/gplv2+ce.html The classpath exception means that any code that runs inside an OpenJDK JVM is not subject to the GPLv2 and is free to be licensed in any way. I think the BCL (Binary Code License) that zlatinb is referring to is related to the OracleJDK and not the OpenJDK, as far as I can see. The OpenJDK is available as packages in most major Linux distributions, so I wonder whether they've already looked into possible export restrictions. |
|
@knaccc I think it's simpler than that. How did you find the download links for the different JDKs? Did you have to click "agree" to any license, or was there text that says something along the lines of "By downloading this software you agree to blah blah"? If yes, then whatever license that is (most likely BCL) is what applies. |
|
@zlatinb If you download the OracleJDK, you have to agree to license conditions before downloading, because that's the Oracle version. The OpenJDK however does not require agreement before downloading. The links are here: https://jdk.java.net/11/ You can even build the OpenJDK from source, using instructions here: http://cr.openjdk.java.net/~ihse/demo-new-build-readme/common/doc/building.html |
|
@knaccc fantastic! No export restrictions then. I would have hated to have to configure our download servers to block the people who need i2p most. |
|
Meeh here. After reading this thread, I can't hold back a comment; omfg so much fucking stupid I've read (aggressive posts, claims, etc). But ok, I'll jump over that and keep the topic. We decided earlier today I'm gonna maintain the I2P-Java's libsam3 from now on. aka libsam3 helpdesk. So this shouldn't be a problem. |
|
@mikalv correct me if I'm wrong. You use new libsam3 in Anoncoin-gost already |
|
@orignal yes I think you're right |
|
I've got some updated information about the export restrictions on strong crypto in the JDK. I've been told that If things are hosted in the US, then web sites like https://openjdk.java.net/ will cover themselves legally by asking people not to export downloaded content to certain countries. There is actually a ToU on the US hosted OpenJDK site for example https://openjdk.java.net/legal/tou/terms For this reason, I've been told about an IBM sponsored JDK binary download site here: https://adoptopenjdk.net/releases.html?variant=openjdk11&jvmVariant=hotspot GNU have a FAQ item about this: https://www.gnu.org/licenses/gpl-faq.en.html#ExportWarranties The bottom line seems to be that since it's GPLv2 software, what counts is that the country you're downloading it from does not enforce export rules. So no US hosting. |
|
@knaccc are you sure these export restrictions do not apply only to the unlimited strength JCE? That is a separate download. Also, it is not clear how much of the crypto gets jlink-ed into the final bundle. |
In the old days, the unlimited strength crypto policy jar was a separate download. They changed this and unlimited strength is now the default. Either way, it's all GPLv2 now. |
|
@knaccc two small things:
|
|
@zlatinb thanks, done and pushed |
|
@jtgrassie Has discovered https://jdk.java.net/jpackage/ https://openjdk.java.net/jeps/343 which will build self-contained msi/exe/deb/rpm/dmg/app files. This is currently scheduled to be available in OpenJDK 13, due for release in September 2019. |
|
I2P-zero GUI just released. Screenshots at the top of the README here: |
|
Fantastic job! |
|
@knaccc I did some experimenting with different compression schemes and managed to get all of I2P + a windows JRE (with full I2P dependencies, not just those used by zero) down to 32MB. To get there, I disabled compression in jlink and in all I2P jars, then compressed the final bundle with lzma. |
|
Is this still relevant ? I think the GUI uses i2p0 now ? |
GUI is working on I2P-zero support. I think this can be closed. |
|
Alright, please reopen if this gets new life as an alternative. +invalid |
and others
I2P-Java is a relatively mature project with active developers.
A few years ago, it would have been unthinkable to have to ask people to separately install a 200 MB JVM on their machine before being able to run a Monero wallet that wanted to route connections through I2P-Java.
Things have changed since then. Java 9 implemented modularization to allow zero dependency native apps with small footprints. This brings the footprint of an app with a bundled JVM down to as low as 22 MB (11 MB with compression). The end user will have no idea they're running a JVM, and no JVM will be installed. This means no conflicts with any other JVMs they may previously have installed. Even better, we now have the open source (GPLv2) OpenJDK. See https://steveperkins.com/using-java-9-modularization-to-ship-zero-dependency-native-apps/
zab_, one of the I2P developers, has noted in the Monero reddit that we could interface with I2P-Java via the SAM library https://github.com/i2p/libsam3 or via the lower level I2CP protocol https://geti2p.net/en/docs/protocol/i2cp (zab_'s comment here).
The text was updated successfully, but these errors were encountered: