Deprecate eth_compile family of methods

[rphmeier]

(Raised and discussed briefly in ethereum/pm#5)

Motivation

The eth_compile* RPC calls are beyond the responsibilities of the eth API, which can be considered to encompass the core functionality of an Ethereum node. As Ethereum deals in low-level EVM bytecode, the API surface should also not expose anything relating to higher-level programming languages. Managing, maintaining, and updating compiler versions, flags, and produced ABIs adds additional complexity to implementation.

Furthermore, having only calls for Solidity, Serpent, and LLL serves to unnaturally limit the usage of other EVM-based languages by giving those 3 an elevated status, potentially stifling development and adoption of new solutions.

Lastly, the eth_compile* family of methods is not sufficient for certain use-cases in DApp development, as they don't allow pinning to specific compiler versions .

Action

Mark eth_compileSolidity, eth_compileSerpent, and eth_compileLLL methods as deprecated in the docs, and suggest usage of proper development tools instead.

Compiler functionality may still be provided, but within node-specific RPC namespaces.

[frozeman]

👍

[axic]

👍

[Arachnid]

👍

[Smithgift]

👍

While these can be used as a (bad) link in a toolchain to get code into in-REPL bytecode, I've actually found it easier to cut and paste the bytecode straight from browser-solidity or whatnot. Or, better, construct a better automatic toolchain. (Or use a framework, if that's your style.)

[karalabe]

👍

[swaldman]

So, I'm just seeing this, and it looks like I'm way out of consensus, but count me as an objector.

I've written a development and interaction tool (a plugin to sbt, the main Scala build tool) that currently relies upon JSON-RPC-based solidity compilation. ( It's https://github.com/swaldman/sbt-ethereum ) Obviously I'm grumpy that this will force some hassle and reorganization to use a local solc instead.

But, beyond that, I think this is a mistake, an unfortunate choice. Here's why.

Currently, most Ethereum development is focused on big dApps fronted by slick websites or mobile apps, funded by token sales, VCs etc.

I kind of think that's surprising, and perhaps even unfortunate. After the DAO hack, Nick Szabo pointed out that maybe inherently experimental DAOs should start not as multimillionaire virtual VCs, but as local clubs between friends that might manage 10s or 100s of dollars in value.

One way to do that is to hire 3 smart contract developers, a mobile app developer, and 4 javascript developers to create DAOs-R-Us, Inc, and build a website that lets a group of friends define and customize a DAO via a pretty GUI and then interact with it by mobile app. But then the DAO experiments won't go beyond whatever DAOs-R-Us decides to offer.

For truly small, local experiments, there has to be a low-rent path to experimentation, deployment, and interaction with smart contracts that don't require such a huge range of skillsets.

I'm really excited about the possibility of a culture of extremely simple, easy-to-tinker-with, command-line-interacted smart contracts, to be played with at (hopefully) low stakes. My hope was to encourage very small, simple smart-contract applications in the form of self-contained, transportable builds. My toolchain relies on JSON-RPC solc not because it's hard to wrap a local compiler, but because that way, a user can clone (or even download a zip of) a repository, run a script, and immediately compile and deploy smart contracts from the command line. I host a json-rpc node, so users will be able to compile and deploy with no almost ceremony at all. The only external dependency is a JVM (which Oracle makes easy for very modestly technical users to install without frustration), the rest of the environment downloads itself upon first execution of a wrapper script. Forcing users to get a local solc running and into ones path is a significant added barrier.

Obviously, if it's just me, who gives a f%$k. I can wrap solc myself in a network service and host that too. But "suggest[ing] usage of proper development tools" is a tacit way of saying that we as a community think that smart-contract development should be for professionals, not so much a thing for amateurs to tinker with. Maybe that's the proper lesson to learn from the DAO hack and other subtle, costly smart-contract mishaps. But I'd take an opposite view, that part of why we've seen such problems is that we've started by pretending there are professionals capable of building applications with large, complicated state spaces in a domain whose quirks we are all still discovering. I think we'd be better off starting with simple bespoke small-scale apps deployed by end-users.

Compilation-as-a-service facilitates that. I thought it fortunate and foresighted that the JSON-RPC API included it, and am very disappointed it seems likely to be removed.

My alternative suggestion would be to modify the API so that one can query a JSON-RPC node for the languages it supports, and the supply the language name as an argument to a generic compile call. I agree that there is no reason to privilege the three earliest EVM languages, and it'd be ugly to have 20 eth_compile_XXX calls as people experiment and invent new ones.

Thanks all of you for the work you do making Ethereum the most interesting ecosystem on the planet.

[miohtama]

@swaldman As far as I understood the core issue here is that it's difficult for average user to have solc on their local computer for SBT-Ethereum to use and you want to provide easy out-of-the-box exprience for developing smart contracts. I agree this is exactly what the community needs.

Instead of mitigating the solc installation woes with a remote compile, I'd suggest taking a different approach; As Scala needs to have a JVM and library dependencies in any case, just add a dependency to a JavaScript interpreter (possible?) Then you can embed Emscripted JavaScript based solc in your local installation and call it directly from Scala.

As even browser-solidity web browser HTML page can run it solc locally we should embrace integrating it natively to other programming languages too. No need to wrap solc as a remote service.

solc-js (also see solc-bin) https://github.com/ethereum/solc-js

[swaldman]

@miohtama Thanks! That's a useful suggestion. (Great minds think alike. @fjl suggested the same, but I misinterpreted him to mean installing something outside the JVM.) If compilation is gone for good, I'll certainly give that a shot. If within-JVM compilation via JavaScript works and is performant, it will solve my immediate problem, for Solidity (which is all I currently support).

Still, in the general case, there may be languages without JavaScript compilers and toolchains that don't have a sufficient JavaScript interpreter readily at hand.

Thanks again for the great suggestion.

[frozeman]

I do agree with it in so far that the node shouldn't provide that, but it could certainly be provided in Mist as an API, as loading solc as a lib into your project is quite huge, if you want to do on the fly compiling, e.g. for a contract building dapp

[rphmeier]

"suggest[ing] usage of proper development tools" is a tacit way of saying that we as a community think that smart-contract development should be for professionals, not so much a thing for amateurs to tinker with

That wasn't at all the intention. The fact is that an ethereum node operates at the level of EVM bytecode, and not at the level of smart contract languages and ABIs. It is unwieldy to impose a requirement to integrate EVM languages into the nodes themselves precisely for this reason. Whether or not we change the API to use a generic "compile" call, we'd still have to force the node to deal changing compiler versions with differing or updated ABI formats between languages -- such that outputs of eth_compile('solidity', CODE) would vary wildly from node to node. In my opinion, this is the complete opposite of a stable API.

The reason for suggesting proper development tools is that these will vary wildly from language to language and there's no sense trying to squeeze all of that into an RPC layer largely meant to interact with raw blockchain data itself. Using correct developer tools is something we want to teach amateurs to do as well: it's not as though students in their first Java class are told to invoke javac directly. Instead, we give them an IDE like Eclipse, NetBeans, or IntelliJ. Projects like Embark or Truffle make testing and deployment in the Ethereum world much more simple, and compilers which can be cross-compiled to JS or WASM are just as flexible if not more so than an RPC endpoint.

[Smithgift]

Even were we to keep eth_compile, I don't think it's reasonable to require light clients to include solc (or what have you.) Considering that light clients will probably be more popular, once they're production-ready, there's a possibility that eth_compile might not work on a given node at all.

[swaldman]

@rphmeier So, I don't want to push this too hard. I think I'm a minority of one in a battle already lost. I appreciate your work.

But... it is possible to specify things, and ask implementations to live up to the specification. That outputs have been underspecified and there have been inconsistencies between nodes that have had to be resolved is a general issue with any evolving spec. The answer to that problem is to agree on a protocol for each language, and adhere to the protocol. I understand that from a node implementers' perspective, the whole process is an added annoyance. But, from at least some users' perspective (at least one this user, and the hypothetical people he hopes to interest), the functionality is useful. The annoyance is not for naught.

I agree about improving toolchain. That's my project very precisely. The issue here is about providing the capabilities necessary to support a rich, performant toolchain with extremely low barriers to entry. Both truffle and embark are node.js projects that can make use of JavaScript solc natively. Since building and deploying binaries is often a platform-dependent hassle with dependency annoyances, without network compilation, we may be restricting the easy-ish-to-use toolchain to node.js. Maybe that's okay, but obviously as the writer of Scala-based tools, I object.

Regarding @Smithgift's comment and light clients, if we were to adopt an API that could use the existing eth_getCompilers call to test the node's available compilers, that could return an empty sequence for very lightweight clients. Indeed, node implementers could opt out of dealing with compilation just by returning an empty sequence from that function, and sending back a "language not supported" error for any call to language-independent eth_compile function. So there need be almost no extra overhead at all. But nodes that wanted to provide the functionality could do so, in a standard way, and if there are more people like me out there (or people who like the tools we would build), non-support of popular languages might be a competitive disadvantage. Since this is an open-source world, probably the right thing for node implementers to do would be to define internal API for hooking in support for languages, allowing each language's enthusiasts to contribute implementations if they wish.

You are right @rphmeier that there is no logical connection between blockchain-related services and language compilation. Certainly, networked compilation could be handled by a separate service; that's a solution I'm likely to consider if I remain out-of-consensus here. But supporting compilation on nodes does make sense from clients' perspective, because clients will want to test and deploy what they compile. Clients will always need to be configured with a JSON-RPC endpoint, so it is a convenient service provider. Networked compilation elsewhere would mean a setting up on the server-side and then configuring for clients an extra network service.

The possibility of treating JavaScript/emscripten/WASM as a kind of universal virtual machine that all toolchain implementers would be expected to have available is interesting. It's kind of an empirical question. Will implementers of new languages find it a reasonable burden to follow in solc's footsteps and compile to JavaScript? Are JavaScript interpreters sufficiently available across languages and sufficiently performant (in memory footprint as well as speed) to render this viable? I don't know. Again, it's an intriguing idea, one I'm likely to explore for solidity if eth_compile is really gone. But it strikes me as premature to rely upon this.

Thank you again for your work.

[chriseth]

@swaldman I think it is better to couple Solidity with the deployment tools instead of the blockchain node. If you don't like running asm.js/wasm inside the JVM, perhaps we / you could give gcc-bridge a shot and also compile Solidity to JVM. What do you think, @romanman ?

[danfinlay]

Regardless of where you stood on this topic, users relied on this method, and they were suddenly surprised to see them removed.

I'm a big fan of deprecation warnings for situations like this. For a period of time before the ultimate removal, throw a warning-level error that user interfaces will display, to get the developer community on the same page about the changes they will have to make.

Breaking software in user-space makes for a difficult platform to build on.

[digitaldonkey]

The JsonRPC language compiling is a great fallback to create easy to use apps in various languages.
Why this ticket is open, while the methods are already gone in current geth?
Any chance/plan to get them back?

[andviane]

Please at least update the documentation at http://ethdocs.org/en/latest/contracts-and-transactions/contracts.html. It still written referring to the compiler methods and it is not obvious for the novice what to do instead when these methods are not available.

[spawn66336]

Because geth deprecated the command-line compiler interface after version 1.6, I was puzzled for a long time.I wrote a simple tutorial to describe how to compile and deploy a contract after geth1.6.
Hope this tutorial will help you.
http://blog.csdn.net/weixin_40401264/article/details/78136346

[redi-lernera]

So, all and all, how do I get started with all this? I am new to this, but what I see is that what limited tutorials there were available, no longer work. I did succeed with using command line solc compilation and pasted the resulting bytes back into my javascript code for deployment, but this is a barbaric approach. Any suggestions/pointers on the up to date tutorials that actually work? Thank you

[VoR0220]

@redi-lernera https://github.com/ethereum/solc-js

[Tectract]

Hello there :)

I wrote a tutorial with some code that is still generally functional that uses browser-solc as the compilation method. Here: How To Learn Solidity: The Ultimate Ethereum Coding Tutorial

relevant source code is here: https://github.com/Tectract/EthDeployer/blob/master/src/Deploy.js

I generally think it would be really nice if the node implementation included the most recent solc, with or without explicit developer support on the node side, because it would make it a hell of a lot easier for people to actually compile their Solidity code, and I see ease-of-adoption as a major consideration for Eth to stay relevant in the face of competition right now.

If you guys make a decision to remove solc permanently, then I would suggest that you consider UPDATING THE DOCUMENTATION as part of that same ticket. Right now almost all the documentation on how to compile solidity code is outdated and it's very difficult for non-professional programmers to figure that stuff out.

oh yeah I also wanted to mention this issue, with a ton of confused people that went on for the last 2 years or so because of this problem, which is now considered closed (without resolution imho):

ethereum/go-ethereum#3793

[github-actions]

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

[github-actions]

This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.