First class language support for proxies and delegatecall

Language Design
1

I believe that Solidity would benefit from first class support that models realistic deployment scenarios for EVM contracts. For a variety of reasons, not the least of which is the size constraint on contracts, complex projects end up releasing multiple contracts that are closely affiliated with each other, and/or which together compose a whole. Unless I am misunderstanding the language, or am uninformed about upcoming features, I believe these issues are not well supported in the existing and/or planned language.

Rather than present a wholly formed set of language proposals, I wanted to state this problem and get some conversation going. Yes, I am of course aware that openzep and other projects have “proxy” contracts etc but I (flame suit on) would generally categorize all of these are cobbled together hacks on top of less than ideal language support.

Speaking philosophically, the target EVM and its associated blockchain tools together with solc compose the complete execution environment and should ideally be more completely integrated, so that realizing a functioning holistic contract system has a smaller error surface than is currently experienced by developers and maintainers. A microcosm of these issues is evident in how external libraries are handled and how errors can occur therein.

There are a number of frustrations that come up when attempting to use solidity 0.8.20 to write, deploy and maintain a closely cooperating set of contracts with the intention of offering a composed interface through a single storage address while also trying to do so with minimal gas costs.

Again, I could take the approach of writing a long paper on this topic but that would delay getting the conversation started. So instead, let me offer a small example. Lets pretend for a minute that I want to define an interface Iface which contains three functions fA, fB, and fC but each of those three functions is so large that it cannot be deployed to the same address. Remember, this is just a contrived example, to discuss the issues. In practice, Iface would have tens of smaller functions which as a group are too large to deploy to a single address.

For devops and language reasons, I end up wanting to declare a storage contract parent CStorage that defines all the storage vars of and which is the Iface. But now, in order to deploy, I must somehow create three derived contracts into which I put one of the three functions above. Since those contracts do not define the other functions, they must be declared abstract. If they are abstract, no code will be emitted for them. If they define stubs for the other functions, then delegatecalls to their addresses made to those functions will succeed when they arguably should fail. If the stubs did not exist, and the delegatecalls failed, it would be possible to write a proxy that (granted, for more gas) could try each available deployment address until one succeeded. While that sounds expensive, consider public view functions which in practice cost no gas because they are not executed on-chain. Under the current system, we have to register function selectors with the proxy and pay the cost of a map lookup for every call in order to successfully dispatch to the correct implementation address. Since function selectors MUST be keccak hashes, there is no way to write a proxy dispatcher which (for example) is instead based on looking at the first byte of the selector or function name (i.e., deterministic precomputed dispatch is impossible). Furthermore, the registration procedure is “outside the language” and error prone. The language offers no way for a programmer to declare that contracts A,B,C together implement Iface, in parts and therefore is not helpful in ensuring that Iface is correctly distributed across those implementation artifacts.

I have many ideas that address these various issues. This is by no means an exhaustive list of the various issues that need to be addressed. Another one, for example, is that I believe there is no way to (without diving into assembly) make a delegatecall through a contract address. External libraries are not enough, as these do not have first class access to storage of the caller without polluting their parameters with storage pointers, which just means that developers are tempted to put all of a contract’s variables into a struct … which is precisely what the storage vars of a contract are (thing this pointer in other languages).

I hope the foregoing is sufficient to start a conversation. I am not conversant with the current priorities of the solidity devs, but unless non of this will matter soon because a WASM evm is around the corner and size and other existing limitations are about to be tossed out the window (i.e., major changes to the execution context are imminent), these issues seem like they should be close to top priority for the language.

I appreciate that I am not an expert in everything solidity and do understand that I may just have it all wrong. Again, this is just meant to start a conversation and share some of my thoughts/experience after working on a large complex project using the available tools.

2

For example, solc could have a mode --emit where no bytecode is emitted for contracts that appear within an emit {} block, except as provided for within the emit {} bock. Other contracts would behave normally. Then, developers could provide for a complete (oversized/total) definition of a contract without regard to deployment concerns. The language and its warnings etc would behave normally, including warnings on undefined functions, abstract contracts etc. Then:

import "WholeContract.sol";

emit {
   contract PartA of WholeContract is SomeMixin {
         @delegatecall PartB addrPartB; //See discussion below for this line
         function fA(...); //emits the bytecode for the function fA()
         function initializer() override {} //emits an instantly defined version of initializer()
         function fallback() ... { /*delegatecall to PartB*/ } //emits an instantly defined version of fallback()
   }
   contract PartB of WholeContract {
         function fB(...);
   }
};

emit sections would cause solc to create bytecode that selectively includes the bytecode definitions from the of WholeContract contract decls elsewhere defined. These partial contracts could include overriding or additional functions as well (from the is ... part of the decl) The goal here is to NOT pollute the “clean/raw/normal” WholeContract definitions that would continue to follow the unencumbered syntax of the language as-is.

The language could also decorators (or perhaps per-function pragmas?) so that devs can control internal elements of the emitted bytecode, and could conceivably limit some of those to emit sections as well. For example an @selector(0xAABBCCDD) would allow assigning a specific selector to a given function regardless of its name and parameters. Any necessary calling guards to prevent mishaps can be implemented, I am oblivious to those concerns as I write this. This would allow efficient proxies to “facets” (to use the term from “that” ERP). Similar could be achieved by allowing a “selector prefix” statement or “selector generator” function to be defined within an emit block.

The point here is to not force coders to pollute the logic oriented WholeContract source code itself with function name/selector registrations in their initialize()/constructor() and allow cleaner separation of concerns (deployment structure/runtime issues vs logic).

I believe the storage layout of contract vars needs to be (re)worked as well. Is there some reason I’m (again) oblivious to, that contract vars are bunched together at low storage addresses? This seems to be a source of much concern for proxy deployments, where any updates to the layout can cause havoc upon proxy updates to existing storage deployments. If those vars were instead widely dispersed to their keccak hash locations that would shift the issue from “how variables are sized and ordered” to “how variables may be renamed in new versions”. Combined with either an @location() decorator or @keccakName() decorator, to cause their location to remain at the original name of a variable, contract updates would be much safer.

I would also like to see something like an @delegatecall decorator that can be placed within emit{} sections. This would cause the following code within WholeContract to emit, within PartA only, a delegatecall to the address, instead of a CALL that would otherwise have to pass through fallback() etc. The storage contract would be initialized with the address of PartA at that var, during deployment.

WholeContract partB;
function fA() { partB.fB(); }

Of course, this pollutes the logic contract with some aspects of how the proxy division will be done in emit{}, so there are better ways … but I’m suggesting something that can be briefly described here. A more complex solution would involve the compiler supporting directives within emit{} that allow it to detect this.fB(); calls in order to replace them with parB.fB(); delegatecalls instead, combined with allowing emit{} sections to declare additional storage vars, and via overrides of lets say the initialize() function of said WholeContract, allow the emit{} section to provide a deployment-specific initializer.

Again, just thinking out loud here and hoping to spur some conversation. IMHO, unless massive changes to the EVM itself are imminent, these issues feel urgent to developers of complex contract systems.

1 Like