Discussion thread: High-level language support for transient storage - Caveats & next Steps

Language Design
1

We are starting this thread to facilitate open discussions around transient storage and acknowledge active contributions made so far by community members.

We would like to thank Hadrien Croubois (@amxx) from OpenZeppelin for taking an initiative and putting in the time and effort to submit a PR that implements native support for transient storage in Solidity. Additionally, we would like to thank @cfromknecht, @tynes, and @moodysalem for contributing to the PR.

Before we dive into details of the matter, here are a few helpful links if you would like to get up to speed with recent material on transient storage from the team:

We’re delighted to see a strong sense of commitment in our community to making EVM more cost-efficient. We want to thank everyone for recognizing the importance of transient storage and for taking the time to read this post. outlining the challenges related to implementing this feature in Solidity and the next steps for moving forward.

However, there are certain challenges with implementation that we would like to share that are crucial to consider for current and future contributors:

  • The prior assumption of the EVM having a fixed set of the data locations means that thorough testing is required and the number of combinations of data locations for which we need to test and verify behaviour increases significantly.
  • Guiding contributors through the testing infrastructure for such changes would be non-trivial for the team as well as the contributors.
  • The intricacies of conversion behaviour make this in particular an issue for complex types, i.e. reference types and mappings.
  • Defining the layout in transient storage, i.e. the packing behaviour and the question of interleaving with storage or defining an independent layout for transient storage variables, is another challenge.
  • transient is not a previously reserved keyword which makes @amxx’s PR draft a breaking change if it were to be merged. There are ways to make this non-breaking through certain parser hacks. Our current plan is for the team to start with building appropriate parser support with the required workarounds.

Given the above challenges and the ramifications of implementing non-trivial changes in a large codebase that impacts several users, we have concluded that it would be best for the team to take the development of this PR forward. This decision is also a product of mutual agreement with @amxx based on discussions in one of the Wednesday design calls.

Previously, our expectation was that community members and major libraries would experiment with building prototypes for transient storage support in Solidity on the basis of the tstore and tload opcodes alone. This would further facilitate community-wide discussions regarding the questions listed above and provide us with a foundation for decision-making around future development.

However, we recognize the importance of making initial progress in the direction of implementing high-level language support for transient storage. Hence, our planned next steps are:

  • For the time-being, make the breaking change due to the missing transient keyword non-breaking using parser hacks. The plan for the next breaking release then being transient becoming a proper keyword.
  • Gather more input on potential (possibly optional) autoclearing behaviour by the compiler and its feasibility for various types.
  • Confirm with the user community that storage and transient storage layouts should be independent (i.e. continuous and potentially overlapping).
  • Work towards supporting value types as preliminary feature that would serve as baseline support for other complex types in Transient Storage.
  • Talk to teams and power users to understand specific use-cases for complex structures like mappings in transient storage.

:link: You can track the progress using the catch-all issue for High-Level Language Support for Transient Storage.

With that being said, it is also important to note that the safe delivery of a complex feature set like full language support for transient storage requires a lot of care and has to be done in separate smaller iterations that are reviewable and verifiable across multiple releases. That means, in particular, that support for reference types in transient storage will take some time. In the meantime, we will be closely watching the growing interest for Transient Storage support in Solidity and encourage the community to continue giving us valuable feedback and insights that would help us in building the desired support for Transient Storage.

:memo: Note: The Solidity team hosts weekly language design discussions to provide a channel for our community to meet with us and contribute to the future of Solidity. If you have issues or pull requests to discuss, or are interested in hearing what the team and contributors are working on, you can join our public team call every Wednesdays at 3PM CET/CEST. The call takes place on Jitsi.

Sincerely,
The Solidity Team