Testnet DayLight | Release Cycle Update #20
Our development planning features a Release Cycle of three weeks, providing a consistent stream of updates to the community, developers, and businesses relying on Dusk technology.
The publicly available Dusk Network GitHub contains over 18 active repositories, each focused on a different technical subject. Progress in each is ongoing and can be followed in real-time. The Release Cycle process is currently applied to some of our most active repositories.
Release Cycle planning has not yet been applied to all repositories. When they are, future installments of the Release Cycle updates will see an expansion of repositories in the spotlight.
For the community's benefit, we’ve chosen a selection of some (but not all) of the work that is being done in our repositories, including recent advancements.
merkle
Open issues
Use criterion for the benchmarks #32
Implement the benchmarks using criterion.
Add detailed explanation for the public types, traits and methods in this crate #31
Flesh up the documentation on public types, traits and methods. Especially the poseidon implementation and opening gadget.
Remove positions map from the tree structure #24
Given the fact that we can arbitrarily walk the tree, it shouldn't be necessary to include existing positions since we could use a walker to determine both how many positions in the tree are filled (Tree::len()), as well which positions are filled (Tree::contains).
Closed issues
Add tree with poseidon hash and opening gadget #29
Add the merkle tree with the poseidon hash and the merkle opening gadget in ZK, both generic over height and arity.
Fix insertion #27
when items get inserted at random positions, the child_index returned by child_location is wrong.
Prepare Opening to be used outside of the crate #25
We want the Opening to be usable outside the crate as well. For this reason we need accessors for the fields of Opening.
Add a Walker for the merkle tree that iterates through the leaves of the tree.
Poseidon252
Open issues
Add sponge framework over fixed size input#215
Add a sponge that is generic over the input length and therefore doesn't need to add a padding at the end of the input.
Closed issues
Remove merkle tree logic from this repo #212
Remove the merkle tree and opening proof generation from this repo. The merkle tree structure is instead implemented in the dusk-merkle crate, see issue #29
citadel
Open issues
Migrate Citadel to use dusk-merkle crate rather than Poseidon252 #41
Merkle opening circuit functionality should be taken from the dusk-merkle crate as Poseidon252 has been reduced to a crate providing hash only and the merkle opening functionality has been moved to the dusk-merkle create.
Extend specs with a section about Citadel smart contract #37
Current specs lack information about the smart contract.
Address code reuse of Citadel data structures and test utilities #36
Some data structs and code are duplicated between Citadel and Rusk, both production code and test code. It needs to be decided if and how to avoid such duplication. Also, some code is used in 'std' while some in 'no_std' environment. It needs to be decided what constitutes Struct implementation and what constitutes test utilities. Currently, some test utilities are written in a form of Struct implementations.
Closed issues
We need to implement a state module that handles the management of the Merkle tree. In the future, it will be integrated with the rest of the stack, but right now, we should be able to simply maintain a Merkle tree locally.
Allow custom Merkle tree sizes #38
The code should allow custom Merkle tree sizes, because the state is not required to be the same used in Phoenix.
Allow the user to craft the challenge #34
Right now the challenge c is hardcoded. We should allow the user to set it at wil.
Research on changing the tx_hash by the ssa #33
Citadel used to compute the signature of the transaction hash tx_hashwhere the license nullification is embedded. There were two reasons:
- Prevent malleability (the same transaction needed to append an NFT, thus everything needed to be signed).
- Prove that the user actually knows the license secret key.
As the session cookie is no longer included in the transaction, we can do better: the session stealth address ssa, which indeed needed to be included into the transaction and seen by everyone, will be signed using lsk and verified into the ZKP, instead of using tx_hash. Thus, it becomes a public input of the circuit.
The methods to use the license in the actual protocol need to be implemented according to the specs.
We need to implement the new structs defined in the specs:
- Request
- License
- LicenseProverParameters
- Session
- SessionCookie
Research on the elements included in the license and request #29
We need to rethink the values needed for license and request. They were designed thinking on Phoenix, but we agreed on having independent structs. So for instance, the element type is not needed anymore because Citadel must be private-by-default.
Research on improving the management of the SessionCookie #28
Right now, in step (5) of the protocol a SessionCookie is computed and sent to the network. This serves two purposes:
- Such a struct contains the openings to the commitments, to verify that the user computed the proof correctly.
- The nullifier is linked to a particular SP.
Actually, there is no need to include the openings in the transaction, as only the correct user knows them. Instead, adding only an identifier for the SP would do the trick. So, a SP's stealth address can link the transaction to that particular SP, and later on, the user provides the openings to verify the correctness of the proof. And, at the very same time, the SP is convinced that the user communicating with them is the one that nullified the license because only them could know the openings.
piecrust
Open issues
Allow for querying call tree #206
Multiple modules may be touched during the context of a call, forming a tree of inter-contract calls from one module to the next. It should be possible to query this tree.
Add module level documentation. Currently the modules do not give any sort of hint on what their functionality includes. For consumers of the libraries it would be nice if the majority of modules have at least a single line of commentary specifying what is contained within and what its purpose is.
Add documentation for piecrust::imports #195
Add documentation to the module imported functions in the imports module of piecrust.
Closed issues
Remove piecrust_uplink::State struct team:Core #209
The piecrust_uplink::State struct is not providing any additional functionality apart from functioning as a container for a contract's state.
Rename "module" to "contract" #207
We should replace the term module in favor of contract. While each contract is technically a WASM module, we should make it clear that we are speaking of smart contracts.
Moving a Session after module instantiation results in dangling pointers #202
If a module is instantiated within the scope of a Session - such as in the case of a deploy or a query/transact call - and the session is subsequently moved, the next call to a module in the context of that same session will produce a SIGSEGV.
This is due to the way in which we extend the lifetime of a Session during module instantiation in WrappedInstance::new
Owner should of arbitrary length or as large as a BLS public keys #201
Currently the owner of a contract is defined to be [u8; 32]. This should change to be either as large as a bls12_381_sign::PublicKey, or perhaps even arbitrary length with Vec<u8>.
This is required to use cryptographic keys as owners of contracts, allowing them to, in turn, be used to represent wallets.
wee_alloc is unmaintained #199
The wee_alloc crate is unmaintained and as such the dependency should be removed, possibly in favor of another allocator.
Add example module using dusk-merkle #197
To ensure we can represent Merkle trees in a module, we should create an example that uses the dusk-merkle crate to store a custom Merkle tree.
Alloc error handler removed #192
In the latest nightly build (rustc 1.71.0-nightly (f5559e338 2023-04-24) the alloc_error_handler attribute has been removed. handle_alloc_error defaults to panic now for no_std environments.
As a result, the CI breaks and divulges from the nightly specified in the repository itself.
wallet-cli
Open issues
Wrong network fallback to default #143
If you run the wallet-cli specifying a network that doesn't exist, the network used is the default one.
Closed issues
Notify the user if he tried to create more than 256 addresses for one wallet #92
We decided to support a maximum of 256 addresses for each wallet (seed), however there is no error message once the user reaches such a limit.
bls_381
Open issues
Scalar derives Hash but explicitly implements PartialEq #106
Scalar derives Hash but also has an explicit PartialEq implementation. Clippy doesn't like this because it wants the traits to agree on the implementation of a PartialEq.
Getting rid of our custom PartialEq implementation would mean losing the constant time equality.
Closed issues
The scalar module is private but some of the constants are being exposed in 'lib.rs'. I propose to expose the constant MODULUS as well.
Fix Cargo.toml invalid categories #75
When we publish the crate, cargo warns us about invalid categories specified. They should be fixed.
Update crate edition to 2021 #73
Update this crate from rust edition 2018 to 2021.
This also requires changing the CI workflow and rust-toolchain file using the same approach we are using in our recent repo.
Update canonical from v0.6 to v0.7 #71
Update canonical and canonical-derive from version 0.6 to 0.7
jubjub
Closed issues
Add tests for compute_windowed_naf #104
Add more unit tests for the compute_windowed_naf method for scalars. Test different scalars and window-sizes.
Dusk-blockchain
Closed issues
Fix extractCommitteeMember #1511
consensus
Open issues
Use constant committee size #60
Adapt to dusk-network/dusk-blockchain#1527
Closed issues
Incorporate get_candidate_block_by_hash call in first_reduction step #58
As per the implementation of GetCandidate procedure in dusk-network/rusk#889, get_candidate_block_by_hash should be now incorporated and used in the first reduction step.
get_candidate_block_by_hash should be called asynchronously as it may need to wait for a missing candidate block to be retrieved from network peers. Due to that MessageHandler::collect should be turned into an async method.
Make Database trait suitable for async calls #56
pub trait Database should be turned into async_trait to be used in async code properly.
Rusk
Open issues
Add Merkle tree support for the Citadel contract module:license-contract #899
Citadel contract is required to hold a Merkle tree instance in its state, and Merkle opening should be made available as an integral part of the license proof.
node: Create a complete synchronization procedure that can download and validate any missing blocks. #898
To complete this task, the following sub-tasks should be covered by studying the design and implementation of the sync-up procedure in the Golang implementation:
- Migrate Synchronizer component with a simple FSM (Finite State Machine)
- Implementing transitions from/to in-sync and out-of-sync states.
- Implementing sequencer to sort downloaded blocks.
- Send topics.GetBlocks message on transition to out-of-sync mode.
Given that the GetBlocks, GetInv, and GetData messages have already been implemented, a successful synchronization procedure for all blocks should be the result of resolving this issue.
Dependency loop with wallet-core #886
Currently, wallet-core depends or rusk-abi - which is in this repository. However, rusk - which is also in this repository - depends on wallet-core.
This generates a really nasty release loop, where we have to release rusk-abi before we might like to, in the hopes of merging PRs that target rusk.
Update to piecrust version 0.3 #883
This is necessary to allow the registration of contract owners, which in turn is necessary for implementing the fee market.
Contracts should be able to comp their caller's fee #882
In the context of the fee market we should allow the contract to comp the gas cost of running one of its functions.
Since someone should still pay the contract's execution, the owner of the contract must have a wallet that can be depleted to pay gas costs.
Allow contracts to levy transaction fees #881
In the context of the fee market implementation we should allow contracts to levy transaction fees.
These transaction fees should be defined either per contract - and changeable by its owner - or per function of the contract.
Implement a contract-driven fee market #880
Contracts should be able to choose how fees are paid for the execution of their code.
Closed issues
node: Use delay_on_resp_msg in testbed script #893
Together with adding delay_on_resp_msg in testbed script, this field should be the only one data_broker config read from CLI params.
Add support for all request-for-data messaging #889
The definition of any wire messages used in sync-up procedure, together with GetCandidate message flow should be added into node-data.
List of messages to be added:
- GetCandidate(payload::GetCandidate),
- GetMempool(payload::GetMempool),
- GetInv(payload::Inv),
- GetBlocks(payload::GetBlocks),
- GetData(payload::GetData),
- CandidateResp(payload::CandidateResp),
Limit the number of response messages in handling GetBlocks request #888
A new databroker section in config.toml should provide the max number of items a node can provide on GetBlocks request. Together with max_inv_entries, max_ongoing_requests and (optional) delay_on_resp_msg configs should be also supported.
Extend RocksDB API for ledger and mempool datasets to facilitate catch-up procedures #887
The API is needed in GetBlocks message handler to retrieve and provide missing blocks. Together with the new API, another Column Family should be added to facilitate the new index (height->block_hash).
Mempool::get_txs_hashes -> Vec
The API is needed in GetMempool message handler to retrieve all Mempool transaction hashes.
node: Implement DataBroker service to expose ledger data in Kadcast Network #872
DataBroker component should run as background tokio task handling following messages from Kadcast network.
- GetCandidate
- GetBlocks
- GetMempool
- Inv
- GetData
On occurrence of any of them we should spawn a tokio::task that makes an attempt to retrieve data and respond accordingly. As we spawn a task per request, we should also limit the number of ongoing requests with semaphore.
node: Enable collecting telemetry data of tokio runtime state #868
We should allow tokio_console to collect telemetry data of tokio runtime state.
node: Ensure consensus shutdown code is executed on aborting #866
On canceling consensus tasks due to an event of an accepted block, it turns out that child (tokio) tasks are not aborted. After investigating, it was found out that the act of aborting the main consensus task does not trigger the calls for aborting agreement and main loop tasks.
Currency inflation on fee charge and block generator reward #790
Charging fees to the transactor currently happens in the push_fee_crossover function which is called right before the last call to update_root. This counts the gas used and produces a note containing the remainder of the fee and appends it to the tree.
The awarding of fees to a block generator is done by counting the gas used in executing each series of transactions and awarding it to the stake contract.
The problem is these two numbers are not the same due to the fact that some code gets executed after the measurement of gas used in the contract. This means a transactor gets charged slightly less than the gas used to execute the transfer contract, while the block generator gets awarded the totality of the gas used.
This leads to currency inflation favoring the block generator, effectively incentivizing them to pick a large number of small value transactions to reap more of an inflationary reward.
As you can see, development is continuous and this list is not exhaustive. For example, minor issues and advancements concerning the Kadcast element have also been opened and resolved. The complete series of repositories can be found on GitHub.
Breakthrough developments will receive a separate spotlight, such as our latest deployment of Daylight. Thank you very much for your understanding and feedback as we continue to find the best way to provide the community with transparency and development information.
In A Nutshell: Release Cycle planning
Release Cycle development planning has been adopted by major companies including Google, Mozilla, and during the development of products such as Ubuntu, Kubernetes, and many more. The reason for this is clear: Release Cycle planning improves the predictability of software development for developers and the community alike.
For a more detailed explanation of the concept (along with frequently asked questions) please scroll down to the bottom of this article.
The below noted additions/changes to the repository can be followed on GitHub, for further details on their status and their function in the stack.
Kadcast v0.5.0
Added
- Add network blocklist implementation [#117]
Changed
Full Changelog: v0.4.1...v0.5.0
Wallet-cli v0.14.1
Fixed
- Add overflow-checks to release mode [#132]
Citadel v0.2.0
Added
- Add the specification for Citadel
- Add the implementation of the new structs
- Add the implementation of the new license methods
- Add the state module
Changed
- Allow the user to craft the challenge
Bls12_381 v0.11.3
Changed
- Update issue links in CHANGELOG [#84]
Fixed
piecrust v0.4.0
Added
- Add call and associated functions and structs [#136]
- Add dlmalloc feature [#199]
- Add crate-specific README [#174]
Changed
- Change module named functions and items to contract [#207]
- Change signature of owner to return [u8; N] instead of [u8; 32] [#201]
Removed
- Remove State struct [#209]
- Remove query and transact and associated functions and structs [#136]
- Remove wee_alloc feature [#199]
Added
Changed
- Change owner parameter type in ModuleData::builder to be [u8; N] [#201]
Fixed
- Fix SIGSEGV caused by moving sessions with instantiate modules [#202]
Removed
- Remove RawQuery/Transact re-rexports [#136]
- Remove Session::query/transact [#136]
- Remove query/transact imports [#136]
phoenix-core v0.19.0
Changed
- Remove enc, R and nonce from note hash [#123]
dusk-merkle v0.2.0
Added
- Add getters for all fields in Opening [#25]
- Add merkle tree implementation with poseidon hash and opening gadget [#29]
Changed
- Change opening branch to hold T instead of Option<T> [#25]
FAQ
We’ve included a small FAQ section below to make sure the community understands our intention with Release Cycles.
What is Release Cycle planning?
Release Cycle planning means that developmental updates are published at consistent intervals on GitHub; in the case of Dusk Network, every three weeks. These releases describe the latest additions, changes, fixes, and assets added to the tech stack by the development team.
Dusk Network currently has over 18+ active repositories on GitHub, each repository covering a different technical project. Release Cycle planning has not yet been applied to all repositories; the current focus on a single GitHub repository for clarity does not mean there are no ongoing developmental efforts occurring in other areas.
Does being featured in a Release mean Deployment?
The Release Cycle updates does not mean immediate deployments on our testnet. It is a release detailing additions, changes, and fixes to the repositories that we are ready to share with the public. All released content is considered stable, consistent, reviewed, and cross-checked with other repositories.
Are Release Cycle updates a spotlight for major deliverables?
No. The Release Cycle approach is strictly a way to provide a consistent shape for publication of our work in the clearest terms possible. They are not deadlines to be made, nor are they tied to specific development sprints. Release Cycle updates aim to raise attention with the community, and give proper coverage to, publishable GitHub releases. Major deliverables will be given their proper publication in dedicated articles.
[Read more about our Release Cycle planning here]
