‐
Luca Franceschini

Dusk Engineering Update: Aug + Sept

Dusk Engineering Update Aug-Sept

August / September

Introduction

Dusk’s engineering teams have been working relentlessly to bring mainnet closer to reality. The past few months have been particularly productive, as several key milestones have been achieved.

The state persistence has been switched to a delta-based representation, greatly improving the overall performance. This is a considerable improvement, as efficient state management is a vital element of a blockchain.

After a feature freeze and extensive testing, the new version of Nocturne has been rolled out, making sure that the network is fully prepared for mainnet,

Let’s take a closer look at these accomplishments and how they propel Dusk toward its mission.

Lunare DevNet

Dusk’s Devnet Lunare has been launched, providing a dedicated environment for testing new features before launching the updated Nocturne testnet. By leveraging Lunare as an innovation sandbox, engineers have been able to rapidly iterate and benchmark new functionalities in a partially controlled environment.

Nocturne Testnet

After the rapid improvements on Lunare, the updated Nocturne has been rolled out on 1st October. This launch allows nodes run from the community to join the network and further stress-test the new features, as well as allowing the engineering teams to cover edge case scenarios.

This is a significant milestone, as it brings Dusk closer to being fully equipped for the forthcoming mainnet launch.

Rusk

Introduction of Archive Node feature

Archive Nodes are nodes responsible for maintaining a comprehensive record of all transactions, blocks, and events ever recorded on the blockchain, from the genesis block. Archive Nodes are the only type of node that not only listen and broadcast events, but also store them.  Archive Nodes are a critical piece of infrastructure because they provide access to extensive historical data, and applications can query these nodes for detailed historical information, enabling complex analyses and queries. Archive Nodes are now available on Dusk, and can be easily run via the "archive" feature flag.

When running the node with:
DUSK_CONSENSUS_KEYS_PASS=password cargo r --release --features archive -p rusk -- -s /tmp/example.state, the archive feature is active and starts archiving contract events from the vm and stores all ContractTxEvent events from a block in a json serialized column.

Events

Added Bloom Filters

We’ve replaced an event hash for each block with a bloom filter. By storing the bloom filter of events instead of a hash, verifying that an event is included in a block becomes significantly more efficient. This is because it's no longer necessary to provide every event to generate the root hash for all events for a given block, instead we can prove inclusion by applying the bloom filter. This simplifies querying whether a specific event was emitted for a given contract in a block, as it eliminates the need to hash and compare all the data.

Enriching Contracts with Additional Events

While stake-related events are crucial, the impact of adding additional event types to contract execution needs to be assessed taking into account the overhead it introduces.

Provisioners, whose primary focus is achieving consensus, have no practical use for events. Their role requires minimizing interaction with clients to maintain efficiency. However, it makes sense for Archive Nodes to create events locally when accepting transactions like those from Moonlight or Phoenix. These events can be customized depending on the client’s needs, maintaining convenience through the Archive Nodes’ mediation.

Staking events, however, remain necessary because Archive Nodes may struggle to capture state changes triggered by transactions not initiated by users, such as contract function calls related to consensus operations (e.g. slashing events). Without these stake events, tracking critical state transitions would be challenging.

Though emitting events during contract execution incurs additional gas costs and overhead, this trade-off is justified for important transactions like those in the Moonlight. This approach ensures necessary events are emitted during contract execution, while avoiding excessive overhead for other event types.

Below, you can find the events that enriched the stake-contract:

Key Stake Events:

  • Stake: Emit an event capturing the account that staked and the staked amount.
  • Unstake: Omit the withdraw receiver, depending on the transfer contract to signal the mint.
  • Withdraw: Similar to un-stake, rely on the transfer contract to handle the mint.
  • Reward: Modify the reward function to accept a vector of keys and values, allowing a single consolidated event. Add a reason for the reward, likely a byte, to streamline this process.
  • Slash & Hard Slash: Merge the suspended and slash events into one. Similarly, consolidate the suspended and hard_slash events for efficiency.

The transfer-contract also witnessed an enrichment in the events emitted, as the following events are emitted when their corresponding functions execute successfully:

  • Mint
  • Withdraw
  • Convert.
  • Deposit
  • Transfer to Contract
  • Transfer to Account

Additionally, an event is emitted at the conclusion of every transaction ingestion. This event will either be a PhoenixTransactionEvent or a MoonlightTransactionEvent, depending on the transaction model used. These events are emitted during the host's call to refund, ensuring that the event data captures both the changes made and the gas spent during the transaction.

As a result of these updates, minor adjustments were required in how notes are inserted into the tree.

RUES Events Dispatch

Previously, the HTTP system and RUES were mixed together in the codebase. By migrating old HTTP events over to RUES, events have now been standardized. The Dusk node now has a truly unified API interface, with this new API being used in both Dusk wallets and explorer.

Added tx ID to Contract Events

To improve the information available to Archive Nodes when events are dispatched via RUES, a new field has been added as a rusk-origin header in the response. This enhancement ensures backward compatibility by not altering the event data, allowing for smooth deserialization.

Importantly, this change does not impact the performance of the VM, as the tx_id is enriched at the node level, rather than during VM execution.

Consolidating Reward Calls

By consolidating reward calls, performance is increased and the number of emitted events is minimized, which positively impacts the storage of Archive Nodes.

Contracts and Transactions

Multisig Support in stake-contract

Previously, the only keys required to run a node were the "provisioner keys" (BLS keys), stored directly on the node's machine. If the node was compromised, these keys could have been misused to transfer stakes and rewards to an unauthorized address.

With the latest update, "provisioner keys" are now strictly limited to signing consensus messages. Any other critical operations, such as un-staking or withdrawing funds now require a multi-signature process involving an additional key, called the "funds key," which is registered during the staking process.

Transaction Memos

Memos are now available in both Moonlight and Phoenix transactions, and can be used for different purposes. (e.g. exchanges to be able to target internal accounts whilst using a single key/address for receiving deposits). With this new memo feature, memo transactions can carry up to 512 bytes of information.

With the addition of memos, Dusk transactions now allow for 4 different transaction types: Regular transfers, contract calls, contract deploys and transfers with memos.

Single-key signature in contracts

Rusk now uses a single-key signature scheme in contracts, leveraging an improved API for both Moonlight and the stake-contract. This update allows the consensus, node, and node-data crates to use the multi-signature scheme more explicitly than before.

Contract-to-Contract Transfers

Transfers between contracts are now achieved via the transfer_to_contract function in the transfer-contract. This function allows one contract to transfer Dusk to another contract, while the receiving contract can choose to accept or reject the transfer based on the accompanying data. This allows transfers to be based on arbitrary data provided by the sender and enables explicit acceptance for contracts.

Owner-Exclusive Contract Methods

Certain contract methods can be now guarded so that only the contract’s owner can execute them. If the contract owner has a valid BLS public key, selected methods can be protected, ensuring that only authorized calls are executed.

Contracts can send funds to Moonlight

Contracts can now send DUSK directly to Moonlight accounts, enabling several use cases (e.g. bridging between blockchains). This is done via the transfer_to_account function, which can chain multiple calls to transfer DUSK to several accounts atomically. 

Added Moonlight Transactions in rusk-wallet 

As part of ongoing efforts to enhance the functionality and user experience of the Rusk wallet, several key updates are being introduced. These updates aim to improve interaction with 3rd-party contracts, streamline staking functions, and enhance the wallet’s API integration. 

rusk-wallet has been improved by adding direct support for staking, un-staking, and rewards withdrawal from Moonlight, and these features are now available as commands within the wallet, making these activities much more streamlined for users.

Contract Deployment and Execution

The wallet now supports smart contract deployment and execution, making it easier for users  to deploy and interact with 3rd-party contracts directly from the Rusk wallet. 

This improvement brings comprehensive smart contract functionality directly to the Rusk wallet.

Sync Method Update

A sync method has been implemented in the transfer-contract, including an optional argument to specify the maximum number of leaves to return. This feature enhances efficiency by enabling partial data retrieval and improving performance.

Added verify_groth16 host function

The groth16 host function allows for generating and verifying groth16 proofs.

The introduction of the verify_groth16 function to the rusk-abi allows contracts to delegate proof verification to the host. As a result, the host now supports two proof systems: Groth16 and PlonK.

Consensus

Improvements in messages propagation

The message propagation and tracing within the system has been improved, and the following enhancements have been added:

  • Repropagating messages from past iterations during the current round
  • Repropagating future messages if signed by eligible provisioners
  • Ensuring Candidate messages are repropagated before the Proposal delay. 

Blocks and Gas

Configurable Block Gas Limit

A configuration option for the block gas limit has been added with a default value of DEFAULT_BLOCK_GAS_LIMIT. The ability to configure the block gas limit allows the network to adapt to varying workloads and optimize performance, preventing unnecessary bottlenecks during peak usage.

Introduced Block Size Limit

Block Size has been limited to  1Mb, which is roughly equivalent to 250 Phoenix transactions. Introducing a block size limit helps prevent certain types of attacks.

Improved Deployment Pricing

Smart contract deployment pricing has been adjusted to prevent spam by ensuring the deployment cost is significant enough to deter misuse.

Refactoring and API improvements

Transition from the HTTP API to RUES

As the HTTP API is being phased out, the rusk-wallet has transitioned to the RUES, replacing the old API with a more modern and efficient system. This change enhances the wallet’s overall performance and prepares it for future upgrades. The previous HTTP client handler has been swapped out for RUES dispatch calls, ensuring the wallet now operates with improved speed, security, and scalability, while remaining compatible with upcoming changes in the Rusk ecosystem.

Migration of wallet-cli and wallet-core to Rusk

The wallet-cli has been relocated from its previous repository to a new workspace within Rusk, called rusk-wallet, for the following reasons

  • It enables the wallet to become a rusk subcommand
  • The wallet-cli provides an essential entry point to the node

Similarly, the wallet-core library has been now integrated into the Rusk workspace, streamlining operations and improving efficiency.

Rusk Versioning 

A version labeling system has been introduced, allowing different node versions to co-exist in the network. This involves releasing binaries tied to specific commits and managing data and peers using different protocol versions.

Additional Improvements

Few additional improvements have been:

- Refactoring of the stake-contract

- Refactoring of LongLivedService

-Refactoring of CLI subcommands

- zk types are re-exposed under the zk-feature

- Added support for configurable HTTP headers

Execution-core

Prove Trait Error Handling

The prove function in execution-core's Prover trait now returns an error when a faulty circuit or trait implementation occurs. Adding an associated type to the Prove trait allows more flexibility in defining the type of error returned, making the trait less restricted. This allows the program to gracefully terminate in such cases. 

Web-wallet

New Web-Wallet Components

The web-wallet team has introduced several new components to enhance the UI:

  • A component for selecting one option from multiple choice
  • A UsageIndicator component.
  • A Navigation component.

Token Migration Bindings

The integration of the DUSK migration contract for BEP20 and ERC20 tokens required creating an ABI for the migration function on the DUSK migration contract. This creation of it enables spending DUSK on behalf of the migration contract. For this reason, new bindings were created, as well as wrapper functions for various ERC20 contract calls.

Improved Wallet Creation & Sync

Wallet creation and synchronization, traditionally time-consuming due to the blockchain size, have been improved. Users can now choose to sync their wallet from the genesis block or a specific block height, while also being able to rely on a sync indicator to track progress.

Support for EVM interaction

EVM-compatible wallet support has been added, which will allow users to migrate their DUSK tokens from EVM chains to native DUSK without external apps or manual input. The web-wallet team also created an interface for holders of DUSK on non-native chains to convert to native DUSK tokens. 

Wallet-core

Added Moonlight <> Phoenix Conversion

Two functions have been added in wallet-core to enable the conversion between Moonlight and Phoenix DUSK. This functionality enables a smooth conversion between the two different ways of representing native DUSK.

Phoenix Notes Filtering

To improve wallet performance and user experience, wallet-core can now filter notes based on ownership. The wallet identifies which notes belong to a specific key, generates corresponding nullifiers, and returns a dictionary mapping nullifiers to their respective notes. This feature accepts a range of indexes to handle multiple keys generated from the same seed, streamlining the process by avoiding unnecessary iterations.

This functionality improves wallet performance by focusing only on relevant data, ensuring faster processing times, especially for users with many notes.

Delegating Proof Generation

wallet-core now supports the delegation of proof generation to an external Prover for both Phoenix and Moonlight,

By delegating proof generation, users can offload complex computations to external Provers (while avoiding malleability).

Block Height in Owned Notes

To improve synchronization, each note stored in map_owned is mapped to a key that now also includes block height alongside the nullifiers.

FFI Function for User Profile

Introducing the FFI function for user profiles allows for smoother integration between external applications and the wallet, improving user experience by making key account information easily accessible. A user profile is the combination of a Phoenix's address and the Moonlight's account, and the FFI function is exposed by wallet-core to generate a user profile from a seed and an index. 

wallet-Core WASM Endpoint

Archive Nodes can now serve the WASM of the wallet-core so that consumers can dynamically fetch and use it (e.g. via the /drivers/ pathname). The build process triggers the wallet-core WASM build and includes the result as include_bytes in the executable, serving it via HTTP.

Introduced FFI in wallet-core

Debugging wallet-core in a WASM environment is challenging, particularly due to the abort-on-panic behavior and the optimizations in release builds. For this reason, a mechanism that enables effective debugging for both WASM and native targets has been introduced, ensuring that debugging code can be automatically stripped out in production builds to avoid any performance or size impact on the generated module.

In wallet-core the FFI also enables the calculation of a balance for a specific address based on a provided list of notes. This feature is essential to allow wallet-core to function as a WASM module, enabling external systems to interface with the core logic and perform balance computations for individual addresses.

By providing this functionality, wallet-core can better integrate with external platforms, streamlining processes that rely on accurate balance calculations for various addresses.

W3sper SDK

The W3sper SDK witnessed considerable improvements, and it’s approaching its final stages.

dusk-deploy-cli

The dusk-deploy-cli tool that allows for deployment of 3rd party contracts on Dusk has been released. Developers that want to deploy smart contracts on Dusk only need a seed phrase, a contract to be deployed, a small configuration file, and a hex string representing the contract's owner. 

Piecrust

Changed state representation to delta representation

Effective state management is essential to ensuring the blockchain can handle growth without degrading performance.

The overall state of the Dusk blockchain is divided into smaller data segments, or "pages," that store essential information like user balances, contract details, and transaction logs. This division allows the system to manage the state efficiently by treating it as a collection of smaller units rather than one large entity. Each page contains a portion of the data, such as contract details or ledger information, optimizing storage and retrieval.

Previously, commits were made independent by hard-linking unchanged pages from earlier commits. However, this approach caused commit times to scale with the total state size (O(S)), leading to inefficiencies as the state grew larger. To solve this, the state persistence was switched to a delta-based representation, improving commit performance. Now, only the changes are stored, with the rest being referenced from the base or main commit, significantly speeding up the process.

When a state update occurs, only the modified pages are now rewritten, while unchanged pages remain linked from previous states. This process enhances both storage efficiency and performance by avoiding unnecessary rewrites.

Add additional "raw" calls

Users can perform their own custom serialization in contracts by using the new emit_raw and feed_raw "raw" calls in piecrust-uplink (similarly to call and call_raw).

Block Explorer

The block explorer witnessed enhancements, including the optimization of auto re-renders and an improved displaying of the gas limit and gas spent values.

Ecosystem

Audits

The audits of the Consensus, Economic Protocol, Rusk node library and Phoenix 2.0 have been completed and can be found here.

Liquid Staking

Liquid Staking is planned to be available on Dusk in Q4, The solution is being developed by Sozu, and you can know more by joining their telegram,

Open RFPs

There are currently two open RFPs waiting for proposals:

Applications can be submitted via the Thesan application form.
 

Resources
Dusk Engineering Update July

Dusk Engineering Update: July

By Luca Franceschini
Dusk June Engineering Update

Dusk Engineering Update: June

By Luca Franceschini