Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c2343edc6b | ||
|
|
eeb5dd4141 | ||
|
|
64a81e4110 | ||
|
|
c5538e35bb | ||
|
|
856779e0bc | ||
|
|
46683c4e21 | ||
|
|
e17155ecdb | ||
|
|
b4e723fd2c | ||
|
|
73f3bca656 | ||
|
|
ae908ee23d |
+4
-3
@@ -1,5 +1,6 @@
|
||||
onchain/cache/
|
||||
onchain/out/
|
||||
out/
|
||||
cache/
|
||||
lib/forge-std
|
||||
|
||||
offchain/target/
|
||||
|
||||
.direnv
|
||||
@@ -1,47 +0,0 @@
|
||||
me: Bounty
|
||||
about: Bounty
|
||||
title: ''
|
||||
labels: bounty
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
# Bounty
|
||||
|
||||
## Scope
|
||||
<!-- A list of specific things which should be done to deliver the bounty. These could be seen as requirements to verify/review bounty against -->
|
||||
|
||||
-
|
||||
-
|
||||
-
|
||||
|
||||
## Deliverables
|
||||
<!-- Artifacts produced as the result of this bounty. Something that could be verified/reviewed. Some examples: updated code, deployment made, blog post published, public event conducted etc -->
|
||||
|
||||
-
|
||||
-
|
||||
|
||||
## Bounty Owner/Gardener
|
||||
<!-- The Role who is responsible for the bounty. -->
|
||||
<@your github username> as **<Role>**
|
||||
|
||||
## Gain for the Role
|
||||
<!-- How the completion of this bounty helps to pursue the Role's purpose -->
|
||||
|
||||
## Roles
|
||||
bounty gardener: <@your git username> / X DAI <!-- likely shouldn't be a % but rather flat -->
|
||||
bounty worker: name / share
|
||||
bounty reviewer: name / share
|
||||
|
||||
## Gardener checklist
|
||||
|
||||
- [ ] I have attached one of the `size-` labels to this bounty
|
||||
|
||||
<!--
|
||||
# Bounty sizes
|
||||
XS / 200 DAI / effort ~ 3h
|
||||
S / 350 DAI / effort ~5h
|
||||
M / 550 DAI / effort ~8h
|
||||
L / 900 DAI / effort ~13h
|
||||
XL / 1400 DAI / effort ~21h
|
||||
-->
|
||||
+124
@@ -0,0 +1,124 @@
|
||||
# Interpreter.md
|
||||
|
||||
## Overview
|
||||
|
||||
This repository provides a **Push3-like interpreter** implemented in Solidity. It uses a _token-based bytecode_ approach to parse programs from a `bytes` array (the `code`) into typed **descriptors**, which are then executed by the interpreter.
|
||||
|
||||
**Push3** is a stack-based language originally designed for genetic programming and autoconstructive evolution. Each data type (e.g. `INT_LITERAL`, `FLOAT`, `BOOLEAN`, etc.) has its own stack. The **`EXEC`** stack manages the flow of execution.
|
||||
|
||||
**In this implementation**:
|
||||
- We store code in a single `bytes` array (`code`).
|
||||
- We parse sublists and integer literals from that array.
|
||||
- We push instructions and literals onto an execution stack.
|
||||
- We pop them in LIFO order and execute them.
|
||||
|
||||
### Key Features
|
||||
|
||||
1. **Token-Based Parsing**: We define small tokens (`0x00` => NOOP, `0x01` => INTEGER_PLUS, `0x02` => INT_LITERAL, `0x03` => SUBLIST). If we see `INT_LITERAL`, we read 4 bytes for a 32-bit integer; if we see `SUBLIST`, we read 2 bytes for the sublist length, then parse it recursively.
|
||||
2. **Descriptor System**: We represent instructions, literals, and sublists using 256-bit “descriptors.” Each descriptor encodes:
|
||||
- A `tag` (top 8 bits): e.g. `INT_LITERAL`, `INSTRUCTION`, `SUBLIST`.
|
||||
- `offset` and `length` for sublists.
|
||||
- A leftover field (e.g. for storing integer data).
|
||||
3. **Reversing Items**: When we pop a sublist descriptor, we parse it into descriptors and then push them onto the `EXEC` stack in reverse order. This ensures the list’s leftmost token executes first, consistent with the Push3 spec.
|
||||
4. **Separation of Parsing and Execution**:
|
||||
- **Parsing**: Transforms raw byte tokens into typed descriptors (instructions, literals, or nested sublists).
|
||||
- **Execution**: The `runInterpreter` function repeatedly pops from `EXEC`, decides if it’s an instruction or literal or sublist, and acts accordingly.
|
||||
|
||||
## Code Structure
|
||||
|
||||
### 1. **Enums**
|
||||
|
||||
- **`CodeTag`**:
|
||||
- `NO_TAG` (unused),
|
||||
- `INSTRUCTION`,
|
||||
- `INT_LITERAL`,
|
||||
- `SUBLIST`.
|
||||
|
||||
- **`OpCode`**:
|
||||
- `NOOP`,
|
||||
- `INTEGER_PLUS`.
|
||||
|
||||
### 2. **Descriptor Layout**
|
||||
|
||||
A **256-bit** word has:
|
||||
```
|
||||
[ top 8 bits | offset(32 bits) | length(32 bits) | leftover(184 bits) ]
|
||||
```
|
||||
- `tag`: The top 8 bits (e.g. `INT_LITERAL`).
|
||||
- `offset, length`: If `tag = SUBLIST`, these define which bytes in `code` represent the sub-sublists.
|
||||
- `leftover`: If `tag = INT_LITERAL`, we store the 32-bit integer in the bottom 32 bits of leftover. If `tag = INSTRUCTION`, we store the opcode (`NOOP`, `INTEGER_PLUS`) in the lowest 8 bits of leftover.
|
||||
|
||||
### 3. **Helper Reads**
|
||||
|
||||
- **`readUint32(code, start)`**: Copies 4 bytes from calldata into a small buffer, returns the final `uint32`. By shifting bits if needed, we ensure we get the integer from the correct part of the loaded word.
|
||||
- **`readUint16(code, start)`**: Similar logic for 2 bytes.
|
||||
|
||||
### 4. **`parseSublist` Function**
|
||||
|
||||
When we see a sublist descriptor (`tag=SUBLIST`), we parse the slice `[off..off+len]` in the `bytes code`. We iterate token by token:
|
||||
- `0x00` => `NOOP`
|
||||
- `0x01` => `INTEGER_PLUS`
|
||||
- `0x02` => read 4 bytes => `INT_LITERAL`
|
||||
- `0x03` => read 2 bytes => sub-sublist length => build a sub-descriptor
|
||||
|
||||
We accumulate these descriptors in an array. Then, later, we push them _in reverse_ onto the `EXEC` stack.
|
||||
|
||||
### 5. **`runInterpreter` Function**
|
||||
|
||||
1. Loads initial CODE, EXEC, and INT stacks from the provided arrays.
|
||||
2. While `EXEC` is non-empty:
|
||||
- Pop the top descriptor.
|
||||
- If `INSTRUCTION`, decode the opcode (e.g. `INTEGER_PLUS`) and do it (pop top 2 ints, push their sum).
|
||||
- If `INT_LITERAL`, push that integer onto the int stack.
|
||||
- If `SUBLIST`, parse that sublist into descriptors, then push those in reverse.
|
||||
|
||||
## How to Contribute
|
||||
|
||||
1. **Add New Instructions**
|
||||
- Expand `OpCode` enum (e.g. `INTEGER_MINUS`, `INTEGER_MUL`, etc.).
|
||||
- Write a `makeInstruction` or `makeIntLiteral` function to build the descriptor for it.
|
||||
- In the main loop, implement the logic (pop the correct number of integers, do the operation, push the result).
|
||||
|
||||
2. **Support More Data Types**
|
||||
- For instance, a `FLOAT` or `BOOLEAN` stack.
|
||||
- Add a new tag (`FLOAT_LITERAL`?), new instructions (`FLOAT.PLUS`?), etc.
|
||||
|
||||
3. **Improve Parsing**
|
||||
- If you want 256-bit literals, define a new token that reads 32 bytes, or do other expansions (like `CODE` type).
|
||||
- Or define advanced sublist structures (like nested `( ( ) )` expansions).
|
||||
|
||||
4. **Gas Optimization**
|
||||
- Check how large arrays are allocated (like in `parseSublist`).
|
||||
- Possibly implement step-by-step execution if large code exceeds block gas limits.
|
||||
|
||||
5. **Add Logging & Debug Tools**
|
||||
- Use `console.log(...)` to trace tokens, track final states, or measure performance.
|
||||
|
||||
6. **Test Coverage**
|
||||
- Provide more unit tests in Foundry for advanced instructions, nested sublists, edge cases with zero-length sublists, etc.
|
||||
|
||||
## Next Steps
|
||||
|
||||
### A. Extending the VM
|
||||
|
||||
A “full” Push3 VM might include:
|
||||
- **Multiple Data Stacks**: `BOOLEAN`, `FLOAT`, `NAME`, etc.
|
||||
- **NAME Binding**: Let `NAME`s reference values or code.
|
||||
- **Control Structures**: `EXEC.IF`, `EXEC.DO*COUNT`, or combinators like `EXEC.Y`.
|
||||
|
||||
### B. Example Project
|
||||
|
||||
You could use this interpreter in:
|
||||
- A genetic programming environment, where random code is generated and mutated.
|
||||
- A puzzle solver that runs tiny “scripts” in EVM-like constraints.
|
||||
|
||||
## Conclusion
|
||||
|
||||
This `Push3Interpreter` is a **minimal** scaffold for a token-based, list-expanding interpreter with typed descriptors. By following the patterns for instructions, sublists, and typed stacks, you can easily expand it into a more complete VM. We welcome contributions in the form of new instructions, data types, and performance optimizations.
|
||||
|
||||
1. **Clone** this repo.
|
||||
2. **Check** the `test/` folder for Foundry-based tests.
|
||||
3. **Add** or modify instructions & sublists in `parseSublist` and `runInterpreter`.
|
||||
4. **Run** `forge test --via-ir -vv` to see logs and confirm your changes.
|
||||
|
||||
Enjoy exploring the **Push3** style VM in Solidity!
|
||||
@@ -1,14 +1,6 @@
|
||||
# **SoLUSH**: A Push 3 Virtual Machine in Solidity
|
||||
|
||||
Welcome to **SoLUSH 3** (pronounced like **"slush"** without the "o", composed from **S**o**L**idity and p**USH** 3), a **Push3-like interpreter** implemented in Solidity. It uses a _token-based bytecode_ approach to parse programs from a `bytes` array (the `code`) into typed **descriptors**, which are then executed by the interpreter.
|
||||
|
||||
**Push3** is a stack-based language originally designed for genetic programming and autoconstructive evolution. Each data type (e.g. `INT_LITERAL`, `FLOAT`, `BOOLEAN`, etc.) has its own stack. The **`EXEC`** stack manages the flow of execution.
|
||||
|
||||
**In this implementation**:
|
||||
- We store code in a single `bytes` array (`code`).
|
||||
- We parse sublists and integer literals from that array.
|
||||
- We push instructions and literals onto an execution stack.
|
||||
- We pop them in LIFO order and execute them.
|
||||
Welcome to **SoLUSH 3** (pronounced like **"slush"** without the "o", composed from **S**o**L**idity and p**USH** 3), a Solidity implementation of the [Push 3 stack machine](http://faculty.hampshire.edu/lspector/push3-description.html), designed to bring evolved genetic algorithms on-chain. By enabling the execution of Push 3 programs directly within Ethereum smart contracts, SoLush 3 paves the way for decentralized, adaptive systems to address complex optimization problems.
|
||||
|
||||
## 🌟 Vision
|
||||
|
||||
@@ -20,97 +12,19 @@ SoLUSH is a key step toward realizing the vision of running decentralized evolut
|
||||
|
||||
To understand the broader vision, check out the [Vision Document](vision.md).
|
||||
|
||||
### Key Features
|
||||
|
||||
1. **Token-Based Parsing**: We define small tokens (`0x00` => NOOP, `0x01` => INTEGER_PLUS, `0x02` => INT_LITERAL, `0x03` => SUBLIST). If we see `INT_LITERAL`, we read 4 bytes for a 32-bit integer; if we see `SUBLIST`, we read 2 bytes for the sublist length, then parse it recursively.
|
||||
2. **Descriptor System**: We represent instructions, literals, and sublists using 256-bit “descriptors.” Each descriptor encodes:
|
||||
- A `tag` (top 8 bits): e.g. `INT_LITERAL`, `INSTRUCTION`, `SUBLIST`.
|
||||
- `offset` and `length` for sublists.
|
||||
- A leftover field (e.g. for storing integer data).
|
||||
3. **Reversing Items**: When we pop a sublist descriptor, we parse it into descriptors and then push them onto the `EXEC` stack in reverse order. This ensures the list’s leftmost token executes first, consistent with the Push3 spec.
|
||||
4. **Separation of Parsing and Execution**:
|
||||
- **Parsing**: Transforms raw byte tokens into typed descriptors (instructions, literals, or nested sublists).
|
||||
- **Execution**: The `runInterpreter` function repeatedly pops from `EXEC`, decides if it’s an instruction or literal or sublist, and acts accordingly.
|
||||
|
||||
## Code Structure
|
||||
|
||||
### 1. **Enums**
|
||||
|
||||
- **`CodeTag`**:
|
||||
- `NO_TAG` (unused),
|
||||
- `INSTRUCTION`,
|
||||
- `INT_LITERAL`,
|
||||
- `SUBLIST`.
|
||||
|
||||
- **`OpCode`**:
|
||||
- `NOOP`,
|
||||
- `INTEGER_PLUS`.
|
||||
|
||||
### 2. **Descriptor Layout**
|
||||
|
||||
A **256-bit** word has:
|
||||
```
|
||||
[ top 8 bits | offset(32 bits) | length(32 bits) | leftover(184 bits) ]
|
||||
```
|
||||
- `tag`: The top 8 bits (e.g. `INT_LITERAL`).
|
||||
- `offset, length`: If `tag = SUBLIST`, these define which bytes in `code` represent the sub-sublists.
|
||||
- `leftover`: If `tag = INT_LITERAL`, we store the 32-bit integer in the bottom 32 bits of leftover. If `tag = INSTRUCTION`, we store the opcode (`NOOP`, `INTEGER_PLUS`) in the lowest 8 bits of leftover.
|
||||
|
||||
### 3. **Helper Reads**
|
||||
|
||||
- **`readUint32(code, start)`**: Copies 4 bytes from calldata into a small buffer, returns the final `uint32`. By shifting bits if needed, we ensure we get the integer from the correct part of the loaded word.
|
||||
- **`readUint16(code, start)`**: Similar logic for 2 bytes.
|
||||
|
||||
### 4. **`parseSublist` Function**
|
||||
|
||||
When we see a sublist descriptor (`tag=SUBLIST`), we parse the slice `[off..off+len]` in the `bytes code`. We iterate token by token:
|
||||
- `0x00` => `NOOP`
|
||||
- `0x01` => `INTEGER_PLUS`
|
||||
- `0x02` => read 4 bytes => `INT_LITERAL`
|
||||
- `0x03` => read 2 bytes => sub-sublist length => build a sub-descriptor
|
||||
|
||||
We accumulate these descriptors in an array. Then, later, we push them _in reverse_ onto the `EXEC` stack.
|
||||
|
||||
### 5. **`runInterpreter` Function**
|
||||
|
||||
1. Loads initial CODE, EXEC, and INT stacks from the provided arrays.
|
||||
2. While `EXEC` is non-empty:
|
||||
- Pop the top descriptor.
|
||||
- If `INSTRUCTION`, decode the opcode (e.g. `INTEGER_PLUS`) and do it (pop top 2 ints, push their sum).
|
||||
- If `INT_LITERAL`, push that integer onto the int stack.
|
||||
- If `SUBLIST`, parse that sublist into descriptors, then push those in reverse.
|
||||
|
||||
|
||||
|
||||
## Next Steps
|
||||
|
||||
### A. Extending the VM
|
||||
|
||||
A “full” Push3 VM might include:
|
||||
- **Multiple Data Stacks**: `BOOLEAN`, `FLOAT`, `NAME`, etc.
|
||||
- **NAME Binding**: Let `NAME`s reference values or code.
|
||||
- **Control Structures**: `EXEC.IF`, `EXEC.DO*COUNT`, or combinators like `EXEC.Y`.
|
||||
|
||||
### B. Example Project
|
||||
|
||||
You could use this interpreter in:
|
||||
- A genetic programming environment, where random code is generated and mutated.
|
||||
- A puzzle solver that runs tiny “scripts” in EVM-like constraints.
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
This `Push3Interpreter` is a **minimal** scaffold for a token-based, list-expanding interpreter with typed descriptors. By following the patterns for instructions, sublists, and typed stacks, you can easily expand it into a more complete VM. We welcome contributions in the form of new instructions, data types, and performance optimizations.
|
||||
SoLUSH 3 is in active development. To contribute or test the functionality, follow these steps:
|
||||
|
||||
1. **Clone** this repo.
|
||||
2. **Check** the `test/` folder for Foundry-based tests.
|
||||
3. **Add** or modify instructions & sublists in `parseSublist` and `runInterpreter`.
|
||||
4. **Run** `forge test --via-ir -vv` to see logs and confirm your changes.
|
||||
1. Clone the repository:
|
||||
```bash
|
||||
git clone https://github.com/your-username/SoLUSH.git
|
||||
cd SoLUSH
|
||||
```
|
||||
2. Install dependencies (if applicable).
|
||||
3. Review the examples and test cases to understand how Push 3 programs are executed on the VM.
|
||||
|
||||
|
||||
### How to contribute
|
||||
|
||||
Check the [Issues](../../../issues) section to find tasks to tackle.
|
||||
|
||||
## 🛠️ Roadmap
|
||||
|
||||
- **Core Features:**
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 113 KiB After Width: | Height: | Size: 92 KiB |
@@ -23,6 +23,9 @@
|
||||
|
||||
# ... any other dependencies we need
|
||||
solc
|
||||
cargo
|
||||
rustc
|
||||
rust-analyzer
|
||||
];
|
||||
|
||||
# Decorative prompt override so we know when we're in a dev shell
|
||||
|
||||
Generated
+4631
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,34 @@
|
||||
[package]
|
||||
name = "offchain"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
[[bin]]
|
||||
name = "gp_tester"
|
||||
path = "src/bin/gp_tester.rs"
|
||||
|
||||
[[bin]]
|
||||
name = "gp_spec_tester"
|
||||
path = "src/bin/gp_spec_tester.rs"
|
||||
|
||||
[[bin]]
|
||||
name = "symreg_experiment"
|
||||
path = "src/bin/symreg_experiment.rs"
|
||||
|
||||
[[bin]]
|
||||
name = "symreg_experiment_local"
|
||||
path = "src/bin/symreg_experiment_local.rs"
|
||||
|
||||
[dependencies]
|
||||
serde = { version = "1.0", features = ["derive"] }
|
||||
serde_json = "1.0"
|
||||
rand = "0.8"
|
||||
hex = "0.4"
|
||||
# ethers = { version = "2.0", features = ["abi"] }
|
||||
anyhow = "1.0"
|
||||
revm = { git = "https://github.com/bluealloy/revm.git", rev = "03ea24bd9550e93b08f343e0b410725557c1adb4", features = ["std"] }
|
||||
database = { git = "https://github.com/bluealloy/revm.git", rev = "03ea24bd9550e93b08f343e0b410725557c1adb4", package = "revm-database" }
|
||||
ethers = "2.0"
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
use anyhow::Result;
|
||||
use rand::thread_rng;
|
||||
use rand::Rng;
|
||||
|
||||
// Suppose you have these in your library:
|
||||
use offchain::gp::generate_spec::ranmdom_code_fixed;
|
||||
use offchain::gp::mutation::mutate_by_index;
|
||||
use offchain::compiler::ast::UntypedAst;
|
||||
|
||||
fn main() -> Result<()> {
|
||||
let mut rng = thread_rng();
|
||||
|
||||
// We'll produce, say, 3 random ASTs ("subjects").
|
||||
let num_subjects = 3;
|
||||
let max_points = 6;
|
||||
|
||||
for i in 0..num_subjects {
|
||||
// Generate a random AST
|
||||
let original = ranmdom_code_fixed(&mut rng, max_points);
|
||||
println!("=== Subject #{} ===", i);
|
||||
println!("Original AST:\n{:#?}", original);
|
||||
|
||||
// Let's do 2 successive mutations on it
|
||||
let mut prev = original.clone();
|
||||
for mut_idx in 0..2 {
|
||||
let mutated = mutate_by_index(&prev, &mut rng, max_points);
|
||||
println!("Mutation #{} result:\n{:#?}", mut_idx, mutated);
|
||||
prev = mutated; // so the next iteration mutates from the last result
|
||||
}
|
||||
|
||||
println!("=== End of Subject #{} ===\n", i);
|
||||
}
|
||||
|
||||
println!("Done generating & mutating random ASTs!");
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
// src/bin/gp_tester.rs
|
||||
|
||||
use anyhow::{anyhow, bail, Result};
|
||||
use rand::{thread_rng, Rng};
|
||||
|
||||
// 1) We'll use the helper that reads creation code from a JSON artifact
|
||||
use offchain::helpers::artifact::get_creation_code;
|
||||
|
||||
// 2) EvmRunner to deploy & run the interpreter
|
||||
use offchain::runner::revm_runner::EvmRunner;
|
||||
|
||||
// 3) AST generation & mutation
|
||||
use offchain::gp::generate::random_ast;
|
||||
use offchain::gp::genetic_ops::mutated_ast;
|
||||
use offchain::compiler::ast::UntypedAst;
|
||||
|
||||
fn main() -> Result<()> {
|
||||
// ----------------------------------------------------------------------
|
||||
// 1) Read the creation code (Push3Interpreter) using our helper function
|
||||
// ----------------------------------------------------------------------
|
||||
let creation_hex_filename = "../onchain/out/Push3Interpreter.sol/Push3Interpreter.json";
|
||||
let creation_bytes = get_creation_code(creation_hex_filename)?;
|
||||
println!(
|
||||
"Loaded creation code from {} ({} bytes)",
|
||||
creation_hex_filename,
|
||||
creation_bytes.len()
|
||||
);
|
||||
|
||||
// 2) Create the ephemeral EVM (deploy the contract)
|
||||
let mut runner = EvmRunner::new(creation_bytes)?;
|
||||
println!("Deployed test interpreter at: 0x{:x}\n", runner.interpreter_addr);
|
||||
|
||||
// 3) We'll generate random ASTs & mutate them
|
||||
let number_of_programs = 3; // how many random ASTs to try
|
||||
let max_depth = 4; // max AST depth
|
||||
let mut rng = thread_rng();
|
||||
|
||||
for i in 0..number_of_programs {
|
||||
// a) Generate a random AST
|
||||
let ast: UntypedAst = random_ast(&mut rng, 0, max_depth);
|
||||
|
||||
println!("=== Program {} ===", i);
|
||||
println!("Random AST:\n{:#?}", ast);
|
||||
|
||||
// b) Run the original AST
|
||||
match runner.run_ast(&ast) {
|
||||
Ok(outputs) => {
|
||||
println!("Original AST ran successfully!");
|
||||
println!("Final CODE stack: {:?}", outputs.final_code_stack);
|
||||
println!("Final EXEC stack: {:?}", outputs.final_exec_stack);
|
||||
println!("Final INT stack: {:?}", outputs.final_int_stack);
|
||||
|
||||
// c) Mutate it a few times
|
||||
let mutated1 = mutated_ast(&ast, &mut rng, 0, max_depth);
|
||||
println!("\n-- MUTATED 1 --\n{:#?}", mutated1);
|
||||
|
||||
match runner.run_ast(&mutated1) {
|
||||
Ok(m1_out) => {
|
||||
println!("MUTATED 1 ran successfully!");
|
||||
println!("Final INT stack: {:?}", m1_out.final_int_stack);
|
||||
}
|
||||
Err(e) => {
|
||||
println!("MUTATED 1 error: {}", e);
|
||||
}
|
||||
}
|
||||
|
||||
let mutated2 = mutated_ast(&ast, &mut rng, 0, max_depth);
|
||||
println!("\n-- MUTATED 2 --\n{:#?}", mutated2);
|
||||
|
||||
match runner.run_ast(&mutated2) {
|
||||
Ok(m2_out) => {
|
||||
println!("MUTATED 2 ran successfully!");
|
||||
println!("Final INT stack: {:?}", m2_out.final_int_stack);
|
||||
}
|
||||
Err(e) => {
|
||||
println!("MUTATED 2 error: {}", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
// If the original fails, we won't bother mutating it
|
||||
println!("Original AST error: {}", e);
|
||||
}
|
||||
}
|
||||
|
||||
println!("=== End of Program {} ===\n", i);
|
||||
}
|
||||
|
||||
println!("Finished running & mutating {} random programs.", number_of_programs);
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,223 @@
|
||||
// src/bin/symreg_experiment.rs
|
||||
|
||||
use anyhow::Result;
|
||||
use rand::{thread_rng, Rng};
|
||||
|
||||
// Our GP + compiler modules (adjust paths as needed)
|
||||
use offchain::compiler::ast::{UntypedAst, Push3Ast};
|
||||
use offchain::helpers::artifact::get_creation_code;
|
||||
use offchain::runner::revm_runner::{
|
||||
EvmRunner,
|
||||
Push3InterpreterInputs,
|
||||
};
|
||||
use offchain::compiler::push3_describtor::make_sublist_descriptor;
|
||||
|
||||
// Our GP operators
|
||||
use offchain::gp::generate_spec::ranmdom_code_fixed;
|
||||
use offchain::gp::mutation::{mutate_by_index, crossover_by_index};
|
||||
|
||||
// Ethers + ABI
|
||||
use ethers::types::U256;
|
||||
use ethers::abi::{encode, Token};
|
||||
|
||||
/// 1) Generate (x, y) samples for f(x)=3x^2 + x + 3
|
||||
fn generate_samples() -> Vec<(i32, i32)> {
|
||||
let mut samples = Vec::new();
|
||||
for x in -5..=5 {
|
||||
let y = 3 * x * x + x + 3;
|
||||
samples.push((x, y));
|
||||
}
|
||||
samples
|
||||
}
|
||||
|
||||
/// 2) Evaluate a single AST on a single input `x` by
|
||||
/// pushing `x` onto the int stack, running in EVM,
|
||||
/// and reading the final top of the int stack.
|
||||
fn evaluate_ast_on_x(
|
||||
runner: &mut EvmRunner,
|
||||
ast: &UntypedAst,
|
||||
x: i32,
|
||||
) -> i32 {
|
||||
// a) Convert AST => push3 bytecode
|
||||
let code_bytes = ast.to_bytecode();
|
||||
let code_len = code_bytes.len() as u32;
|
||||
|
||||
// b) Build a sublist descriptor
|
||||
let descriptor = make_sublist_descriptor(0, code_len);
|
||||
|
||||
// c) Provide x in `init_int_stack`
|
||||
let inputs = Push3InterpreterInputs {
|
||||
code: code_bytes,
|
||||
init_code_stack: Vec::new(),
|
||||
init_exec_stack: vec![descriptor],
|
||||
init_int_stack: vec![x as i128],
|
||||
};
|
||||
|
||||
// d) Actually run the interpreter
|
||||
let outputs = match runner.run_interpreter(&inputs) {
|
||||
Ok(o) => o,
|
||||
Err(_) => {
|
||||
// If the call fails or reverts, return 0
|
||||
return 0;
|
||||
}
|
||||
};
|
||||
|
||||
// e) If final_int_stack is empty => 0, else top item
|
||||
if outputs.final_int_stack.is_empty() {
|
||||
i32::MAX
|
||||
} else {
|
||||
*outputs.final_int_stack.last().unwrap() as i32
|
||||
}
|
||||
}
|
||||
|
||||
/// 3) Evaluate an AST on all (x, y) samples => compute MSE
|
||||
fn evaluate_fitness(
|
||||
runner: &mut EvmRunner,
|
||||
ast: &UntypedAst,
|
||||
samples: &[(i32, i32)]
|
||||
) -> f64 {
|
||||
let mut error_sum = 0.0;
|
||||
for &(x, target_y32) in samples {
|
||||
// cast to i64
|
||||
let target_y = target_y32 as i64;
|
||||
|
||||
// evaluate in i64 as well
|
||||
let predicted_i64 = evaluate_ast_on_x(runner, ast, x) as i64;
|
||||
|
||||
let diff = predicted_i64.wrapping_sub(target_y);
|
||||
// or you could saturating_sub if you prefer
|
||||
|
||||
error_sum += (diff as f64) * (diff as f64);
|
||||
}
|
||||
error_sum / samples.len() as f64
|
||||
}
|
||||
|
||||
|
||||
fn main() -> Result<()> {
|
||||
// 1) Generate samples for f(x)=3x^2 + x + 3
|
||||
let samples = generate_samples();
|
||||
|
||||
// 2) Create ephemeral EVM runner for the interpreter
|
||||
let creation_hex_filename = "../onchain/out/Push3Interpreter.sol/Push3Interpreter.json";
|
||||
let creation_bytes = get_creation_code(creation_hex_filename)?;
|
||||
let mut runner = EvmRunner::new(creation_bytes)?;
|
||||
|
||||
// 3) GP parameters
|
||||
let pop_size = 4000; // must be divisible by 4 for our scheme
|
||||
let generations = 100;
|
||||
let early_stop_threshold = 1.0;
|
||||
let max_points = 20;
|
||||
|
||||
assert_eq!(pop_size % 4, 0, "pop_size must be multiple of 4 for our scheme");
|
||||
let quarter = pop_size / 4;
|
||||
|
||||
let mut rng = thread_rng();
|
||||
|
||||
// 4) Initialize population (random)
|
||||
let mut population: Vec<UntypedAst> = (0..pop_size)
|
||||
.map(|_| ranmdom_code_fixed(&mut rng, max_points))
|
||||
.collect();
|
||||
|
||||
// 5) Main GP loop
|
||||
for gen in 0..generations {
|
||||
// (a) Evaluate
|
||||
let mut scored: Vec<(UntypedAst, f64)> = population.into_iter()
|
||||
.map(|ast| {
|
||||
let err = evaluate_fitness(&mut runner, &ast, &samples);
|
||||
(ast, err)
|
||||
})
|
||||
.collect();
|
||||
|
||||
// (b) Sort ascending by error
|
||||
scored.sort_by(|a, b| a.1.partial_cmp(&b.1).unwrap());
|
||||
|
||||
// (c) Print generation summary
|
||||
let best_err = scored[0].1;
|
||||
println!("\n=== Generation {gen} ===");
|
||||
println!("Best error = {best_err}");
|
||||
|
||||
// Print top 5 subjects in pretty format
|
||||
// let top_n = 5.min(scored.len());
|
||||
// println!("Top {top_n} subjects:");
|
||||
// for i in 0..top_n {
|
||||
// println!(
|
||||
// " #{i}, err={}, AST:\n{:#?}",
|
||||
// scored[i].1,
|
||||
// scored[i].0
|
||||
// );
|
||||
// }
|
||||
|
||||
// early stop?
|
||||
if best_err < early_stop_threshold {
|
||||
println!("Best error < {early_stop_threshold}, stopping early!");
|
||||
// we can keep 'scored' for final if we want
|
||||
population = scored.into_iter().map(|(a, _)| a).collect();
|
||||
break;
|
||||
}
|
||||
|
||||
// (d) Reproduction => build new population
|
||||
let mut new_pop = Vec::with_capacity(pop_size);
|
||||
|
||||
// keep best quarter
|
||||
for i in 0..quarter {
|
||||
new_pop.push(scored[i].0.clone());
|
||||
}
|
||||
|
||||
// add random quarter
|
||||
for _ in 0..quarter {
|
||||
let fresh = ranmdom_code_fixed(&mut rng, max_points);
|
||||
new_pop.push(fresh);
|
||||
}
|
||||
|
||||
// crossover quarter
|
||||
let top_half_count = scored.len() / 2;
|
||||
let crossover_pairs = quarter / 2; // each yields 2 kids => quarter total
|
||||
for _ in 0..crossover_pairs {
|
||||
let idx1 = rng.gen_range(0..top_half_count);
|
||||
let idx2 = rng.gen_range(0..top_half_count);
|
||||
let (child1, child2) = crossover_by_index(
|
||||
&scored[idx1].0,
|
||||
&scored[idx2].0,
|
||||
&mut rng
|
||||
);
|
||||
new_pop.push(child1);
|
||||
new_pop.push(child2);
|
||||
}
|
||||
|
||||
// mutate quarter
|
||||
while new_pop.len() < pop_size {
|
||||
let idx = rng.gen_range(0..top_half_count);
|
||||
let mutated = mutate_by_index(&scored[idx].0, &mut rng, max_points);
|
||||
new_pop.push(mutated);
|
||||
}
|
||||
|
||||
population = new_pop;
|
||||
}
|
||||
|
||||
// 6) Final population => evaluate & sort, then print only top 10
|
||||
println!("\n=== Final Population (Top 10) ===");
|
||||
|
||||
// a) Evaluate each item in the final population
|
||||
let mut final_scored: Vec<(UntypedAst, f64)> = population
|
||||
.into_iter()
|
||||
.map(|ast| {
|
||||
let err = evaluate_fitness(&mut runner, &ast, &samples);
|
||||
(ast, err)
|
||||
})
|
||||
.collect();
|
||||
|
||||
// b) Sort ascending by error
|
||||
final_scored.sort_by(|a, b| a.1.partial_cmp(&b.1).unwrap());
|
||||
|
||||
// c) Print the top 10 (or fewer if the population is smaller)
|
||||
let top_n = 10.min(final_scored.len());
|
||||
for i in 0..top_n {
|
||||
let (ref ast, err) = final_scored[i];
|
||||
println!(
|
||||
"Subject #{i}, err={err}, AST:\n{:#?}",
|
||||
ast
|
||||
);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
// src/bin/symreg_experiment_local.rs
|
||||
|
||||
use anyhow::Result;
|
||||
use rand::{thread_rng, Rng};
|
||||
|
||||
// Our GP + compiler modules (adjust paths if needed)
|
||||
use offchain::compiler::ast::{UntypedAst, Push3Ast};
|
||||
use offchain::helpers::artifact::get_creation_code;
|
||||
use offchain::runner::revm_runner::{EvmRunner, Push3InterpreterInputs};
|
||||
use offchain::compiler::push3_describtor::make_sublist_descriptor;
|
||||
|
||||
// Our random code and local mutation
|
||||
use offchain::gp::generate_spec::ranmdom_code_fixed;
|
||||
use offchain::gp::local_mutation::local_mutation_fixed;
|
||||
|
||||
// Ethers + ABI
|
||||
use ethers::types::U256;
|
||||
use ethers::abi::{encode, Token};
|
||||
|
||||
/// 1) Generate (x, y) samples for f(x) = 3x^2 + x + 3
|
||||
fn generate_samples() -> Vec<(i32, i32)> {
|
||||
let mut samples = Vec::new();
|
||||
for x in -5..=5 {
|
||||
let y = 3 * x * x + x + 3;
|
||||
samples.push((x, y));
|
||||
}
|
||||
samples
|
||||
}
|
||||
|
||||
/// 2) Evaluate a single AST on a single input `x`, returning the final top of the int stack.
|
||||
fn evaluate_ast_on_x(
|
||||
runner: &mut EvmRunner,
|
||||
ast: &UntypedAst,
|
||||
x: i32,
|
||||
) -> i32 {
|
||||
// Convert AST => push3 bytecode
|
||||
let code_bytes = ast.to_bytecode();
|
||||
let code_len = code_bytes.len() as u32;
|
||||
|
||||
// Build a sublist descriptor
|
||||
let descriptor = make_sublist_descriptor(0, code_len);
|
||||
|
||||
// Provide x in `init_int_stack`
|
||||
let inputs = Push3InterpreterInputs {
|
||||
code: code_bytes,
|
||||
init_code_stack: Vec::new(),
|
||||
init_exec_stack: vec![descriptor],
|
||||
init_int_stack: vec![x as i128],
|
||||
};
|
||||
|
||||
// Run
|
||||
let outputs = match runner.run_interpreter(&inputs) {
|
||||
Ok(o) => o,
|
||||
Err(_) => {
|
||||
// If revert => 0
|
||||
return 0;
|
||||
}
|
||||
};
|
||||
|
||||
if outputs.final_int_stack.is_empty() {
|
||||
i32::MAX // or 0, or i32::MAX to strongly penalize weird runs
|
||||
} else {
|
||||
*outputs.final_int_stack.last().unwrap() as i32
|
||||
}
|
||||
}
|
||||
|
||||
/// 3) Evaluate an AST on all samples => compute MSE
|
||||
fn evaluate_fitness(
|
||||
runner: &mut EvmRunner,
|
||||
ast: &UntypedAst,
|
||||
samples: &[(i32, i32)]
|
||||
) -> f64 {
|
||||
let mut error_sum = 0.0;
|
||||
for &(x, target_y32) in samples {
|
||||
// cast to i64
|
||||
let target_y = target_y32 as i64;
|
||||
|
||||
// evaluate in i64
|
||||
let predicted_i64 = evaluate_ast_on_x(runner, ast, x) as i64;
|
||||
|
||||
// wrapping_sub to avoid panic
|
||||
let diff = predicted_i64.wrapping_sub(target_y);
|
||||
error_sum += (diff as f64) * (diff as f64);
|
||||
}
|
||||
error_sum / samples.len() as f64
|
||||
}
|
||||
|
||||
fn main() -> Result<()> {
|
||||
// 1) Generate samples
|
||||
let samples = generate_samples();
|
||||
|
||||
// 2) Setup ephemeral EVM runner
|
||||
let creation_hex_filename = "../onchain/out/Push3Interpreter.sol/Push3Interpreter.json";
|
||||
let creation_bytes = get_creation_code(creation_hex_filename)?;
|
||||
let mut runner = EvmRunner::new(creation_bytes)?;
|
||||
|
||||
// 3) GP parameters
|
||||
let pop_size = 3600; // must be multiple of 3 for our scheme
|
||||
let generations = 100;
|
||||
let early_stop_threshold = 1.0;
|
||||
let max_points = 12; // random code size limit
|
||||
|
||||
// Ensure pop_size is multiple of 3
|
||||
assert_eq!(pop_size % 3, 0, "pop_size must be multiple of 3 for our scheme");
|
||||
let third = pop_size / 3;
|
||||
|
||||
let mut rng = thread_rng();
|
||||
|
||||
// 4) Initialize random population
|
||||
let mut population: Vec<UntypedAst> = (0..pop_size)
|
||||
.map(|_| ranmdom_code_fixed(&mut rng, max_points))
|
||||
.collect();
|
||||
|
||||
// 5) Main loop
|
||||
for gen in 0..generations {
|
||||
// a) Evaluate
|
||||
let mut scored: Vec<(UntypedAst, f64)> = population.into_iter()
|
||||
.map(|ast| {
|
||||
let err = evaluate_fitness(&mut runner, &ast, &samples);
|
||||
(ast, err)
|
||||
})
|
||||
.collect();
|
||||
|
||||
// b) Sort ascending by error
|
||||
scored.sort_by(|a, b| a.1.partial_cmp(&b.1).unwrap());
|
||||
|
||||
let best_err = scored[0].1;
|
||||
println!("\n=== Generation {gen} ===");
|
||||
println!("Best error = {best_err}");
|
||||
|
||||
// (Optionally) Print top 5 in pretty format
|
||||
let top_n = 5.min(scored.len());
|
||||
println!("Top {top_n} subjects:");
|
||||
for i in 0..top_n {
|
||||
println!(
|
||||
" #{i}, err={}, AST:\n{:#?}",
|
||||
scored[i].1,
|
||||
scored[i].0
|
||||
);
|
||||
}
|
||||
|
||||
// check early stop
|
||||
if best_err < early_stop_threshold {
|
||||
println!("Best error < {early_stop_threshold}, stopping early!");
|
||||
population = scored.into_iter().map(|(a, _)| a).collect();
|
||||
break;
|
||||
}
|
||||
|
||||
// c) Build new population:
|
||||
// - 1/3 = best survivors
|
||||
// - 1/3 = fresh random
|
||||
// - 1/3 = local mutation
|
||||
let mut new_pop = Vec::with_capacity(pop_size);
|
||||
|
||||
// keep best third
|
||||
for i in 0..third {
|
||||
new_pop.push(scored[i].0.clone());
|
||||
}
|
||||
|
||||
// random third
|
||||
for _ in 0..third {
|
||||
let fresh = ranmdom_code_fixed(&mut rng, max_points);
|
||||
new_pop.push(fresh);
|
||||
}
|
||||
|
||||
// local mutation third
|
||||
let top_half_count = scored.len() / 2;
|
||||
// we'll pick parents from top half for local mutation
|
||||
while new_pop.len() < pop_size {
|
||||
let idx = rng.gen_range(0..top_half_count);
|
||||
// we assume you have local_mutation_fixed
|
||||
// if you prefer passing a custom InstructionSet, do local_mutation(..., &instr_set)
|
||||
let mutated = offchain::gp::local_mutation::local_mutation_fixed(
|
||||
&scored[idx].0,
|
||||
&mut rng
|
||||
);
|
||||
new_pop.push(mutated);
|
||||
}
|
||||
|
||||
// store for next generation
|
||||
population = new_pop;
|
||||
}
|
||||
|
||||
// 6) Final => evaluate & sort => top 10
|
||||
println!("\n=== Final Population (Top 10) ===");
|
||||
let mut final_scored: Vec<(UntypedAst, f64)> = population
|
||||
.into_iter()
|
||||
.map(|ast| {
|
||||
let err = evaluate_fitness(&mut runner, &ast, &samples);
|
||||
(ast, err)
|
||||
})
|
||||
.collect();
|
||||
final_scored.sort_by(|a, b| a.1.partial_cmp(&b.1).unwrap());
|
||||
|
||||
let top_n = 10.min(final_scored.len());
|
||||
for i in 0..top_n {
|
||||
let (ref ast, err) = final_scored[i];
|
||||
println!("Subject #{i}, err={err}, AST:\n{:#?}", ast);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,273 @@
|
||||
//! src/compiler/ast.rs
|
||||
//!
|
||||
//! This module defines an **abstract** untyped AST for our Push3-like language,
|
||||
//! plus a mapping for how to encode opcodes as bytes. We do *not* hardcode numeric
|
||||
//! values in the `OpCode` enum. Instead, we provide a trait that can map
|
||||
//! opcodes to bytes. This allows us to expand or change instructions easily
|
||||
//! (e.g., when the on-chain interpreter adds new opcodes or changes their IDs).
|
||||
|
||||
/// A trait describing how to convert an AST into Push3 bytecode.
|
||||
///
|
||||
/// This is deliberately minimal for now. In the future, we could add more methods
|
||||
/// (like parsing from bytecode or AST mutation).
|
||||
pub trait Push3Ast {
|
||||
/// Convert this AST into a bytecode vector that the on-chain interpreter
|
||||
/// can parse and execute.
|
||||
fn to_bytecode(&self) -> Vec<u8>;
|
||||
}
|
||||
|
||||
/// A trait that **maps** each [`OpCode`] variant to its **single-byte** representation.
|
||||
///
|
||||
/// By centralizing this, we can easily add or reorder instructions without rewriting
|
||||
/// big chunks of code everywhere. For example, if we add `OpCode::MyNewOp`, we just
|
||||
/// update this trait’s logic.
|
||||
pub trait OpCodeMapping {
|
||||
/// Given an `OpCode` enum variant, return the corresponding single `u8`
|
||||
/// that the interpreter expects.
|
||||
fn opcode_byte(&self, op: &OpCode) -> u8;
|
||||
|
||||
// If you need reverse-lookup (byte => OpCode), you could add a method like:
|
||||
// fn from_byte(&self, b: u8) -> Option<OpCode>;
|
||||
// for now, we omit it since we only do forward mapping in this file.
|
||||
}
|
||||
|
||||
/// Our untyped AST node:
|
||||
/// - `IntLiteral(i32)` holds a literal integer,
|
||||
/// - `Instruction(OpCode)` holds one opcode,
|
||||
/// - `Sublist(Vec<UntypedAst>)` holds a collection of nested AST nodes.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum UntypedAst {
|
||||
IntLiteral(i32),
|
||||
Instruction(OpCode),
|
||||
Sublist(Vec<UntypedAst>),
|
||||
}
|
||||
|
||||
/// An **abstract** set of opcodes. We do *not* assign numeric values here.
|
||||
/// Instead, see [`OpCodeMapping::opcode_byte`] for how we convert them to bytes.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum OpCode {
|
||||
Noop,
|
||||
Plus,
|
||||
Minus,
|
||||
Mult,
|
||||
Dup,
|
||||
Pop,
|
||||
// Add new instructions here as needed. E.g.:
|
||||
// MyNewOp, etc.
|
||||
}
|
||||
|
||||
impl UntypedAst {
|
||||
/// Encode this AST into bytecode, using a provided [`OpCodeMapping`].
|
||||
///
|
||||
/// This method is more flexible than `to_bytecode()`, because you can pass in
|
||||
/// *any* mapping if needed. The method used by the trait’s `to_bytecode()`
|
||||
/// relies on the global `DEFAULT_OP_MAPPING`.
|
||||
pub fn to_bytecode_with_mapping<M: OpCodeMapping>(&self, mapping: &M) -> Vec<u8> {
|
||||
match self {
|
||||
// For an integer literal, we write the “tag byte” for int-literal, then 4 bytes (LE).
|
||||
UntypedAst::IntLiteral(val) => {
|
||||
// Hardcode 0x02 => INT_LITERAL.
|
||||
// You *could* put that in the mapping if you want to make that flexible, too.
|
||||
let mut bytes = Vec::with_capacity(1 + 4);
|
||||
bytes.push(0x02);
|
||||
bytes.extend_from_slice(&val.to_be_bytes());
|
||||
bytes
|
||||
}
|
||||
UntypedAst::Instruction(op) => {
|
||||
// Use the mapping to find the correct opcode byte:
|
||||
let b = mapping.opcode_byte(op);
|
||||
vec![b]
|
||||
}
|
||||
UntypedAst::Sublist(children) => {
|
||||
// Hardcode 0x03 => SUBLIST, then big-endian length, then child payload
|
||||
let mut payload = Vec::new();
|
||||
for child in children {
|
||||
let child_bytes = child.to_bytecode_with_mapping(mapping);
|
||||
payload.extend(child_bytes);
|
||||
}
|
||||
let sub_len = payload.len() as u16;
|
||||
let mut bytes = Vec::with_capacity(1 + 2 + payload.len());
|
||||
bytes.push(0x03);
|
||||
bytes.extend_from_slice(&sub_len.to_be_bytes()); // big-endian length
|
||||
bytes.extend(payload);
|
||||
bytes
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// For convenience, we implement `Push3Ast` using a *default* mapping.
|
||||
impl Push3Ast for UntypedAst {
|
||||
fn to_bytecode(&self) -> Vec<u8> {
|
||||
self.to_bytecode_with_mapping(&DEFAULT_OP_MAPPING)
|
||||
}
|
||||
}
|
||||
|
||||
/// A default mapping that corresponds to your current on-chain byte definitions.
|
||||
///
|
||||
/// This way, if your interpreter changes (e.g., `Minus` becomes 0x0A),
|
||||
/// you just update this mapping.
|
||||
pub struct DefaultOpCodeMapping;
|
||||
|
||||
impl OpCodeMapping for DefaultOpCodeMapping {
|
||||
fn opcode_byte(&self, op: &OpCode) -> u8 {
|
||||
match op {
|
||||
OpCode::Noop => 0x00, // 0x00 => NOOP
|
||||
OpCode::Plus => 0x01, // 0x01 => INTEGER_PLUS
|
||||
OpCode::Minus => 0x04, // 0x04 => INTEGER_MINUS
|
||||
OpCode::Mult => 0x05, // 0x05 => INTEGER_MULT
|
||||
OpCode::Dup => 0x06, // 0x06 => INTEGER_DUP
|
||||
OpCode::Pop => 0x07, // 0x07 => INTEGER_POP
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A convenient global `const` or `static` for quick usage.
|
||||
pub const DEFAULT_OP_MAPPING: DefaultOpCodeMapping = DefaultOpCodeMapping;
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// S-Expression Parsing Helpers
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
/// A simple S-expression tree for textual representation: either an `Atom` or a `List`.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum SExpr {
|
||||
Atom(String),
|
||||
List(Vec<SExpr>),
|
||||
}
|
||||
|
||||
/// Tokenize a string into parentheses and symbols (atoms).
|
||||
pub fn tokenize(s: &str) -> Vec<String> {
|
||||
let mut tokens = Vec::new();
|
||||
let mut current = String::new();
|
||||
|
||||
for c in s.chars() {
|
||||
match c {
|
||||
'(' | ')' => {
|
||||
if !current.trim().is_empty() {
|
||||
tokens.push(current.trim().to_string());
|
||||
}
|
||||
current.clear();
|
||||
tokens.push(c.to_string());
|
||||
}
|
||||
' ' | '\t' | '\n' | '\r' => {
|
||||
if !current.trim().is_empty() {
|
||||
tokens.push(current.trim().to_string());
|
||||
current.clear();
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
current.push(c);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if !current.trim().is_empty() {
|
||||
tokens.push(current.trim().to_string());
|
||||
}
|
||||
|
||||
tokens
|
||||
}
|
||||
|
||||
/// Recursive helper to parse a single S-expression from a list of tokens, advancing `pos`.
|
||||
fn parse_sexpr_internal(tokens: &[String], pos: &mut usize) -> Result<SExpr, String> {
|
||||
if *pos >= tokens.len() {
|
||||
return Err("Unexpected end of tokens".to_string());
|
||||
}
|
||||
|
||||
let token = &tokens[*pos];
|
||||
match token.as_str() {
|
||||
"(" => {
|
||||
*pos += 1; // consume '('
|
||||
let mut items = Vec::new();
|
||||
while *pos < tokens.len() && tokens[*pos] != ")" {
|
||||
let expr = parse_sexpr_internal(tokens, pos)?;
|
||||
items.push(expr);
|
||||
}
|
||||
if *pos >= tokens.len() {
|
||||
return Err("Missing closing parenthesis".to_string());
|
||||
}
|
||||
*pos += 1; // consume ')'
|
||||
Ok(SExpr::List(items))
|
||||
}
|
||||
")" => Err("Unexpected ')'".to_string()),
|
||||
_ => {
|
||||
let atom_str = token.clone();
|
||||
*pos += 1;
|
||||
Ok(SExpr::Atom(atom_str))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse a full string into a single top-level `SExpr`, assuming balanced parentheses.
|
||||
pub fn parse_string_to_sexpr(s: &str) -> Result<SExpr, String> {
|
||||
let tokens = tokenize(s);
|
||||
let mut pos = 0;
|
||||
let expr = parse_sexpr_internal(&tokens, &mut pos)?;
|
||||
if pos < tokens.len() {
|
||||
Err(format!(
|
||||
"Extra tokens after parse: {:?}",
|
||||
&tokens[pos..]
|
||||
))
|
||||
} else {
|
||||
Ok(expr)
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert an S-expression to an **untyped** AST node.
|
||||
pub fn sexpr_to_untyped(expr: &SExpr) -> Result<UntypedAst, String> {
|
||||
match expr {
|
||||
SExpr::Atom(text) => {
|
||||
// 1) Try parse as integer
|
||||
if let Ok(val) = text.parse::<i32>() {
|
||||
Ok(UntypedAst::IntLiteral(val))
|
||||
} else {
|
||||
// 2) Otherwise interpret as an opcode
|
||||
match text.to_uppercase().as_str() {
|
||||
"+" => Ok(UntypedAst::Instruction(OpCode::Plus)),
|
||||
"-" => Ok(UntypedAst::Instruction(OpCode::Minus)),
|
||||
"*" => Ok(UntypedAst::Instruction(OpCode::Mult)),
|
||||
"DUP" => Ok(UntypedAst::Instruction(OpCode::Dup)),
|
||||
"POP" => Ok(UntypedAst::Instruction(OpCode::Pop)),
|
||||
// unknown => treat as Noop
|
||||
_ => Ok(UntypedAst::Instruction(OpCode::Noop)),
|
||||
}
|
||||
}
|
||||
}
|
||||
SExpr::List(items) => {
|
||||
let mut sub_asts = Vec::new();
|
||||
for child in items {
|
||||
sub_asts.push(sexpr_to_untyped(child)?);
|
||||
}
|
||||
Ok(UntypedAst::Sublist(sub_asts))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// (Optional) Placeholder for a future typed AST
|
||||
// ----------------------------------------------------------------------------
|
||||
|
||||
// /// In the future, we plan to define a `TypedAst` enum that ensures each operation
|
||||
// /// has the correct number of typed children. We might also handle errors
|
||||
// /// if an operation expects two integer children, but the user tries to attach
|
||||
// /// a sublist or zero children, etc.
|
||||
// ///
|
||||
// /// For now, the untyped AST suffices for quick prototyping, but typed checks
|
||||
// /// will be crucial in more advanced genetic programming or static analysis scenarios.
|
||||
// #[derive(Debug, Clone)]
|
||||
// pub enum TypedAst {
|
||||
// // e.g. IntLiteral(i32),
|
||||
// // Instruction(InstructionKind),
|
||||
// }
|
||||
|
||||
// #[derive(Debug, Clone)]
|
||||
// pub enum InstructionKind {
|
||||
// // e.g. Plus(Box<TypedAst>, Box<TypedAst>),
|
||||
// // Minus(Box<TypedAst>, Box<TypedAst>),
|
||||
// // Mult(Box<TypedAst>, Box<TypedAst>),
|
||||
// // Dup,
|
||||
// // Pop,
|
||||
// }
|
||||
|
||||
// // etc.
|
||||
@@ -0,0 +1,2 @@
|
||||
pub mod ast;
|
||||
pub mod push3_describtor;
|
||||
@@ -0,0 +1,67 @@
|
||||
//! src/compiler/push3_descriptor.rs
|
||||
//!
|
||||
//! A small module of helper functions to construct or interpret the 256-bit descriptors
|
||||
//! used by our on-chain `Push3Interpreter`. Each descriptor encodes a tag, an offset,
|
||||
//! a length, and any leftover bits (e.g., immediate data).
|
||||
|
||||
use ethers::types::U256;
|
||||
|
||||
/// The 256-bit descriptor layout is as follows:
|
||||
/// [ 8 bits: tag | 32 bits: offset | 32 bits: length | 184 bits: leftover ]
|
||||
///
|
||||
/// We'll define small helper functions to build or parse these.
|
||||
/// Our `make_sublist_descriptor` is an example for tag=3 (SUBLIST).
|
||||
|
||||
/// Tag constants, if you want them:
|
||||
pub const TAG_SUBLIST: u8 = 3;
|
||||
pub const TAG_INT_LITERAL: u8 = 2;
|
||||
pub const TAG_INSTRUCTION: u8 = 1;
|
||||
pub const TAG_NONE: u8 = 0;
|
||||
|
||||
/// A small helper to shift the tag into the top 8 bits.
|
||||
#[inline]
|
||||
fn tag_bits(tag: u8) -> U256 {
|
||||
U256::from(tag) << 248
|
||||
}
|
||||
|
||||
/// Build a descriptor for a SUBLIST with the given offset and length (in bytes),
|
||||
/// leftover=0.
|
||||
///
|
||||
/// # Example
|
||||
/// ```
|
||||
/// let desc = make_sublist_descriptor(0, code_bytes.len() as u32);
|
||||
/// // pass `desc` into the interpreter's exec stack
|
||||
/// ```
|
||||
pub fn make_sublist_descriptor(offset: u32, length: u32) -> U256 {
|
||||
tag_bits(TAG_SUBLIST as u8)
|
||||
| (U256::from(offset) << 216)
|
||||
| (U256::from(length) << 184)
|
||||
| U256::zero() // leftover
|
||||
}
|
||||
|
||||
/// Possibly you want a more general helper that builds any descriptor:
|
||||
/// If you have other uses for leftover bits, you can pass that in:
|
||||
pub fn make_descriptor(tag: u8, offset: u32, length: u32, leftover: U256) -> U256 {
|
||||
tag_bits(tag)
|
||||
| (U256::from(offset) << 216)
|
||||
| (U256::from(length) << 184)
|
||||
| leftover
|
||||
}
|
||||
|
||||
/// If you want to parse a descriptor, you can define getTag, getOffset, getLength, etc:
|
||||
pub fn get_tag(desc: U256) -> u8 {
|
||||
// top 8 bits => shift right 248
|
||||
(desc >> 248).as_u32() as u8
|
||||
}
|
||||
|
||||
pub fn get_offset(desc: U256) -> u32 {
|
||||
((desc >> 216) & U256::from(u32::MAX)).as_u32()
|
||||
}
|
||||
|
||||
pub fn get_length(desc: U256) -> u32 {
|
||||
((desc >> 184) & U256::from(u32::MAX)).as_u32()
|
||||
}
|
||||
|
||||
pub fn get_low_184(desc: U256) -> U256 {
|
||||
desc & ((U256::from(1u64) << 184) - 1)
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
use rand::Rng;
|
||||
use crate::compiler::ast::{UntypedAst, OpCode};
|
||||
|
||||
pub fn random_sublist_ast(rng: &mut impl Rng, max_depth: usize) -> UntypedAst {
|
||||
// 1) Choose how many children the top-level `Sublist` will have.
|
||||
let len = rng.gen_range(1..=3);
|
||||
|
||||
// 2) Build a vector of sub-ASTs by calling `random_ast` for each child
|
||||
let mut children = Vec::with_capacity(len);
|
||||
for _ in 0..len {
|
||||
// We start at `depth=1` because the top-level sublist itself is `depth=0`.
|
||||
children.push(random_ast(rng, 1, max_depth));
|
||||
}
|
||||
|
||||
// 3) Return a `Sublist` as the root node
|
||||
UntypedAst::Sublist(children)
|
||||
}
|
||||
|
||||
/// Generate a random `UntypedAst` by recursively building sub-trees,
|
||||
/// int literals, or instructions.
|
||||
///
|
||||
/// - `depth` tracks how deep we are in the tree.
|
||||
/// - `max_depth` is the maximum allowed depth to prevent infinite recursion.
|
||||
pub fn random_ast(rng: &mut impl Rng, depth: usize, max_depth: usize) -> UntypedAst {
|
||||
if depth >= max_depth {
|
||||
// Return something "terminal,"
|
||||
// e.g. an IntLiteral or a single Instruction
|
||||
random_terminal(rng)
|
||||
} else {
|
||||
// Weighted choice:
|
||||
// 0 => IntLiteral,
|
||||
// 1 => single Instruction,
|
||||
// 2 => Sublist with children
|
||||
let choice = rng.gen_range(0..3);
|
||||
match choice {
|
||||
0 => UntypedAst::IntLiteral(rng.gen_range(-10..10)),
|
||||
1 => UntypedAst::Instruction(random_opcode(rng)),
|
||||
2 => {
|
||||
// Make a sublist with 1..=3 children
|
||||
let len = rng.gen_range(1..=3);
|
||||
let mut children = Vec::with_capacity(len);
|
||||
for _ in 0..len {
|
||||
children.push(random_ast(rng, depth + 1, max_depth));
|
||||
}
|
||||
UntypedAst::Sublist(children)
|
||||
}
|
||||
_ => unreachable!(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Generate a random "terminal" node.
|
||||
/// Typically either an int literal or a single instruction.
|
||||
fn random_terminal(rng: &mut impl Rng) -> UntypedAst {
|
||||
// Weighted choice:
|
||||
// 0 => IntLiteral
|
||||
// 1 => single Instruction
|
||||
let choice = rng.gen_range(0..2);
|
||||
match choice {
|
||||
0 => UntypedAst::IntLiteral(rng.gen_range(-10..10)),
|
||||
1 => UntypedAst::Instruction(random_opcode(rng)),
|
||||
_ => unreachable!(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Generate a random `OpCode` from your known set of instructions.
|
||||
pub fn random_opcode(rng: &mut impl Rng) -> OpCode {
|
||||
let choice = rng.gen_range(0..6);
|
||||
// because we have 6 variants: Noop, Plus, Minus, Mult, Dup, Pop
|
||||
match choice {
|
||||
0 => OpCode::Noop,
|
||||
1 => OpCode::Plus,
|
||||
2 => OpCode::Minus,
|
||||
3 => OpCode::Mult,
|
||||
4 => OpCode::Dup,
|
||||
5 => OpCode::Pop,
|
||||
_ => unreachable!(),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
use crate::compiler::ast::{UntypedAst, OpCode};
|
||||
use rand::Rng;
|
||||
use rand::prelude::SliceRandom;
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub enum InstructionAtom {
|
||||
/// A normal opcode, e.g. `Plus`, `Minus`, `Dup`, etc.
|
||||
Opcode(OpCode),
|
||||
/// An ephemeral random constant placeholder.
|
||||
/// If chosen, we generate a random literal on the spot.
|
||||
EphemeralInt,
|
||||
// If you want ephemeral floats, booleans, etc., add more variants
|
||||
}
|
||||
|
||||
/// A small struct to hold our entire “instruction set.”
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct InstructionSet {
|
||||
pub atoms: Vec<InstructionAtom>,
|
||||
}
|
||||
|
||||
impl InstructionSet {
|
||||
pub fn new_default() -> Self {
|
||||
// Example: Some opcodes + ephemeral constant
|
||||
// Tweak as you like
|
||||
use InstructionAtom::*;
|
||||
use OpCode::*;
|
||||
|
||||
Self {
|
||||
atoms: vec![
|
||||
Opcode(Noop),
|
||||
Opcode(Plus),
|
||||
Opcode(Minus),
|
||||
Opcode(Mult),
|
||||
Opcode(Dup),
|
||||
Opcode(Pop),
|
||||
EphemeralInt,
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
/// Pick a random atom from this set.
|
||||
/// If it's `EphemeralInt`, we produce `UntypedAst::IntLiteral(...)`.
|
||||
/// If it's `Opcode(...)`, we produce `UntypedAst::Instruction(...)`.
|
||||
pub fn random_atom_as_ast(&self, rng: &mut impl Rng) -> UntypedAst {
|
||||
let idx = rng.gen_range(0..self.atoms.len());
|
||||
match &self.atoms[idx] {
|
||||
InstructionAtom::Opcode(op) => UntypedAst::Instruction(op.clone()),
|
||||
InstructionAtom::EphemeralInt => {
|
||||
// For ephemeral int, produce a random literal in some range
|
||||
let val = rng.gen_range(-30..30);
|
||||
UntypedAst::IntLiteral(val)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn ranmdom_code_fixed(rng: &mut impl Rng, max_points: usize) -> UntypedAst {
|
||||
let instr_set = InstructionSet::new_default();
|
||||
return random_code(rng, &instr_set, max_points);
|
||||
}
|
||||
|
||||
/// The "entry point": random_code(max_points).
|
||||
/// 1) Choose actual_points in [1..=max_points].
|
||||
/// 2) Return the result of random_code_with_size.
|
||||
pub fn random_code(rng: &mut impl Rng, instr_set: &InstructionSet, max_points: usize) -> UntypedAst {
|
||||
let actual_points = rng.gen_range(1..=max_points);
|
||||
random_code_with_size(rng, instr_set, actual_points)
|
||||
}
|
||||
|
||||
/// The main logic:
|
||||
/// If points == 1 => pick a single instruction/ephemeral
|
||||
/// Else =>
|
||||
/// - We "decompose" (points - 1) among sub-codes
|
||||
/// - For each sub-points, we recursively call random_code_with_size
|
||||
/// - We return a Sublist of those children, possibly in random order
|
||||
pub fn random_code_with_size(
|
||||
rng: &mut impl Rng,
|
||||
instr_set: &InstructionSet,
|
||||
points: usize,
|
||||
) -> UntypedAst {
|
||||
use UntypedAst::*;
|
||||
|
||||
// Base case
|
||||
if points == 1 {
|
||||
// Choose a random "atom" from instr_set
|
||||
return instr_set.random_atom_as_ast(rng);
|
||||
}
|
||||
|
||||
// If `points > 1`, let's produce a Sublist
|
||||
// We'll break (points - 1) into sub-points via `decompose`
|
||||
let subpoints_list = decompose(rng, points - 1, points - 1);
|
||||
// subpoints_list is e.g. [2, 3, 5] and sums to (points-1)
|
||||
|
||||
// Then for each "subpoints", we do random_code_with_size
|
||||
let mut sub_asts: Vec<UntypedAst> = subpoints_list
|
||||
.into_iter()
|
||||
.map(|sp| random_code_with_size(rng, instr_set, sp))
|
||||
.collect();
|
||||
|
||||
// The spec says "Return a list containing the results, in random order"
|
||||
// so let's shuffle sub_asts
|
||||
sub_asts.shuffle(rng);
|
||||
|
||||
Sublist(sub_asts)
|
||||
}
|
||||
|
||||
/// Decompose a number into random parts.
|
||||
///
|
||||
/// - If number is 1 or max_parts is 1 => return [number].
|
||||
/// - Otherwise pick a random split "this_part" and recurse on the remainder.
|
||||
fn decompose(rng: &mut impl Rng, number: usize, max_parts: usize) -> Vec<usize> {
|
||||
if number == 1 || max_parts == 1 {
|
||||
return vec![number];
|
||||
}
|
||||
// pick a random "this_part" in [1 .. (number-1)]
|
||||
let this_part = rng.gen_range(1..number);
|
||||
let mut remainder = decompose(rng, number - this_part, max_parts - 1);
|
||||
let mut result = vec![this_part];
|
||||
result.append(&mut remainder);
|
||||
result
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
use rand::Rng;
|
||||
use crate::compiler::ast::{UntypedAst, OpCode};
|
||||
use crate::gp::generate::{random_ast, random_opcode};
|
||||
|
||||
/// Returns a new `UntypedAst` that is a mutated version of `original`.
|
||||
/// We do not modify `original`; we clone or build new pieces as needed.
|
||||
///
|
||||
/// - `depth`: current depth of this node,
|
||||
/// - `max_depth`: max allowed depth (to prevent infinite recursion).
|
||||
pub fn mutated_ast(original: &UntypedAst, rng: &mut impl Rng, depth: usize, max_depth: usize) -> UntypedAst {
|
||||
// 10% chance: completely replace this node with a fresh random subtree
|
||||
if rng.gen_range(0..100) < 10 {
|
||||
return random_ast(rng, depth, max_depth);
|
||||
}
|
||||
|
||||
match original {
|
||||
UntypedAst::IntLiteral(val) => {
|
||||
// Another small chance to replace the entire node with random subtree
|
||||
if rng.gen_range(0..100) < 10 {
|
||||
return random_ast(rng, depth, max_depth);
|
||||
}
|
||||
|
||||
// Otherwise, "tweak" the int by a small offset
|
||||
let delta = rng.gen_range(-2..=2);
|
||||
let new_val = val.saturating_add(delta);
|
||||
UntypedAst::IntLiteral(new_val)
|
||||
}
|
||||
UntypedAst::Instruction(op) => {
|
||||
// 10% chance to replace node
|
||||
if rng.gen_range(0..100) < 10 {
|
||||
return random_ast(rng, depth, max_depth);
|
||||
}
|
||||
|
||||
// 50% chance to pick a new opcode
|
||||
// or else keep the same
|
||||
if rng.gen_bool(0.5) {
|
||||
UntypedAst::Instruction(random_opcode(rng))
|
||||
} else {
|
||||
// Just clone the original
|
||||
UntypedAst::Instruction(op.clone())
|
||||
}
|
||||
}
|
||||
UntypedAst::Sublist(children) => {
|
||||
// 10% chance to replace this sublist entirely
|
||||
if rng.gen_range(0..100) < 10 {
|
||||
return random_ast(rng, depth, max_depth);
|
||||
}
|
||||
|
||||
// Otherwise, mutate zero or more children
|
||||
let mut new_children = children.clone();
|
||||
if !new_children.is_empty() {
|
||||
// Choose how many kids to mutate:
|
||||
// e.g., 1..=min(2, children.len())
|
||||
let mut_count = rng.gen_range(1..=new_children.len().min(2));
|
||||
|
||||
for _ in 0..mut_count {
|
||||
let idx = rng.gen_range(0..new_children.len());
|
||||
new_children[idx] = mutated_ast(&new_children[idx], rng, depth+1, max_depth);
|
||||
}
|
||||
}
|
||||
UntypedAst::Sublist(new_children)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
//! src/gp/local_mutation.rs
|
||||
//! A localized mutation operator that makes small tweaks depending on node type.
|
||||
|
||||
use rand::Rng;
|
||||
use crate::compiler::ast::{UntypedAst, OpCode};
|
||||
use crate::gp::mutation::{
|
||||
enum_nodes_dfs, get_subtree, replace_subtree,
|
||||
};
|
||||
use crate::gp::generate_spec::{
|
||||
InstructionAtom, InstructionSet,
|
||||
};
|
||||
|
||||
/// A top-level function to perform a localized mutation with a chosen `InstructionSet`.
|
||||
pub fn local_mutation(
|
||||
original: &UntypedAst,
|
||||
rng: &mut impl Rng,
|
||||
instr_set: &InstructionSet,
|
||||
) -> UntypedAst {
|
||||
let all_paths = enum_nodes_dfs(original);
|
||||
let idx = rng.gen_range(0..all_paths.len());
|
||||
let chosen_path = &all_paths[idx];
|
||||
|
||||
let old_subtree = get_subtree(original, chosen_path);
|
||||
let new_subtree = local_mutation_node(&old_subtree, rng, instr_set);
|
||||
replace_subtree(original, chosen_path, new_subtree)
|
||||
}
|
||||
|
||||
/// A convenience wrapper that always uses `InstructionSet::new_default()`.
|
||||
pub fn local_mutation_fixed(
|
||||
original: &UntypedAst,
|
||||
rng: &mut impl Rng,
|
||||
) -> UntypedAst {
|
||||
let instr_set = InstructionSet::new_default();
|
||||
local_mutation(original, rng, &instr_set)
|
||||
}
|
||||
|
||||
fn local_mutation_node(
|
||||
subtree: &UntypedAst,
|
||||
rng: &mut impl Rng,
|
||||
instr_set: &InstructionSet,
|
||||
) -> UntypedAst {
|
||||
match subtree {
|
||||
UntypedAst::IntLiteral(val) => {
|
||||
let delta = rng.gen_range(-5..=5);
|
||||
let new_val = val.saturating_add(delta);
|
||||
UntypedAst::IntLiteral(new_val)
|
||||
}
|
||||
UntypedAst::Instruction(op) => {
|
||||
let mut new_op = pick_random_opcode(rng, instr_set);
|
||||
for _ in 0..3 {
|
||||
if &new_op != op {
|
||||
break;
|
||||
}
|
||||
new_op = pick_random_opcode(rng, instr_set);
|
||||
}
|
||||
UntypedAst::Instruction(new_op)
|
||||
}
|
||||
UntypedAst::Sublist(children) => {
|
||||
let choice = rng.gen_range(0..3);
|
||||
let mut new_children = children.clone();
|
||||
|
||||
match choice {
|
||||
0 => {
|
||||
// remove one child if possible
|
||||
if !new_children.is_empty() {
|
||||
let i = rng.gen_range(0..new_children.len());
|
||||
new_children.remove(i);
|
||||
}
|
||||
}
|
||||
1 => {
|
||||
// insert a new small node
|
||||
let i = rng.gen_range(0..=new_children.len());
|
||||
let node = create_small_node(rng, instr_set);
|
||||
new_children.insert(i, node);
|
||||
}
|
||||
2 => {
|
||||
// reorder two children
|
||||
if new_children.len() > 1 {
|
||||
let i = rng.gen_range(0..new_children.len());
|
||||
let j = rng.gen_range(0..new_children.len());
|
||||
new_children.swap(i, j);
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
UntypedAst::Sublist(new_children)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn pick_random_opcode(rng: &mut impl Rng, instr_set: &InstructionSet) -> OpCode {
|
||||
use crate::compiler::ast::OpCode;
|
||||
|
||||
let mut opcode_list: Vec<&OpCode> = Vec::new();
|
||||
for atom in &instr_set.atoms {
|
||||
if let InstructionAtom::Opcode(ref op) = atom {
|
||||
opcode_list.push(op);
|
||||
}
|
||||
}
|
||||
if opcode_list.is_empty() {
|
||||
// fallback => default to Noop
|
||||
return OpCode::Noop;
|
||||
}
|
||||
let i = rng.gen_range(0..opcode_list.len());
|
||||
opcode_list[i].clone()
|
||||
}
|
||||
|
||||
fn create_small_node(
|
||||
rng: &mut impl Rng,
|
||||
instr_set: &InstructionSet,
|
||||
) -> UntypedAst {
|
||||
instr_set.random_atom_as_ast(rng)
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
pub mod population;
|
||||
pub mod genetic_ops;
|
||||
pub mod generate;
|
||||
pub mod generate_spec;
|
||||
pub mod mutation;
|
||||
pub mod local_mutation;
|
||||
@@ -0,0 +1,147 @@
|
||||
use rand::Rng;
|
||||
use crate::compiler::ast::UntypedAst;
|
||||
use crate::gp::generate_spec::ranmdom_code_fixed; // or push3-based generator
|
||||
|
||||
/// A "path" is a list of indices leading from the root to a child.
|
||||
/// For example, [] is the root, [0] is the root's first child, [0,1] is that child's second child, etc.
|
||||
pub type Path = Vec<usize>;
|
||||
|
||||
/// Enumerate all nodes in a DFS order (root first).
|
||||
/// For each node, we store the `Path` (list of child indices).
|
||||
pub fn enum_nodes_dfs(ast: &UntypedAst) -> Vec<Path> {
|
||||
let mut paths = Vec::new();
|
||||
dfs_helper(ast, &mut paths, &mut vec![]);
|
||||
paths
|
||||
}
|
||||
|
||||
/// Recursive helper: `current_path` is a stack we push child indices onto.
|
||||
/// `paths` is the global list of final results.
|
||||
fn dfs_helper(ast: &UntypedAst, paths: &mut Vec<Path>, current_path: &mut Vec<usize>) {
|
||||
// Record the current path as a valid node
|
||||
paths.push(current_path.clone());
|
||||
|
||||
match ast {
|
||||
UntypedAst::Sublist(children) => {
|
||||
// For each child, push its index, recurse, then pop
|
||||
for (i, child) in children.iter().enumerate() {
|
||||
current_path.push(i);
|
||||
dfs_helper(child, paths, current_path);
|
||||
current_path.pop();
|
||||
}
|
||||
}
|
||||
UntypedAst::IntLiteral(_) | UntypedAst::Instruction(_) => {
|
||||
// Leaf node => no children
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Replace the node at `path` in `original` with `replacement`, returning a new AST.
|
||||
/// If `path` is empty => we replace the root entirely.
|
||||
pub fn replace_subtree(
|
||||
original: &UntypedAst,
|
||||
path: &[usize],
|
||||
replacement: UntypedAst
|
||||
) -> UntypedAst {
|
||||
if path.is_empty() {
|
||||
// means we’re replacing the root
|
||||
return replacement;
|
||||
}
|
||||
// otherwise, we need to walk the sublist chain
|
||||
match original {
|
||||
UntypedAst::Sublist(children) => {
|
||||
// let path_head = path[0], path_tail = path[1..]
|
||||
let (first_idx, tail_path) = (path[0], &path[1..]);
|
||||
let mut new_children = children.clone();
|
||||
new_children[first_idx] = replace_subtree(&children[first_idx], tail_path, replacement);
|
||||
UntypedAst::Sublist(new_children)
|
||||
}
|
||||
// If the path is not empty but we’re not in a Sublist => error or do nothing
|
||||
UntypedAst::IntLiteral(_) | UntypedAst::Instruction(_) => {
|
||||
// Possibly, if the path is invalid, we ignore. Or we treat it as "replace anyway."
|
||||
// Here, we might treat it as an error or a no-op.
|
||||
original.clone()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Produces a new AST by choosing exactly one node in `original` at random
|
||||
/// (by enumerating them), and replacing it with a new random subtree.
|
||||
pub fn mutate_by_index(
|
||||
original: &UntypedAst,
|
||||
rng: &mut impl Rng,
|
||||
max_points: usize,
|
||||
) -> UntypedAst {
|
||||
// 1) Enumerate all nodes => get a vector of `Path`
|
||||
let all_paths = enum_nodes_dfs(original);
|
||||
// pick a random path
|
||||
let idx = rng.gen_range(0..all_paths.len());
|
||||
let chosen_path = &all_paths[idx];
|
||||
|
||||
// 2) Generate a new subtree with our "fixed" random code generator
|
||||
let new_subtree = ranmdom_code_fixed(rng, max_points);
|
||||
|
||||
// 3) Replace the subtree at `chosen_path` in `original` with `new_subtree`
|
||||
replace_subtree(original, chosen_path, new_subtree)
|
||||
}
|
||||
|
||||
/// Return the subtree of `original` at `path`,
|
||||
/// cloning it as a `UntypedAst`.
|
||||
/// If `path` is empty => returns the entire `original`.
|
||||
pub fn get_subtree(original: &UntypedAst, path: &[usize]) -> UntypedAst {
|
||||
if path.is_empty() {
|
||||
// The entire AST is the subtree
|
||||
return original.clone();
|
||||
}
|
||||
|
||||
match original {
|
||||
UntypedAst::Sublist(children) => {
|
||||
let (first_idx, tail_path) = (path[0], &path[1..]);
|
||||
if first_idx >= children.len() {
|
||||
// If path index is out of range, just clone the entire AST or
|
||||
// do something fallback
|
||||
return original.clone();
|
||||
}
|
||||
get_subtree(&children[first_idx], tail_path)
|
||||
}
|
||||
UntypedAst::IntLiteral(_) | UntypedAst::Instruction(_) => {
|
||||
// If path is non-empty but this is a leaf,
|
||||
// fallback: just return the leaf.
|
||||
original.clone()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Perform a subtree crossover between two ASTs.
|
||||
/// 1) We pick a random node in `a` and a random node in `b`,
|
||||
/// 2) We swap those subtrees,
|
||||
/// 3) Return the two new ASTs.
|
||||
///
|
||||
/// Example usage:
|
||||
/// ```rust
|
||||
/// let (childA, childB) = crossover_by_index(&parentA, &parentB, &mut rng);
|
||||
/// ```
|
||||
pub fn crossover_by_index(
|
||||
a: &UntypedAst,
|
||||
b: &UntypedAst,
|
||||
rng: &mut impl Rng,
|
||||
) -> (UntypedAst, UntypedAst) {
|
||||
// 1) enumerate nodes in a
|
||||
let paths_a = enum_nodes_dfs(a);
|
||||
let idx_a = rng.gen_range(0..paths_a.len());
|
||||
let chosen_a = &paths_a[idx_a];
|
||||
|
||||
// 2) enumerate nodes in b
|
||||
let paths_b = enum_nodes_dfs(b);
|
||||
let idx_b = rng.gen_range(0..paths_b.len());
|
||||
let chosen_b = &paths_b[idx_b];
|
||||
|
||||
// 3) get subtree from a, from b
|
||||
let subtree_a = get_subtree(a, chosen_a);
|
||||
let subtree_b = get_subtree(b, chosen_b);
|
||||
|
||||
// 4) replace them
|
||||
let new_a = replace_subtree(a, chosen_a, subtree_b);
|
||||
let new_b = replace_subtree(b, chosen_b, subtree_a);
|
||||
|
||||
(new_a, new_b)
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Individual {
|
||||
pub fitness: f64,
|
||||
// your Expr or bytecode or both
|
||||
}
|
||||
|
||||
pub struct Population {
|
||||
pub individuals: Vec<Individual>,
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
//! src/helpers/artifact.rs
|
||||
//! A small helper module to read a Forge artifact JSON and extract the creation code.
|
||||
|
||||
use anyhow::{anyhow, bail, Result};
|
||||
use serde::Deserialize;
|
||||
use std::fs;
|
||||
|
||||
/// This mirrors the JSON structure: "bytecode": { "object": "...hex..." }
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct BytecodeObject {
|
||||
object: String,
|
||||
}
|
||||
|
||||
/// The top-level artifact with a "bytecode" field.
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct MyContractArtifact {
|
||||
bytecode: BytecodeObject,
|
||||
}
|
||||
|
||||
/// Reads the given JSON file (a Forge artifact) and returns the raw creation code bytes.
|
||||
///
|
||||
/// * `filename`: path to the artifact JSON (e.g. `../onchain/out/Push3Interpreter.sol/Push3Interpreter.json`)
|
||||
pub fn get_creation_code(filename: &str) -> Result<Vec<u8>> {
|
||||
// 1) Read the JSON file
|
||||
let creation_json = fs::read_to_string(filename)
|
||||
.map_err(|e| anyhow!("Failed to read JSON file {}: {}", filename, e))?;
|
||||
|
||||
// 2) Parse
|
||||
let contract_artifact: MyContractArtifact = serde_json::from_str(&creation_json)
|
||||
.map_err(|e| anyhow!("Failed to parse JSON artifact: {}", e))?;
|
||||
|
||||
// 3) Extract the hex code
|
||||
let raw_hex = &contract_artifact.bytecode.object;
|
||||
let stripped_hex = raw_hex.trim_start_matches("0x");
|
||||
|
||||
// 4) Convert from hex => bytes
|
||||
match hex::decode(stripped_hex) {
|
||||
Ok(bytes) => Ok(bytes),
|
||||
Err(e) => bail!("Invalid hex for creation code: {e}"),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
pub mod artifact;
|
||||
@@ -0,0 +1,4 @@
|
||||
pub mod compiler;
|
||||
pub mod runner;
|
||||
pub mod gp;
|
||||
pub mod helpers;
|
||||
@@ -0,0 +1,118 @@
|
||||
// src/main.rs
|
||||
use std::env;
|
||||
|
||||
// We’ll use your existing AST definitions for parsing
|
||||
use offchain::compiler::ast::{
|
||||
parse_string_to_sexpr,
|
||||
sexpr_to_untyped,
|
||||
Push3Ast, // The trait
|
||||
UntypedAst, // The untyped AST enum
|
||||
};
|
||||
|
||||
use offchain::runner::revm_runner::{EvmRunner, Push3InterpreterOutputs}; // <--- We import our EvmRunner
|
||||
|
||||
use anyhow::{anyhow, bail, Result};
|
||||
use hex;
|
||||
use serde::Deserialize; // For JSON artifact loading
|
||||
|
||||
// If you still need these (not strictly required anymore):
|
||||
// use database::CacheDB;
|
||||
// use revm::{
|
||||
// context::Context,
|
||||
// context_interface::result::{ExecutionResult, Output},
|
||||
// database_interface::EmptyDB,
|
||||
// handler::EthHandler,
|
||||
// primitives::{Bytes, TxKind},
|
||||
// EvmCommit, MainEvm,
|
||||
// };
|
||||
use ethers::abi::{decode, ParamType};
|
||||
use ethers::types::U256;
|
||||
|
||||
// ----------------------------------------------------------------------
|
||||
// JSON artifact struct, as before
|
||||
// ----------------------------------------------------------------------
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct BytecodeObject {
|
||||
object: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct MyContractArtifact {
|
||||
bytecode: BytecodeObject,
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------
|
||||
// MAIN
|
||||
// ----------------------------------------------------------------------
|
||||
fn main() -> Result<()> {
|
||||
// 1) Read CLI arguments for the Push3 program
|
||||
let args: Vec<String> = env::args().collect();
|
||||
if args.len() < 2 {
|
||||
eprintln!("Usage: cargo run -- '<program>'");
|
||||
eprintln!("Example: cargo run -- '((3 5 +) DUP MUL)'");
|
||||
std::process::exit(1);
|
||||
}
|
||||
let program_str = &args[1];
|
||||
|
||||
// 2) Parse the string into an S-expression
|
||||
let sexpr = parse_string_to_sexpr(program_str)
|
||||
.map_err(|e| anyhow!("Error parsing S-expression: {}", e))?;
|
||||
|
||||
// 3) Convert SExpr => UntypedAst
|
||||
let ast: UntypedAst = sexpr_to_untyped(&sexpr)
|
||||
.map_err(|e| anyhow!("Error converting to UntypedAst: {}", e))?;
|
||||
|
||||
// 4) Convert UntypedAst => Push3 bytecode (for debugging, so we can see it)
|
||||
let bytecode = ast.to_bytecode();
|
||||
|
||||
// 5) Print the resulting AST and bytecode (in hex)
|
||||
println!("AST: {:?}", ast);
|
||||
println!("Bytecode (hex): 0x{}", hex::encode(&bytecode));
|
||||
|
||||
// ----------------------------------------------------------------------
|
||||
// Load the creation code (bytecode) from a JSON artifact
|
||||
// ----------------------------------------------------------------------
|
||||
let creation_hex_filename = "../onchain/out/Push3Interpreter.sol/Push3Interpreter.json";
|
||||
let creation_json = std::fs::read_to_string(creation_hex_filename)
|
||||
.map_err(|e| anyhow!("Failed to read JSON file {}: {}", creation_hex_filename, e))?;
|
||||
let contract_artifact: MyContractArtifact = serde_json::from_str(&creation_json)
|
||||
.map_err(|e| anyhow!("Failed to parse JSON artifact: {}", e))?;
|
||||
|
||||
let raw_hex = &contract_artifact.bytecode.object;
|
||||
let stripped_hex = raw_hex.trim_start_matches("0x");
|
||||
|
||||
let creation_bytes = match hex::decode(stripped_hex) {
|
||||
Ok(bytes) => bytes,
|
||||
Err(e) => bail!("Invalid hex for creation code: {e}"),
|
||||
};
|
||||
|
||||
println!(
|
||||
"Loaded creation code ({} bytes) from {}",
|
||||
creation_bytes.len(),
|
||||
creation_hex_filename
|
||||
);
|
||||
|
||||
// ----------------------------------------------------------------------
|
||||
// 6) Deploy the interpreter in an ephemeral EVM using `EvmRunner`
|
||||
// ----------------------------------------------------------------------
|
||||
let mut runner = EvmRunner::new(creation_bytes)?;
|
||||
println!("Interpreter deployed successfully in ephemeral EVM at: 0x{:x}", runner.interpreter_addr);
|
||||
|
||||
// ----------------------------------------------------------------------
|
||||
// 7) Evaluate the AST in our EVM
|
||||
// ----------------------------------------------------------------------
|
||||
// We'll call `runner.run_ast(&ast)`, which:
|
||||
// - Converts AST => push3 code,
|
||||
// - Builds a sublist descriptor in the exec stack,
|
||||
// - Calls `runInterpreter`,
|
||||
// - Returns final stacks in a `Push3InterpreterOutputs`.
|
||||
let outputs = runner.run_ast(&ast)?;
|
||||
println!("Ran the AST successfully!");
|
||||
|
||||
// 8) Print the final stacks
|
||||
println!("Final CODE stack: {:?}", outputs.final_code_stack);
|
||||
println!("Final EXEC stack: {:?}", outputs.final_exec_stack);
|
||||
println!("Final INT stack: {:?}", outputs.final_int_stack);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
pub mod revm_runner;
|
||||
@@ -0,0 +1,232 @@
|
||||
//! src/evm_integration/revm_runner.rs
|
||||
//!
|
||||
//! This module provides a thin wrapper around REVM to deploy a Push3Interpreter
|
||||
//! contract, then call its `runInterpreter(...)` function with flexible inputs/outputs,
|
||||
//! using the same style (Context::builder(), .modify_tx_chained, etc.) you had in your main.
|
||||
|
||||
use anyhow::{anyhow, bail, Result};
|
||||
use ethers::abi::{encode, decode, Token, ParamType};
|
||||
use ethers::types::U256;
|
||||
use ethers::utils;
|
||||
use database::CacheDB;
|
||||
use revm::{
|
||||
context::Context,
|
||||
context::{BlockEnv, TxEnv, CfgEnv},
|
||||
context_interface::result::{ExecutionResult, Output},
|
||||
database_interface::EmptyDB,
|
||||
handler::EthHandler,
|
||||
primitives::{Bytes, TxKind}, // 4 generics
|
||||
EvmCommit,
|
||||
MainEvm,
|
||||
};
|
||||
|
||||
// We import your AST definitions so we can call `ast.to_bytecode()`.
|
||||
use crate::compiler::ast::{UntypedAst, Push3Ast};
|
||||
|
||||
// If you have a descriptor helper (like make_sublist_descriptor), bring it in:
|
||||
use crate::compiler::push3_describtor::make_sublist_descriptor;
|
||||
|
||||
/// The input parameters for `runInterpreter(...)`: four fields (code, codeStack, execStack, intStack).
|
||||
pub struct Push3InterpreterInputs {
|
||||
pub code: Vec<u8>,
|
||||
pub init_code_stack: Vec<U256>,
|
||||
pub init_exec_stack: Vec<U256>,
|
||||
pub init_int_stack: Vec<i128>,
|
||||
}
|
||||
|
||||
/// The outputs from `runInterpreter(...)`: three arrays for code/exec/int stacks.
|
||||
pub struct Push3InterpreterOutputs {
|
||||
pub final_code_stack: Vec<U256>,
|
||||
pub final_exec_stack: Vec<U256>,
|
||||
pub final_int_stack: Vec<i128>,
|
||||
}
|
||||
|
||||
/// A thin wrapper around REVM, parameterized by the 4 generics (DB, BLOCK, TX, CFG).
|
||||
/// - We store the ephemeral EVM instance,
|
||||
/// - We store the deployed address of your `Push3Interpreter`,
|
||||
/// - We provide `run_interpreter` and `run_ast`.
|
||||
pub struct EvmRunner {
|
||||
/// The 4-parameter MainEvm:
|
||||
/// DB = CacheDB<EmptyDB>,
|
||||
/// BLOCK = BlockEnv,
|
||||
/// TX = TxEnv,
|
||||
/// CFG = CfgEnv
|
||||
pub evm: MainEvm<CacheDB<EmptyDB>, BlockEnv, TxEnv, CfgEnv>,
|
||||
|
||||
/// The address where Push3Interpreter was deployed.
|
||||
pub interpreter_addr: revm::primitives::Address,
|
||||
}
|
||||
|
||||
impl EvmRunner {
|
||||
/// Deploy a new ephemeral EVM with a "Create" transaction for the given `creation_code`.
|
||||
///
|
||||
/// This matches your older style of:
|
||||
/// `MainEvm::new(Context::builder()... .with_db(...), EthHandler::default())`
|
||||
pub fn new(creation_code: Vec<u8>) -> Result<Self> {
|
||||
// 1) Create the EVM using your old style: `Context::builder()...`
|
||||
// Then pass to MainEvm::new(...).
|
||||
// The difference is that we explicitly say MainEvm<DB,BLOCK,TX,CFG>.
|
||||
let mut evm = MainEvm::new(
|
||||
// a) Prepare context with a chained transaction to CREATE
|
||||
Context::builder()
|
||||
.modify_tx_chained(|tx| {
|
||||
tx.transact_to = TxKind::Create;
|
||||
tx.data = Bytes::from(creation_code.clone());
|
||||
})
|
||||
// b) Provide ephemeral DB
|
||||
.with_db(CacheDB::<EmptyDB>::default()),
|
||||
// c) The EthHandler for block/tx environment logic
|
||||
EthHandler::default(),
|
||||
);
|
||||
|
||||
// 2) Execute the CREATE transaction
|
||||
let creation_result = evm.exec_commit()?;
|
||||
let ExecutionResult::Success {
|
||||
output: Output::Create(_, Some(deployed_addr)),
|
||||
..
|
||||
} = creation_result
|
||||
else {
|
||||
bail!("Interpreter deployment failed or no address returned: {creation_result:#?}");
|
||||
};
|
||||
|
||||
// 3) Return the EvmRunner
|
||||
Ok(Self {
|
||||
evm,
|
||||
interpreter_addr: deployed_addr,
|
||||
})
|
||||
}
|
||||
|
||||
/// Call `runInterpreter(bytes,uint256[],uint256[],int256[])` on the deployed contract,
|
||||
/// returning the final code/exec/int stacks.
|
||||
pub fn run_interpreter(
|
||||
&mut self,
|
||||
inputs: &Push3InterpreterInputs
|
||||
) -> Result<Push3InterpreterOutputs> {
|
||||
// 1) Build function selector
|
||||
let func_selector = &utils::id("runInterpreter(bytes,uint256[],uint256[],int256[])")[0..4];
|
||||
|
||||
// 2) Convert each field to `ethers::abi::Token`
|
||||
let code_token = Token::Bytes(inputs.code.clone());
|
||||
let init_code_stack = Token::Array(
|
||||
inputs.init_code_stack
|
||||
.iter()
|
||||
.map(|&u| Token::Uint(u))
|
||||
.collect()
|
||||
);
|
||||
let init_exec_stack = Token::Array(
|
||||
inputs.init_exec_stack
|
||||
.iter()
|
||||
.map(|&u| Token::Uint(u))
|
||||
.collect()
|
||||
);
|
||||
let init_int_stack = Token::Array(
|
||||
inputs.init_int_stack
|
||||
.iter()
|
||||
.map(|&val_i128| {
|
||||
// store i128 as a U256. If you need negative values, do a two’s complement approach.
|
||||
let as_u128 = val_i128 as u128;
|
||||
Token::Int(U256::from(as_u128))
|
||||
})
|
||||
.collect()
|
||||
);
|
||||
|
||||
// 3) Encode
|
||||
let encoded_args = encode(&[
|
||||
code_token,
|
||||
init_code_stack,
|
||||
init_exec_stack,
|
||||
init_int_stack,
|
||||
]);
|
||||
|
||||
// 4) Build final call data
|
||||
let mut call_data = Vec::from(func_selector);
|
||||
call_data.extend_from_slice(&encoded_args);
|
||||
|
||||
// 5) Modify the transaction to CALL the deployed interpreter
|
||||
self.evm.context.modify_tx(|tx| {
|
||||
tx.transact_to = TxKind::Call(self.interpreter_addr);
|
||||
tx.data = Bytes::from(call_data);
|
||||
tx.nonce = 1; // increment nonce to avoid reuse
|
||||
});
|
||||
|
||||
// 6) Execute the call
|
||||
let call_result = self.evm.transact()?;
|
||||
match &call_result.result {
|
||||
ExecutionResult::Success {
|
||||
output: Output::Call(return_data),
|
||||
..
|
||||
} => {
|
||||
// 7) Decode (uint256[], uint256[], int256[])
|
||||
let param_types = &[
|
||||
ParamType::Array(Box::new(ParamType::Uint(256))), // finalCodeStack
|
||||
ParamType::Array(Box::new(ParamType::Uint(256))), // finalExecStack
|
||||
ParamType::Array(Box::new(ParamType::Int(256))), // finalIntStack
|
||||
];
|
||||
let decoded = decode(param_types, return_data)
|
||||
.map_err(|e| anyhow!("Failed to decode return data: {e}"))?;
|
||||
|
||||
// parse each array
|
||||
let final_code_stack = match &decoded[0] {
|
||||
Token::Array(arr) => arr.iter().filter_map(|t| {
|
||||
if let Token::Uint(u) = t { Some(*u) } else { None }
|
||||
}).collect(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
let final_exec_stack = match &decoded[1] {
|
||||
Token::Array(arr) => arr.iter().filter_map(|t| {
|
||||
if let Token::Uint(u) = t { Some(*u) } else { None }
|
||||
}).collect(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
let final_int_stack = match &decoded[2] {
|
||||
Token::Array(arr) => arr.iter().filter_map(|t| {
|
||||
if let Token::Int(u256_val) = t {
|
||||
// read only the lower 128 bits => ignoring sign extension
|
||||
let lo = u256_val.low_u128();
|
||||
Some(lo as i128)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}).collect(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
|
||||
Ok(Push3InterpreterOutputs {
|
||||
final_code_stack,
|
||||
final_exec_stack,
|
||||
final_int_stack,
|
||||
})
|
||||
}
|
||||
ExecutionResult::Revert { gas_used, output } => {
|
||||
bail!("Call reverted: gas used={gas_used:?}, output={output:?}")
|
||||
}
|
||||
other => {
|
||||
bail!("Call failed: {other:?}")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A convenience method to run an `UntypedAst`:
|
||||
/// - Convert AST => push3 code,
|
||||
/// - Build a sublist descriptor in the exec stack,
|
||||
/// - Call `run_interpreter`.
|
||||
pub fn run_ast(&mut self, ast: &UntypedAst) -> Result<Push3InterpreterOutputs> {
|
||||
// 1) Convert AST => push3 bytecode
|
||||
let code_bytes = ast.to_bytecode();
|
||||
let code_len = code_bytes.len() as u32;
|
||||
|
||||
// 2) Build a sublist descriptor
|
||||
let descriptor = make_sublist_descriptor(0, code_len);
|
||||
|
||||
// 3) Put code in `code` and sublist descriptor in `execStack`
|
||||
let inputs = Push3InterpreterInputs {
|
||||
code: code_bytes,
|
||||
init_code_stack: Vec::new(),
|
||||
init_exec_stack: vec![descriptor],
|
||||
init_int_stack: Vec::new(),
|
||||
};
|
||||
|
||||
// 4) Run interpreter
|
||||
self.run_interpreter(&inputs)
|
||||
}
|
||||
}
|
||||
@@ -43,7 +43,7 @@ contract Push3InterpreterTest is Test {
|
||||
// [9..12] = 0x00000020 => 32
|
||||
// [13] = 0x01 => plus
|
||||
|
||||
bytes memory code = hex"03000B020000000A020000002001";
|
||||
bytes memory code = hex"03000B020000000A020000002001";
|
||||
|
||||
// Or fully concatenated:
|
||||
// 0x03 00 0B 02 00 00 00 0A 02 00 00 00 20 01
|
||||
|
||||
@@ -1,202 +1,53 @@
|
||||
# Sovraigns: Verifiable Optimization for DeFi Protocols
|
||||
# SoLUSH: Running Intelligence On-Chain
|
||||
|
||||
## The Problem
|
||||
In decentralized finance, systems often involve tasks such as determining optimal liquidity ranges for automated market makers, allocating staking rewards in a way that balances fairness and participation incentives, or dynamically adjusting collateralization ratios to minimize risks in lending protocols. These challenges are critical for ensuring the efficiency and stability of protocols, yet they are particularly difficult to address without centralization. Off-chain solutions often rely on external computation and centralized entities or oracles, introducing trust dependencies and undermining the decentralized ethos of blockchain systems.
|
||||
|
||||
DeFi protocols lose approximately $1.5B annually through inefficient parameter settings. Uniswap V3 liquidity providers lose ~$400M from suboptimal position ranges. Lending protocols forfeit ~$300M by maintaining overly conservative collateral ratios. MEV extraction costs protocols ~$800M due to predictable parameter updates.
|
||||
Artificial intelligence is increasingly being explored in decentralized finance to enhance token ecosystems and liquidity management. This shift represents a practical application of AI tools aimed at addressing complex optimization problems directly within blockchain ecosystems. While large language models (LLMs) dominate discussions around AI innovation due to their general-purpose capabilities, they are poorly suited to the demands of decentralized adaptive systems. In contrast, **genetic algorithms** (GAs) offer a more natural and efficient fit for systems requiring iterative improvements, combining computational simplicity with adaptive behavior.
|
||||
|
||||
Protocols need optimization algorithms, but current approaches require trust:
|
||||
## **Introduction to Genetic Algorithms**
|
||||
|
||||
- External consultants (Gauntlet, Chaos Labs) operate as black boxes with enterprise-only access
|
||||
- AI services require sharing sensitive protocol data with third parties
|
||||
- On-chain execution is prohibitively expensive - a simple neural network would cost millions in gas
|
||||
Genetic algorithms (GAs) are optimization techniques inspired by the process of natural selection. They operate by evolving a population of candidate solutions over multiple generations, using mechanisms such as mutation, crossover, and selection.
|
||||
|
||||
Protocols must choose between inefficiency or centralization.
|
||||
- **Population:** A set of potential solutions to a problem.
|
||||
- **Fitness Function:** A way to evaluate how well each solution performs.
|
||||
- **Selection:** The process of choosing the best-performing solutions to carry forward.
|
||||
- **Crossover and Mutation:** Methods to combine and modify solutions, creating new ones that explore the problem space.
|
||||
|
||||
## Our Approach
|
||||
Through repeated iterations, GAs converge toward optimal or near-optimal solutions, making them highly effective for solving complex, dynamic problems. GAs have recently gained momentum with the development of the [Push 3 language](http://faculty.hampshire.edu/lspector/push3-description.html) by Lee Spector. Push 3’s compact and efficient architecture shares similarities with stack-based virtual machine systems like the Ethereum Virtual Machine (EVM). Although not designed specifically for EVMs, Push 3’s simplicity and flexibility make it conceptually compatible with blockchain execution environments. Push 3 has achieved milestones such as evolving [programs for quantum computers](https://faculty.hampshire.edu/lspector/aqcp/) and supporting autoconstructive evolution. A Push 3 VM could facilitate the deployment of genetic algorithms in decentralized systems, leveraging Push 3’s support for expressive program evolution and adaptive behavior in tasks like resource optimization.
|
||||
|
||||
Sovraigns enables verifiable optimization through genetic algorithms, zero-knowledge proofs, and based rollups:
|
||||
## On Agents
|
||||
|
||||
1. Algorithms evolve off-chain through competition between optimizers
|
||||
2. Zero-knowledge proofs verify correct execution without revealing the algorithm
|
||||
3. A based rollup coordinates requests and stores proven results
|
||||
An agent is defined as an entity capable of perceiving its environment, processing information, and taking autonomous actions to achieve specific objectives. LLMs can be structured into agents using frameworks that provide memory and feedback loops, but they do not fit on-chain and, at the same time, cannot securely manage secrets like private keys. Without either of these capabilities, LLM-based agents require external key management, increasing risks and undermining decentralized security principles. In contrast, genetic algorithms exhibit agentic characteristics by evolving their behavior over time to meet specific goals. When deployed on-chain within smart contracts, GAs can autonomously adapt to real-time inputs and optimize decisions, all while operating entirely within the secure and deterministic framework of blockchain systems.
|
||||
|
||||
Protocols receive optimized parameters with cryptographic proof of correctness, without trusting any party.
|
||||
---
|
||||
|
||||
## Market Size
|
||||
## **Genetic Algorithms vs. LLMs for Decentralized Systems**
|
||||
|
||||
Current DeFi inefficiencies:
|
||||
1. **Efficiency and Feasibility for Blockchain Execution**
|
||||
|
||||
| **DeFi Inefficiency** | **Annual Value Lost** | **Root Cause** |
|
||||
|---|---|---|
|
||||
| Uniswap V3 LP Losses | ~$400M | Suboptimal position management |
|
||||
| Lending Protocol Inefficiency | ~$300M | Conservative parameters |
|
||||
| MEV/Arbitrage Extraction | ~$800M | Predictable protocol behavior |
|
||||
| **Total Market** | **$1.5B+** | **Lack of intelligent optimization** |
|
||||
- **LLMs are resource-intensive:** The computational complexity and size of LLMs make them impractical for direct execution in decentralized environments, where every operation incurs costs. Running even the simplest inference for an LLM would require off-chain computation and trust in external oracles, undermining decentralization.
|
||||
- **GAs are lightweight:** Genetic algorithms, by design, are computationally efficient. Their iterative, population-based approach to optimization fits naturally into resource-constrained environments. This efficiency ensures GAs can evolve and execute entirely on-chain, preserving transparency and trustless execution.
|
||||
|
||||
With DeFi TVL at $50B, we estimate the optimization market at $2-3B annually today, potentially reaching $10B as protocols become more complex.
|
||||
2. **Evolution vs. Training**
|
||||
|
||||
### Current Solutions vs. Sovraigns
|
||||
- **GAs evolve in parallel:** Genetic algorithms enable evolutionary processes to run across distributed populations simultaneously. This parallelism contrasts with the centralized training required for deep neural networks, which demand synchronized computation and significant infrastructure. GAs’ decentralized nature aligns well with blockchain environments.
|
||||
|
||||
| **Aspect** | **Current Solutions** | **Sovraigns** |
|
||||
|---|---|---|
|
||||
| **Trust Model** | Centralized (Gauntlet, Chaos Labs) | Trustless ZK proofs |
|
||||
| **Accessibility** | Enterprise contracts only | Permissionless for all protocols |
|
||||
| **Transparency** | Black box optimization | Fully auditable algorithms |
|
||||
| **Cost** | High consulting fees | Pay-per-optimization |
|
||||
| **Adaptability** | Manual updates | Continuous evolution |
|
||||
| **Security** | Single point of failure | Decentralized validation |
|
||||
- **Deep neural networks require centralized training:** Neural networks demand substantial computational resources and centralized coordination to train effectively.
|
||||
|
||||
3. **Auditability of Evolution vs Training**
|
||||
|
||||
## Why Genetic Algorithms Over LLMs
|
||||
- **Training LLMs is opaque:** The progress of training deep neural networks is often difficult to evaluate and verify because testing its capabilities relies on secret data, which cannot be audited by external parties.
|
||||
- **GAs offer transparent evolution:** Genetic algorithms allow the progress of evolution to be easily audited against a target fitness function. This process is fully observable and can be verified at each step with minimal effort.
|
||||
|
||||
Technical comparison for decentralized systems:
|
||||
---
|
||||
|
||||
| **Aspect** | **LLMs** | **Genetic Algorithms** |
|
||||
|---|---|---|
|
||||
| **Computational Requirements** | GPUs, massive memory, centralized inference | Lightweight, runs anywhere |
|
||||
| **Blockchain Compatibility** | Impossible on-chain, requires external APIs | Efficient ZK-proof verification |
|
||||
| **Trust Model** | Black box, external dependencies | Fully verifiable with cryptographic proofs |
|
||||
| **Cost** | $1000s per complex optimization | Pennies per optimization |
|
||||
| **Adaptability** | Fixed after training | Continuously evolves to market conditions |
|
||||
| **Determinism** | Non-deterministic, unreliable | Deterministic, predictable outputs |
|
||||
| **Decentralization** | Requires centralized providers | Fully decentralized execution |
|
||||
## **Theoretical Framework for Decentralized Algorithm Evolution**
|
||||
|
||||
Genetic algorithms provide the optimization capabilities of LLMs while remaining verifiable, deterministic, and cost-effective for blockchain environments.
|
||||
A protocol providing agents as a service could leverage genetic algorithms to address complex problems within blockchain environments. Read through this [real-life usecase explaining the applicability of agents to DeFi](example.md). Central to such a system would be the development of a Push 3 Virtual Machine (VM), enabling efficient execution of genetic algorithms. An "Optimizer Pool", an network of nodes could collaboratively train and evolve these algorithms offchain. These nodes would compete to improve the fitness of the algorithms. Once an improvement, a measuruable better solution to the fitness function is found in off-chain computation, a node would submit it to be accepted into the population of solutions registered on chain, in exchange for a rewards payment by the protocol. The nodes of the optimizer pool would run under an incentivization model simira to proof-of-work systems.
|
||||
|
||||
## Technical Architecture
|
||||
|
||||
Three core components enable verifiable optimization on Ethereum:
|
||||
|
||||
1. **Based Rollup Coordination**
|
||||
- Manages algorithm registry and optimization requests
|
||||
- Batches proofs for cost-effective verification
|
||||
- Inherits Ethereum's security model
|
||||
|
||||
2. **OpenVM Execution Environment**
|
||||
- Runs genetic algorithms in a provable VM
|
||||
- Generates STARK proofs for each optimization
|
||||
- Ensures deterministic execution
|
||||
|
||||
3. **Optimizer Network**
|
||||
- Independent nodes compete to evolve better algorithms
|
||||
- Council validates quality before on-chain registration
|
||||
- Economic incentives tied to algorithm performance
|
||||
|
||||
### Architecture Overview
|
||||
|
||||
The protocol operates across three layers:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ETHEREUM L1 │
|
||||
│ • Final Settlement • Proof Verification • Protocol Updates │
|
||||
└────────────────────────────┬───────────────────────────────────┘
|
||||
│ State Bridge
|
||||
┌────────────────────────────┴───────────────────────────────────┐
|
||||
│ BASED ROLLUP (Sidechain) │
|
||||
│ • Task Registry • Algorithm Storage • Coordination │
|
||||
│ • Reward Pools • Version History • Batch Requests │
|
||||
└────────────────────────────┬───────────────────────────────────┘
|
||||
│ ZK Proofs
|
||||
┌────────────────────────────┴───────────────────────────────────┐
|
||||
│ OFF-CHAIN COMPUTE │
|
||||
│ • Algorithm Evolution • OpenVM Execution • Proof Generation │
|
||||
│ • Council Validation • Optimizer Competition │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
| **Layer** | **Purpose** | **Key Components** |
|
||||
|---|---|---|
|
||||
| **Ethereum L1** | Final settlement and security | Smart contracts, State verification |
|
||||
| **Based Rollup** | Decentralized coordination | Task registry, Algorithm storage |
|
||||
| **Off-Chain** | Computation and evolution | OpenVM, Genetic algorithms, ZK proofs |
|
||||
|
||||
### Based Rollup: Decentralized Coordination
|
||||
1. **Task Registration**: DeFi protocols register optimization tasks on the based rollup, including:
|
||||
- Fitness functions (optimization goals)
|
||||
- Evaluation parameters and constraints
|
||||
- Reward pools for successful optimizations
|
||||
|
||||
2. **Algorithm Registry**: The best algorithms for each task are registered on-chain after council validation
|
||||
- Algorithm hashes stored on-chain for verification
|
||||
- ZK proofs verify the exact algorithm (matching hash) was executed
|
||||
- Version history maintained for auditability
|
||||
- Permissionless access for all protocols
|
||||
|
||||
3. **Execution Requests**: When protocols need optimization:
|
||||
- Send request to based rollup with current data
|
||||
- Request triggers off-chain execution
|
||||
- Multiple requests batched for efficiency
|
||||
|
||||
### Off-Chain: Evolution and Zero-Knowledge Execution
|
||||
1. **Algorithm Evolution**:
|
||||
- Optimizers run genetic algorithms using standard Push 3 implementations
|
||||
- Evolution happens on local high-performance infrastructure
|
||||
- Best solutions submitted to council for validation
|
||||
- No ZK proofs needed during evolution phase
|
||||
|
||||
2. **Zero-Knowledge Proof Generation** (triggered by on-chain requests):
|
||||
- Registered algorithms execute in OpenVM with request data
|
||||
- OpenVM verifies the algorithm matches the on-chain hash
|
||||
- STARK proofs generated via Plonky3
|
||||
- Proofs include complete execution trace and results
|
||||
- Multiple proofs batched into rollup blocks
|
||||
|
||||
3. **Council Validation**:
|
||||
- Distributed validators verify algorithm improvements
|
||||
- Consensus required before on-chain registration
|
||||
- Ensures quality without centralized control
|
||||
|
||||
### L1 Integration: Trustless Protocol Updates
|
||||
1. **State Bridge**: Based rollup posts state roots to Ethereum
|
||||
2. **Proof Verification**: Smart contracts verify STARK proofs
|
||||
3. **Action Execution**: DeFi protocols execute optimizations based on verified results
|
||||
|
||||
### Key Components
|
||||
- **Registry (based rollup)**: Task and algorithm management with ZK verification
|
||||
- **OpenVM Runtime**: Custom Push 3 implementation for genetic programming
|
||||
- **Optimizers (off-chain)**: Compete to evolve better solutions
|
||||
- **Council (distributed)**: Validates algorithm quality before registration
|
||||
- **Proof Aggregator**: Batches multiple ZK proofs for efficient verification
|
||||
- **State Bridge**: Synchronizes rollup state with Ethereum L1
|
||||
|
||||
## Timing
|
||||
|
||||
Three technical developments make this feasible now:
|
||||
|
||||
1. **ZK Infrastructure**
|
||||
- STARK proving systems now production-ready
|
||||
- Proof generation costs decreased 100x since 2022
|
||||
- OpenVM provides suitable execution environment
|
||||
|
||||
2. **Based Rollups**
|
||||
- Ethereum's rollup infrastructure mature enough for production
|
||||
- Provides coordination without sacrificing decentralization
|
||||
- Cost structure makes continuous optimization viable
|
||||
|
||||
3. **Market Demand**
|
||||
- Protocols reaching limits of manual parameter tuning
|
||||
- Increased scrutiny requires auditable optimization
|
||||
- Competition driving need for efficiency gains
|
||||
|
||||
## Competitive Position
|
||||
|
||||
**Technical Differentiation:**
|
||||
- First implementation of ZK-verified genetic algorithms for DeFi
|
||||
- Custom VM optimized for this use case
|
||||
- Integration with Ethereum's rollup infrastructure
|
||||
|
||||
**Network Effects:**
|
||||
- More protocols using the system improves algorithm quality
|
||||
- Optimizer competition creates continuous improvement
|
||||
- Registry accumulates valuable optimization history
|
||||
|
||||
**Barriers to Entry:**
|
||||
- Requires expertise in genetic programming, zero-knowledge systems, and DeFi mechanics
|
||||
- Complex coordination between multiple system layers
|
||||
- Significant development time to reach production quality
|
||||
|
||||
We aim to become the standard optimization layer for DeFi protocols, addressing a multi-billion dollar inefficiency.
|
||||

|
||||
|
||||
To evaluate the effectiveness of distributed evolution, a "Council" mechanism needs to be created. This Council would consist of validators who execute the latest evolved algorithms and assess their performance against a predefined target fitness function. Validators would then cast votes on whether the new algorithms deliver measurable improvements. This transparent voting process ensures that only the most effective solutions are rewarded and integrated, fostering trust and collaboration across the Optimizer Pool. By aligning incentives with measurable progress, such a framework provides a reliable and decentralized method for advancing algorithmic evolution.
|
||||
|
||||
Ultimately, this system creates a foundation for scalable agent development, aligning with the broader concept of agents as a service, where users and protocols in DeFi can deploy and leverage optimized, evolving algorithms tailored to specific tasks.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user